Graph-Sparse Logistic Regression for sparse and connected support classification.
problem Sparse and connected support classification problems.
method Introduces Graph-Sparse Logistic Regression algorithm.
result Validated and benchmarked against L1-regularized Logistic Regression.
Deep neural network estimates support of sparse signals for improved phase retrieval.
problem Sparse phase retrieval from Fourier magnitudes with support estimation.
method Trained deep neural network (DNN) provides extended support estimate E larger than the support T. result DNN-based support estimation improves signal reconstruction performance with lower complexity.
Algorithm recovers sparse PCA support from incomplete data.
problem Sparse PCA with incomplete and noisy data.
method Semidefinite program (SDP) relaxation of non-convex l1-regularized PCA. result SDP enables exact recovery of true support of sparse leading eigenvector.
New algorithm recovers dictionaries with arbitrary supports in polynomial time.
problem Learning dictionaries with arbitrary supports in polynomial time.
method Semirandom model with a mix of arbitrary and random supports; polynomial time algorithm.
result Polynomial time recovery of incoherent over-complete dictionaries with arbitrary supports.
Study on recovering supports of multiple sparse vectors from mixed linear measurements.
problem Recovering supports of multiple sparse vectors from a mixture of linear measurements.
method Developed algorithms to identify the support of all component vectors using polynomial and quasi-polynomial number of measurements.
result Polynomial and quasi-polynomial number of measurements sufficient for recovering the supports of all component vectors.
Screening is an effective technique for speeding up the training process of a sparse learning model by removing the features that are guaranteed to be inactive the process. In this paper, we present a efficient screening technique for sparse support vector machine based on variational inequality. The technique is both …
This letter presents a novel Block Bayesian Hypothesis Testing Algorithm (Block-BHTA) for reconstructing block sparse signals with unknown block structures. The Block-BHTA comprises the detection and recovery of the supports, and the estimation of the amplitudes of the block sparse signal. The support detection and rec…
Sparse Gaussian processes with compact kernels for faster inference.
problem Efficient Gaussian process inference with high computational complexity.
method Parametric families of compactly-supported kernels for sparse matrix representations.
result Sub-quadratic inference complexity and improved performance on real-world tasks.
Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples a…
IRKSN algorithm achieves sparse recovery with wider applicability conditions.
problem Sparse recovery challenges due to NP-hard nature and restrictive conditions.
method IRKSN algorithm based on k-support norm regularizer. result Achieves sparse recovery with explicit constants and standard linear rate.
The paper analyzes sparse PCA for incomplete data and proves support recovery conditions.
problem Support recovery in sparse PCA with non-random missing data.
method Semidefinite relaxation of the ℓ1-regularized PCA problem. result Support of the sparse leading eigenvector can be recovered with high probability.
Paper proposes estimators for sparse PCA with oracle property.
problem Estimating sparse principal subspace in high-dimensional settings.
method Semidefinite relaxation with novel regularizations.
result One estimator achieves exact support recovery and statistical rate.
GRRT recovers sparse signals without prior sparsity or noise variance knowledge.
problem Recovering sparse signals without prior sparsity or noise variance knowledge.
method Generalized residual ratio thresholding (GRRT) for SOMP and BOMP.
result Finite sample and finite SNR guarantees for exact support recovery.
Efficient algorithms for sparse parameter recovery in mixture models.
problem Support recovery of high-dimensional sparse latent vectors in mixture models.
method Efficient algorithms with logarithmic sample complexity dependence on dimensionality.
result First guarantees on support recovery for various mixture models.
The Lasso performs well in ultra-sparse linear models with finite support size.
problem Performance analysis of Lasso in ultra-sparse linear models.
method Novel application of replica method from statistical physics, rigorous analysis of average case performance.
result Average performance of Lasso assessed without scaling assumptions, offering sample complexity bounds.
Study on limits of recovering sparse variables from phaseless measurements.
problem Support recovery in phase retrieval model with noisy phaseless measurements.
method Information-theoretic analysis, considering discrete and Gaussian models, Gaussian measurement matrices.
result Sharp thresholds with near-matching constant factors for sparsity and signal-to-noise ratio in various scaling regimes.
The paper examines properties of GW optimal transport plans, showing they can be sparse and permutation-supported.
problem Properties of Gromov-Wasserstein optimal transport plans.
method Exploration of sparsity, permutation support, and cyclical monotonicity properties.
result GW optimal plans can be sparse and permutation-supported under certain conditions.
We derive a novel norm that corresponds to the tightest convex relaxation of sparsity combined with an ℓ2 penalty. We show that this new {\em k-support norm} provides a tighter relaxation than the elastic net and is thus a good replacement for the Lasso or the elastic net in sparse prediction problems. Through …
Sparse representations improve reinforcement learning performance.
problem TD Learning struggles with large state spaces and simple control tasks.
method Learned sparse representations to reduce state space and support generalization.
result Sparse representations enhance reinforcement learning performance on challenging tasks.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
This paper proposes a new algorithm for multiple sparse regression in high dimensions, where the task is to estimate the support and values of several (typically related) sparse vectors from a few noisy linear measurements. Our algorithm is a "forward-backward" greedy procedure that -- uniquely -- operates on two disti…
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
Quantum SVM uses fewer features for faster training.
problem Training high-dimensional SVMs efficiently.
method Quantum linear programming for sparse SVM training.
result Quantum sparse SVM can be trained in sublinear time.
We consider high dimensional sparse regression, and develop strategies able to deal with arbitrary -- possibly, severe or coordinated -- errors in the covariance matrix X. These may come from corrupted data, persistent experimental errors, or malicious respondents in surveys/recommender systems, etc. Such non-stochas…
Paper introduces DP methods for high-dimensional variable selection.
problem Sparse variable selection in high-dimensional learning.
method Pure differentially private estimators using Integer Programming.
result Achieves state-of-the-art empirical support recovery.
Sparse-Gen uses generative models to improve compressed sensing with full signal recovery.
problem Recovering signals with fewer measurements than traditional methods allow.
method Sparse-Gen framework that allows for sparse deviations from the support set.
result Achieves full signal recovery over the full space of signals, not just the support.
New research challenges the idea that counterfactual explanations should be sparse.
problem Predictive multiplicity leads to multiple models giving almost equal solutions.
method Derive a general upper bound for counterfactual costs under multiplicity and compare sparse vs. data support approaches.
result Data support methods are more robust to multiplicity but have higher counterfactual costs.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
New method connects Sparse PCA and Sparse Linear Regression.
problem Sparse Principal Component Analysis and Sparse Linear Regression.
method Transforming a solver for Sparse Linear Regression into an algorithm for Sparse Principal Component Analysis.
result The derived SPCA algorithm achieves near state-of-the-art guarantees for testing and support recovery.
The paper efficiently estimates parameters from truncated Gaussian and linear models.
problem Estimating parameters from truncated Gaussian and linear models.
method Minimizes finite population negative log-likelihood function with an l1-regularization term.
result Efficient estimation of parameters from truncated samples.
In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e.g., random projection, random hashing), for large-scale high-dimensional classification. Previous theoretical results on randomized reduction methods hinge on strong assumptio…
New method speeds up sparse Gaussian processes for large datasets.
problem Efficiently modeling large datasets with many inducing variables.
method Projecting a GP onto B-spline basis functions for sparse linear algebra.
result Efficiently models fast-varying spatial phenomena with tens of thousands of inducing variables.
New design method improves Lasso performance in sparse regression.
problem Sparse linear regression with correlated design columns.
method Introduces partially-rotated designs to improve Lasso's RE constant.
result Lasso achieves better prediction error with high probability.
Sparse LR-LSSVM improves kernel machine performance.
problem Improving kernel machine performance with controlled model size.
method Introduces LR-LSSVM with low rank kernels and a two-step optimization algorithm.
result Proposed algorithm's performance is comparable or superior to existing kernel machines.
Gradient descent on autoencoders can solve dictionary learning problems under certain conditions.
problem Recovering incoherent matrices and sparse vectors from observations.
method Rigorous analysis of gradient descent on autoencoder loss function.
result Gradient descent can solve dictionary learning problems under mild assumptions.
This paper improves support recovery in universal one-bit compressed sensing with fewer measurements.
problem Support recovery in universal one-bit compressed sensing.
method Developed algorithms to recover the support of sparse signals with a small number of false positives.
result Support recovery with ildeO(k3/2) measurements, improving to ildeO(k) with known dynamic range. Distributed-OMP recovers sparse vectors with low communication costs.
problem High-dimensional sparse linear regression with limited computation and communication.
method Distributed orthogonal matching pursuit (OMP) scheme.
result Support of the regression vector can be recovered with linear communication per machine and logarithmic in dimension.
This letter proposes a low-computational Bayesian algorithm for noisy sparse recovery in the context of one bit compressed sensing with sensing matrix perturbation. The proposed algorithm which is called BHT-MLE comprises a sparse support detector and an amplitude estimator. The support detector utilizes Bayesian hypot…
Proposes novel wSVMs for sparse learning and accurate probability estimation.
problem Sparse features with redundant noise limit the performance of existing wSVMs.
method Develops ℓ1-norm and elastic net regularized wSVMs for automatic variable selection and probability estimation. result Elastic net regularized wSVMs achieve superior performance in variable selection and probability estimation.
Meta-learning improves support recovery in high-dimensional PCA.
problem Support recovery in high-dimensional Principal Component Analysis.
method Meta-learning approach to reduce sample complexity and support recovery.
result Support recovery can be achieved with significantly fewer samples than traditional methods.
New SVM model balances sparsity and robustness in noisy data.
problem Noise sensitivity and lack of sparsity in traditional SVM models.
method Combines elastic net loss with robust loss framework, integrates with SVM, uses half-quadratic algorithm.
result Proves sparsity and robustness, outperforms traditional SVMs in noisy environments.
msPCA solves sparse PCA for multiple components efficiently.
problem Sparse principal component analysis with multiple components.
method Alternating maximization algorithm for sparse loading vectors, with orthogonality or zero correlation constraints.
result Achieves high variance explained with sparse components and controlled feasibility violations.
Recent breakthrough results in compressive sensing (CS) have established that many high dimensional signals can be accurately recovered from a relatively small number of non-adaptive linear observations, provided that the signals possess a sparse representation in some basis. Subsequent efforts have shown that the perf…
Efficiently solves large-scale SVMs with sparse semismooth Newton method.
problem Numerical difficulties in solving large-scale SVMs.
method Sparse semismooth Newton based augmented Lagrangian method.
result Outperforms state-of-the-art solvers for large-scale SVMs.
The paper reduces sample complexity for estimating novel task parameters with few meta-learning tasks.
problem Meta-learning sparse linear regression with limited data.
method Accessing multiple similar tasks to recover common support and reduce novel task sample complexity.
result The sample complexity for estimating the parameter of a novel task is greatly reduced to O(1) with respect to the number of tasks.
Sparse attention model reduces long-context inference time with exponential accuracy guarantees.
problem Efficiently processing long-context queries in large language models.
method Formalizes attention as a projection onto key vectors, analyzes entropic relaxation, and introduces Vashista Sparse Attention.
result Sparse attention concentrates on a constant-size active face, leading to exponential decay of inactive tokens' mass and linear scaling of active face error.
This paper improves support recovery in universal one-bit compressed sensing.
problem Support recovery in one-bit compressed sensing for sparse signals.
method Proposes approximate support recovery and superset recovery algorithms with polynomial-time complexity.
result Achieves improved support recovery with fewer measurements compared to existing methods.
SuperMix uses sparse regularization to accurately estimate discrete mixing measures.
problem Estimating discrete mixing measures in kernel mixture models.
method Data fitting and regularization convex program with l1-regularization.
result The estimator accurately identifies the true mixing measure with support localization.