Method decomposes streaming data into sparse and low-rank components from compressive measurements.
problem Online decomposing compressive streaming data efficiently.
method Solves n n n - ℓ 1 \ell_1 ℓ 1 cluster-weighted minimization to decompose sparse and low-rank components. result Outperforms existing methods for numerical and video data.
Accelerates machine learning algorithms for sparse data.
problem Efficiently solving composite convex minimization problems.
method Accelerated dual-averaging primal-dual method for composite convex minimization.
result Demonstrates advantages in handling sparse data both theoretically and empirically.
Paper proposes an efficient algorithm for clustering with sparse feature selection.
problem Estimating labels and sparse weights in unsupervised clustering.
method Alternating minimization of Frobenius norm criterion with K-sparse algorithm.
result Significantly improves clustering results on single-cell RNA sequencing datasets.
Dual IHT algorithm solves NP-hard non-convex sparse minimization problems.
problem Non-convex sparse minimization with ℓ 2 \ell_2 ℓ 2 -regularized loss function. method Developed a dual IHT algorithm for maximizing the non-smooth dual objective.
result Sparse recovery performance is invariant to RIP, superior to primal IHT algorithms.
This paper considers compressed sensing and affine rank minimization in both noiseless and noisy cases and establishes sharp restricted isometry conditions for sparse signal and low-rank matrix recovery. The analysis relies on a key technical tool which represents points in a polytope by convex combinations of sparse v…
Efficient algorithm solves sparse nonconvex regression problems.
problem Sparse nonconvex square-root-loss regression problems.
method Proximal majorization-minimization (PMM) algorithm with sparse semismooth Newton method.
result Converges to a d-stationary point with Kurdyka-Łojasiewicz property.
This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…
Sparse coding is a basic task in many fields including signal processing, neuroscience and machine learning where the goal is to learn a basis that enables a sparse representation of a given set of data, if one exists. Its standard formulation is as a non-convex optimization problem which is solved in practice by heuri…
We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficient…
Paper proposes a nonconvex approach for sparse reduced rank regression.
problem Sparse reduced rank regression model estimation problem.
method Formulated as a nonconvex optimization problem with alternating minimization method.
result Nonconvex function leads to better estimation accuracy and efficiency.
New RNN reconstructs video frames from sparse measurements.
problem Sequential signal reconstruction from compressive measurements.
method Unfolding proximal gradient method for l1-l1 minimization.
result Outperforms state-of-the-art RNN models in video frame reconstruction.
Paper presents a hierarchical learning strategy for sparse data representation.
problem Sparse representation of multivariate datasets.
method Hierarchical approximation spaces at finer scales, stability and convergence analysis.
result Efficient data reconstruction and error minimization in prediction.
This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…
Proposes GCCA for detecting latent relations in multiview data with sparse structures.
problem Sparse CCA limitations for multiple datasets.
method Developed a GCCA algorithm based on distributed alternating iteration approach.
result Demonstrated effectiveness on synthetic and real-world datasets.
Significant attention has been given to minimizing a penalized least squares criterion for estimating sparse solutions to large linear systems of equations. The penalty is responsible for inducing sparsity and the natural choice is the so-called l 0 l_0 l 0 norm. In this paper we develop a Momentumized Iterative Shrinkage Th…
The paper improves sparse Gaussian processes by optimizing predictive loss.
problem Optimizing predictive loss in sparse Gaussian processes.
method Direct loss minimization (DLM) for log-loss and square loss, with product sampling (uPS) and biased Monte Carlo (bMC) for non-conjugate cases.
result DLM shows significant performance improvement in both log-loss and square loss cases.
The paper provides guarantees for an alternating minimization algorithm in dictionary learning.
problem Dictionary learning problem of factorizing samples into a basis and sparse vectors.
method Alternating minimization procedure switching between ℓ 1 \ell_1 ℓ 1 minimization and gradient descent. result Local convergence guarantees for the alternating minimization algorithm under a new matrix infinity norm condition.
Sparse tensor additive regression models tensor covariates for scalar responses.
problem Modeling scalar responses from tensor covariates with sparse and low-rank structures.
method Proposes a non-convex optimization problem and an efficient penalized alternating minimization algorithm.
result Establishes an error bound for the estimator and demonstrates the model's efficacy in simulations and online advertising.
New algorithm minimizes regret in sparse reinforcement learning.
problem Sparse reinforcement learning with unknown sparsity.
method Doubly robust approach combining feature vectors of all actions and novel analysis.
result Regret bound of i l d e O ( σ min − 1 s ⋆ H N ) ilde{O}(σ^{-1}_{\min} s_{\star} H \sqrt{N}) i l d e O ( σ m i n − 1 s ⋆ H N ) . GLAD learns a compact model to recover sparse graphs from data.
problem Recovering sparse conditional independence graphs from data.
method GLAD uses an Alternating Minimization (AM) algorithm as a model inductive bias and learns parameters via supervised learning.
result GLAD learns a very compact and effective model for sparse graph recovery.
Adapts PALM to solve NMF with smooth and sparse solutions.
problem Non-negative matrix factorization for dimensionality reduction and source separation.
method Adapted PALM for convex minimization with non-differentiable constraints.
result Solves NMF with smooth and/or sparse solutions.
Research shows finiteness in triangulations with girth constraints.
problem Finiteness of cellular partial triangulations with girth constraints.
method Characterization of sparse graphs and contraction-minimal graphs.
result There are finitely many (3,6)-tight and (3,3)-tight graphs.
New method for factor analysis using nuclear and ℓ 0 \ell_0 ℓ 0 norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, ℓ 0 \ell_0 ℓ 0 norm, and KL divergence. Used alternating minimization algorithm. result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
Paper finds sparse representation of functions using inverse scale space flow.
problem Finding sparse representation of L 2 L^2 L 2 functions. method Inverse scale space flow to minimize L 2 L^2 L 2 loss. result Convergence to optimal solution in ideal and noisy cases.
Autoencoders fail to capture sparse structure in 1-bit data compression.
problem Proving the performance of shallow autoencoders on sparse data compression.
method Gradient descent analysis and approximate message passing.
result Gradient descent minimizer for sparse data is the identity (up to permutation) above critical sparsity.
Network Lasso clusters sparse graph clusters efficiently.
problem Local graph clustering of sparse and chain-like clusters.
method Network Lasso minimizes total variation of cluster indicator signals.
result Network Lasso handles sparse clusters difficult for spectral clustering.
SANs use sparse activation functions to minimize model complexity.
problem Model complexity in unsupervised learning.
method Introduce φ metric, define activation functions, present Sparsely Activated Networks (SANs).
result SANs with selected activation functions have small description length and interpretable kernels.
Noise Collector detects sparse signals from noisy data efficiently.
problem Efficiently detecting sparse signals from noisy high-dimensional data.
method Introduces Noise Collector (NC) matrix to solve augmented system A ρ + C η = b 0 + e A ρ+ C η= b_0 + e A ρ + C η = b 0 + e . result The l 1 l_1 l 1 -norm minimal solution of the augmented system has zero false discovery rate for any level of noise. Proposes a new method for kernel density estimation using stagewise minimization and a simple dictionary.
problem Kernel density estimation with data-adaptive weighting parameters and sparse representation.
method Stagewise minimization algorithm based on U U U -divergence and a simple dictionary. result Develops non-asymptotic error bound for the proposed estimator.
ERFit identifies dynamic equations from data with minimal supervision.
problem Data-driven sparse system identification in science and engineering.
method Entropic Regression method.
result ERFit package simplifies sparse system identification for various applications.
The paper analyzes ℓ q \ell_q ℓ q optimization methods for high-dimensional linear regression.
problem Estimating sparse parameters from noisy observations in high-dimensional settings.
method Introduces and analyzes ℓ q \ell_q ℓ q optimization methods for sparse estimation. result Shows stable recovery properties and bounds for ℓ q \ell_q ℓ q minimization and regularization methods. New method outperforms standard procedures in heavy-tailed problems.
problem Regression function estimation under heavy-tailed conditions.
method Regularized risk minimization procedure based on median-of-means tournaments.
result The new procedure achieves near optimal accuracy and confidence in heavy-tailed problems.
Algorithm estimates sparse signals from linear measurements, improving recovery guarantees.
problem Estimating gradient-sparse signals from noisy linear measurements.
method Iterative alpha expansion with proximal descent and geometric penalty decay.
result Global recovery guarantees under cut-restricted isometry property for Gaussian designs.
The sparse representation classifier (SRC) has been utilized in various classification problems, which makes use of L1 minimization and works well for image recognition satisfying a subspace assumption. In this paper we propose a new implementation of SRC via screening, establish its equivalence to the original SRC und…
A new heuristic strategy improves sparse BSS performance.
problem Efficiently solving non-convex penalized matrix factorization problems.
method Combining heuristic approach with PALM algorithm.
result Significantly improved separation results on astrophysical data.
Paper proposes an algorithm for robust estimation using Huber's criterion.
problem Non-convexity and non-robustness of joint maximum likelihood estimation.
method Block-wise minimization majorization framework with data-adaptive step sizes.
result Improved convergence and robustness in sparse learning.
Recent research in off-the-grid compressed sensing (CS) has demonstrated that, under certain conditions, one can successfully recover a spectrally sparse signal from a few time-domain samples even though the dictionary is continuous. In particular, atomic norm minimization was proposed in \cite{tang2012csotg} to recove…
Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC appro…
Paper uses sparse learning to estimate quasi-potential and drift components in stochastic systems.
problem Estimating quasi-potential and drift components in stochastic systems.
method Sparse identification of non-linear dynamics (SINDy) combined with action minimization methods.
result Evaluation of quasi-potential landscape from a single trajectory.
Simplified screening tests for data points in optimization.
problem Discarding irrelevant data points in empirical risk minimization.
method Designing loss functions and regularizing convex losses to induce sparsity, using ellipsoidal approximations.
result Automatic discarding of data samples without losing optimization guarantees.
This work proposes a method to learn sparse representations that are more efficient for large-scale data retrieval.
problem Efficient retrieval of high-dimensional representations from large databases is computationally challenging.
method The approach minimizes the number of floating-point operations (FLOPs) by learning sparse embeddings with uniform non-zero entries.
result The proposed method achieves a similar or better speed-vs-accuracy tradeoff compared to existing baselines.
Thresholded Lasso bandit minimizes regret in sparse linear bandits.
problem Sparse stochastic contextual linear bandits with large feature vectors.
method Uses Lasso framework with thresholding to estimate reward function and its sparse support.
result Non-asymptotic regret upper bounds scaling as O ( log d + T ) \mathcal{O}( \log d + \sqrt{T}) O ( log d + T ) . New algorithms minimize regret in SSP with optimal sparse updates.
problem Minimizing regret in Stochastic Shortest Path models.
method Implicit finite-horizon approximation for analysis, model-free and model-based algorithms developed.
result Minimax optimal regret for both model-free and model-based algorithms.
Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) regret or O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) -consistent inference, with trade-off between exploration and exploitation. Paper proposes a new method for recovering missing samples in images.
problem Missing sample recovery in image signals.
method Iterative sparse recovery algorithm using constrained l 1 l_1 l 1 -norm minimization with a new CSIM fidelity metric. result Simulation results demonstrate the efficiency of the proposed method.
We present a comprehensive framework for structured sparse coding and modeling extending the recent ideas of using learnable fast regressors to approximate exact sparse codes. For this purpose, we develop a novel block-coordinate proximal splitting method for the iterative solution of hierarchical sparse coding problem…
New model leads to optimal test loss in sparse linear regression.
problem Sparse linear regression with low test loss despite interpolating training data.
method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.
result Training via gradient descent leads to an interpolator with near-optimal test loss.
Self-imitation learning improves RL in sparse, episodic reward settings.
problem Suboptimal performance of RL algorithms in sparse or episodic reward settings.
method Formulate policy optimization as a divergence minimization problem using Jensen-Shannon divergence, and learn shaped rewards from experience replays.
result Our algorithm performs comparably to existing algorithms in dense reward settings and significantly better in sparse and episodic reward settings.