CD converges linearly for MCP/SCAD penalized least squares.
problem Recovering sparse signals from data.
method Coordinate descent for MCP/SCAD penalized least squares.
result CD converges linearly to solutions of MCP/SCAD penalized least squares.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “sparse least squares”
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an -constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…
A new screening rule 'dynamic Sasvi' improves sparse optimization speed.
problem Sparse optimization problem identification.
method Flexible framework based on Fenchel-Rockafellar duality for norm-regularized least squares.
result Dynamic Sasvi can eliminate more features and increase solver speed.
Sparse LR-LSSVM improves kernel machine performance.
problem Improving kernel machine performance with controlled model size.
method Introduces LR-LSSVM with low rank kernels and a two-step optimization algorithm.
result Proposed algorithm's performance is comparable or superior to existing kernel machines.
We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
Bayesian system ID improves robustness to sparse, noisy data.
problem Robust system identification with sparse, noisy data.
method Probabilistic formulation of system identification using Bayesian posterior.
result The log posterior is more robust and less affected by multiple minima.
Efficient method for high-dimensional American option pricing and hedging.
problem High-dimensional American option pricing and hedging.
method Gradient-enhanced sparse Hermite polynomial expansions combined with least squares Monte Carlo.
result Outperforms state-of-the-art methods in high dimensions with comparable computational cost.
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
Optimal hashing embeddings reduce linear least squares solving time.
problem Efficiently solving large-scale linear least squares problems.
method Optimal hashing sketching matrices for linear least squares.
result Ski-LLS outperforms state-of-the-art solvers on various problem types.
Unified analysis of reweighted least-squares algorithms for linear models.
problem Recovering unknown signals from linear measurements using reweighted least squares.
method Unified asymptotic analysis of IRLS, lin-RFM, and alternating minimization algorithms.
result The algorithms can achieve favorable performance in a few iterations with appropriate reweighting.
Quantization can be used to form new vectors/matrices with shared values close to the original. In recent years, the popularity of scalar quantization for value-sharing applications has been soaring as it has been found huge utilities in reducing the complexity of neural networks. Existing clustering-based quantization…
In this paper, we study a fast approximation method for {\it large-scale high-dimensional} sparse least-squares regression problem by exploiting the Johnson-Lindenstrauss (JL) transforms, which embed a set of high-dimensional vectors into a low-dimensional space. In particular, we propose to apply the JL transforms to …
Sparse Identification of Nonlinear Dynamical Systems via Reweighted -regularized Least Squaresstat.ML
Method improves SINDy for noisy nonlinear systems.
problem Recover nonlinear dynamical systems from noisy data.
method Reweighted -regularized least squares.
result Improved accuracy and robustness in noisy conditions.
Dual-sPLS improves feature selection and prediction in high-dimensional data.
problem Relating variables to a response in high-dimensional chemometric problems.
method Generalizes PLS1 algorithm with dual norm penalizations and a shrinking ratio parameter.
result Favorably compares to similar regression methods on simulated and real chemical data.
From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identificationstat.ML
This work improves SINDy-type algorithms for system identification using score-guided dictionary selection.
problem Improving accuracy and interpretability in dynamical system identification.
method Score-guided library selection to refine dictionary terms in sparse regression.
result Score-guided methods enhance SINDy's robustness in discovering governing equations.
Paper improves RLS for sparse outlier detection in linear models.
problem Outliers contaminate linear regression models infrequently.
method Hierarchical-optimization recursive least squares with sparsity-inducing regularization.
result The method robustly estimates linear filters/systems with outliers.
Recursive Group lassostat.ME
We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal -penalized recursive least squares (R…
A joint optimization approach to identifying sparse dynamics using least squares kernel collocationstat.ME
Develops a new method for learning ODEs from sparse data.
problem Learning systems of ODEs from scarce, partial, and noisy data.
method Combines sparse recovery and RKHS techniques.
result Significant gains in accuracy, sample efficiency, and robustness to noise.
Paper introduces a novel matrix-wise sparse MNNLS formulation and algorithm.
problem Sparse nonnegative least squares with multiple right-hand sides.
method Matrix-wise sparsity constraint, two-step algorithm.
result More accurate results compared to state-of-the-art methods.
Non-asymptotic analysis of the performance of the penalized least trimmed squares in sparse modelsstat.ML
This study analyzes LTS in sparse models with finite sample error bounds.
problem Robust regression in high-dimensional sparse models with limited data.
method Non-asymptotic analysis of LTS error bounds.
result Established finite sample error bounds for LTS in sparse models.
We study the problem of inferring a sparse vector from random linear combinations of its components. We propose the Accelerated Orthogonal Least-Squares (AOLS) algorithm that improves performance of the well-known Orthogonal Least-Squares (OLS) algorithm while requiring significantly lower computational costs. While OL…
We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an l1-l2 norm regularizer. By systematically treating the non-differentiable regulariz…
Structured and sparse partial least squares coherence for multivariate cortico-muscular analysisstat.AP
New algorithm extracts shared latent space for cortico-muscular interactions.
problem Challenges of high dimensionality and limited sample sizes in multivariate cortico-muscular analysis.
method Structured and sparse partial least squares coherence (ssPLSC) algorithm.
result ssPLSC achieves competitive or better performance in scenarios with limited sample sizes and high noise levels.
Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squares Minimizationcs.LG
This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…
A new NMF variant tackles underdetermined problems with sparse and separable assumptions.
problem Underdetermined blind source separation, especially multispectral image unmixing.
method Sparse Separable Nonnegative Matrix Factorization (SSNMF) combining separability and sparsity assumptions. Algorithm based on SNPA and sparse nonnegative least squares.
result In noiseless settings, the algorithm recovers true underlying sources.
We consider the Orthogonal Least-Squares (OLS) algorithm for the recovery of a -dimensional -sparse signal from a low number of noisy linear measurements. The Exact Recovery Condition (ERC) in bounded noisy scenario is established for OLS under certain condition on nonzero elements of the signal. The new result a…
R package spca computes sparse principal components efficiently.
problem Sparse principal components analysis (SPCA) for interpretable data.
method Least squares sparse principal component analysis (LS-SPCA) with efficient C++ backend.
result Computes sparse principal components that maximize variance and maintain strong correlations with PCs.
In this paper, we propose a low-rank approximation method based on discrete least-squares for the approximation of a multivariate function from random, noisy-free observations. Sparsity inducing regularization techniques are used within classical algorithms for low-rank approximation in order to exploit the possible sp…
A new method speeds up ALS for recommender systems by subsampling key elements.
problem High computational cost of ALS for large-scale datasets.
method Core-elements subsampling method for efficient ALS approximation.
result Achieves similar accuracy with significantly reduced computational time.
Develops a privacy-preserving algorithm for sparse robust regression.
problem Privacy-preserving machine learning for sparse robust regression.
method Develops FRAPPE algorithm for non-smooth loss under differential privacy.
result Achieves better privacy and statistical accuracy trade-off.
Proposes a new method for joint sample and feature selection in multi-view data.
problem Cannot detect latent subsets of samples and remove outliers.
method Weighted Sparse Partial Least Squares (-wsPLS) method for joint sample and feature selection.
result Developed globally convergent algorithm and iterative algorithms for multi-view data fusion.
It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. \ci…
We solve robust regression and matrix completion problems with sparse and low-rank models.
problem Adversarial contamination and noisy matrix completion in high-dimensional settings.
method Subgaussian statistical learning framework, trace-regression with matrix decomposition, novel Huber-type loss.
result Near-optimal estimation rates for robust regression and matrix completion.
Study on learning sparse fixed-structure Gaussian Bayesian networks with near-optimal sample complexity.
problem Learning a fixed-structure Gaussian Bayesian network up to a bounded error in total variation distance.
method Analysis of node-wise least squares regression and introduction of BatchAvgLeastSquares and CauchyEst algorithms.
result BatchAvgLeastSquares and CauchyEstTree have near-optimal sample complexity.
For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations of larger (kernel) model classes lead to algorithms, which naturally scale linear…
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensionsstat.ML
We develop and analyze stochastic optimization algorithms for problems in which the expected loss is strongly convex, and the optimum is (approximately) sparse. Previous approaches are able to exploit only one of these two structures, yielding an $\order(\pdim/T)$ convergence rate for strongly convex objectives in $\pd…
Demixing problems in many areas such as hyperspectral imaging and differential optical absorption spectroscopy (DOAS) often require finding sparse nonnegative linear combinations of dictionary elements that match observed data. We show how aspects of these problems, such as misalignment of DOAS references and uncertain…
Gradient descent with specific initialization and step size achieves optimal sparse signal recovery.
problem Reconstructing a sparse signal from underdetermined linear measurements.
method Gradient descent with specific initialization, step size, and stopping time.
result Achieves the minimax rate with poly-logarithmic factors and adapts to instance difficulty.
Paper presents novel online MTL methods using WRLS and OSLSSVR.
problem Online Multi-Task Learning (MTL) Regression Problems
method Develops recursive versions of WRLS and OSLSSVR for MTL.
result Achieves exact and approximate recursions with quadratic cost.
In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM…
We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS occurs naturally in a wide variety of applications where an unknown, non-negative quantity must be recovered from linear measurements. We present a unified framework for S-NNLS based on a rectified power exponential scale mixture prior on the spars…
Federated learning improves SPCA for sparse components.
problem Data privacy and sharing constraints in machine learning.
method Federated learning framework applied to SPCA with L1 regularization and smoothing.
result Federated SPCA achieves sparse component loadings with improved interpretability.
We consider solving the -regularized least-squares (-LS) problem in the context of sparse recovery, for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a r…
Quantum LS-SVM simplifies matrix inversion for faster machine learning.
problem Speeding up machine learning algorithms for large datasets.
method Introduces a novel quantum algorithm using continuous variables to simplify matrix inversion in LS-SVM, and proposes a hybrid quantum-classical approach for sparse solutions.
result Quantum LS-SVM achieves exponential speed-up and can solve classically difficult tasks.
We propose a novel algorithm for greedy forward feature selection for regularized least-squares (RLS) regression and classification, also known as the least-squares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on each iteration adds the feat…