Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “$\ell_1$-regression”
Safe screening rules reduce computation time in logistic regression with regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
New algorithm solves -norm constrained multilinear logistic regression for tensor data.
problem Non-convex and nonsmooth -norm constraints in multilinear logistic regression.
method APALM method for globally convergent optimization.
result APALM ensures convergence to a first-order critical point.
Safe screening rules reduce -regression computation by fixing 76% of variables.
problem Efficiently solving -regression problems with large datasets.
method Convex relaxation and safe screening rules to eliminate variables.
result 76% of variables can be fixed to their optimal values, reducing computational burden.
The paper analyzes -LinR for Ising model selection using statistical mechanics.
problem Model selection consistency of -LinR for Ising models.
method Replica method from statistical mechanics, -regularized linear regression (-LinR).
result Model selection consistency with sample complexity $M=\mathcal{O}\left(\log N
ight)$.
Improved sketching for logistic and regression with near-linear dimensions.
problem Efficiently approximate and logistic regression problems.
method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for and logistic regression.
Improved algorithms solve -norm regression problems efficiently.
problem Efficiently solving -norm regression problems for .
method Iterative refinement scheme using smoothed -norms to improve solutions.
result Solves -norm regression to accuracy in iterations.
The paper analyzes optimization methods for high-dimensional linear regression.
problem Estimating sparse parameters from noisy observations in high-dimensional settings.
method Introduces and analyzes optimization methods for sparse estimation.
result Shows stable recovery properties and bounds for minimization and regularization methods.
Positive results for agnostic regression with various losses.
problem Agnostic regression with bounded sample compression.
method Generic and efficient sample compression schemes for real-valued functions.
result Exact and approximate compression schemes for specific losses.
A neural network solves logistic regression with regularization efficiently.
problem Efficiently solving logistic regression with regularization due to non-differentiability of norm.
method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
Paper introduces -DER for regression tasks using morphological operators and convex-concave procedure.
problem Developing a universal approximator for regression tasks.
method Introduces -DER model, trains it using a convex-concave procedure (CCP) to minimize least-squares.
result Outperforms other hybrid morphological models and state-of-the-art approaches.
We speed up cross-validation in multinomial logistic regression with an -regularization formula.
problem Slow cross-validation in multinomial logistic regression with -regularization.
method Perturbative approach using large data size and model dimensionality.
result Significant reduction in computational time for cross-validation.
Least Angle Regression is a promising technique for variable selection applications, offering a nice alternative to stepwise regression. It provides an explanation for the similar behavior of LASSO (-penalized regression) and forward stagewise regression, and provides a fast implementation of both. The idea has…
Paper proposes a new method for linear regression with heavy-tailed data.
problem Linear regression with heavy-tailed distributions.
method Truncated minimization problem with absolute loss.
result Achieves high-probability risk bound without exponential moment conditions.
NACT improves tensor regression predictions with regularization.
problem Improving tensor regression predictions with structural information.
method Noise-Augmented regularization on Tucker decomposition.
result Achieves exact regularization on core tensor in linear and generalized linear tensor regression.
Sparse logistic regression recovers any discrete pairwise graph model.
problem Recovering the Markov graph of discrete pairwise graphical models.
method Maximum conditional log-likelihood with convex optimization.
result The algorithm can recover any arbitrary discrete pairwise graphical model.
New MIP framework solves high-dimensional -regularized regression problems.
problem Exact computation of -regularized regression estimators is challenging for large .
method Specialized nonlinear branch-and-bound (BnB) framework with first-order optimization.
result Achieves speedups of at least 5000x compared to state-of-the-art exact methods.
This work provides efficient algorithms for approximating ℓ_p sensitivities and related statistics.
problem Estimating the importance of datapoints in high-dimensional datasets.
method Efficient algorithms for computing α-approximation of ℓ_1 sensitivities and total sensitivity using importance sampling and sensitivity computations.
result Real-world datasets have significantly lower intrinsic effective dimensionality than theoretical predictions.
New algorithm solves constrained ℓ_p regression problems efficiently.
problem Minimizing ℓ_p regression over unit vectors with constraints.
method Uses core-sets and provable constant factor approximation.
result First provable constant factor approximation algorithm.
Given a matrix and a vector , we consider the regression problem with guarantees: finding a vector such that where $x^*=\arg\min_{x\in \mathbb{R}^d}\|Ax-b\|…
We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.
problem Analyzing coresets for regularized regression problems.
method Examined coresets for ridge regression and proposed a modified lasso problem.
result No coreset for regularized regression can be smaller than the unregularized version when .
Unified framework for accurate coresets in latent variable models and regularized regression.
problem Efficiently training models on large datasets.
method Unified framework for constructing accurate coresets for latent variable models and -regularized regression.
result Unified framework reduces coreset size for latent variable models and -regularized regression.
Paper develops algorithms for sparse linear regression with generalized elastic net penalty.
problem Sparse linear regression with robust penalty for high-dimensional data.
method Iterative Reweighted Framework based on ADMM and PMM with SNN.
result Efficient algorithms provide superior performance in both simulated and real data.
Improved robustness in kernel-based regression via novel loss function and IRLS.
problem Noise sensitivity in kernel-based regression methods.
method Proposed -loss function and iteratively reweighted least squares (IRLS) optimization.
result Improved noise robustness in kernel-based regression methods.
We develop coresets for multiple ℓ_p regression problems, improving approximation sizes and efficiency.
problem Efficiently approximating multiple ℓ_p regression problems with coresets.
method Construct coresets of size sublinear in m for multiple ℓ_p regression, improving bounds for different p values.
result We construct coresets with size nearly optimal in d and independent of m for multiple ℓ_p regression.
In recent years, stochastic gradient descent (SGD) methods and randomized linear algebra (RLA) algorithms have been applied to many large-scale problems in machine learning and data analysis. We aim to bridge the gap between these two methods in solving constrained overdetermined linear regression problems---e.g., $\el…
Regularized linear regression improves binary classification performance, especially with ridge and regularization.
problem Improving binary classification accuracy with noisy labels.
method Systematic study of regularization strengths on linear classifiers trained on noisy binary classification data.
result Ridge regression consistently improves classification error, while regularization can induce sparsity and regularization can concentrate weights to two values.
Sparse linear regression -- finding an unknown vector from linear measurements -- is now known to be possible with fewer samples than variables, via methods like the LASSO. We consider the multiple sparse linear regression problem, where several related vectors -- with partially shared support sets -- have to be recove…
New method speeds up subspace clustering by 20-30x.
problem Efficiently clustering high-dimensional data with correlated variables.
method Modified sparse subspace clustering using Ordered Weighted (OWL) regression.
result Significantly reduces computational complexity and achieves better clustering results.
In multivariate regression, a -dimensional response vector is regressed upon a common set of covariates, with a matrix of regression coefficients. We study the behavior of the multivariate group Lasso, in which block regularization based on the norm is used for supp…
In this paper we consider the problem of grouped variable selection in high-dimensional regression using regularization (), which can be viewed as a natural generalization of the regularization (the group Lasso). The key condition is that the dimensionality can…
Optimal sketching bounds for sparse linear regression under various loss functions are established.
problem Sparse linear regression under different loss functions.
method Distribution over oblivious sketches for sparse norm regression and hinge-like loss functions.
result Optimal sketching bounds with rows for sparse norm regression and rows for hinge-like loss functions.
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by -norms of interval projections, introduces a new bandwidth selection procedure.
result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
New model leads to optimal test loss in sparse linear regression.
problem Sparse linear regression with low test loss despite interpolating training data.
method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.
result Training via gradient descent leads to an interpolator with near-optimal test loss.
Study on theoretical limits of sparse-regression algorithms using Fl RDT.
problem Understanding the performance limits of norm based optimization algorithms in compressed sensing and sparse regression.
method Utilized Fully lifted random duality theory (Fl RDT) to analyze the maximum-likelihood (ML) decoding performance.
result Uncovered phase-transition (PT) and descending (d) curves that separate successful and unsuccessful algorithm performance.
Study on Transfer Elastic Net error bounds and grouping effect.
problem Estimation error and grouping effect in Transfer Elastic Net.
method Derives non-asymptotic error bound and examines grouping effect scenarios.
result Effective error bounds and grouping effect observed in Transfer Elastic Net.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
Two-layer NN with channel attention learns low-degree spherical polynomials efficiently.
problem Learning low-degree spherical polynomials with over-parameterized neural networks.
method Two-layer neural network with channel attention, vanilla gradient descent, learnable channel selection.
result Minimally improved sample complexity of $n \asymp Θ(d^{\ell_0}/\eps)$ for learning low-degree polynomials.
The -penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weigh…
Proposes a new robust expectile regression method for high-dimensional data.
problem Heterogeneity in high-dimensional data with heteroscedastic variance or inhomogeneous covariate effects.
method Iteratively reweighted ℓ1-penalization for robust expectile regression (retire).
result Oracle convergence rate after log(log d) iterations in high-dimensional settings.
Early stopping improves logistic regression's calibration and consistency in high dimensions.
problem Improving the statistical performance of gradient descent in overparameterized logistic regression.
method Investigates the effects of early stopping on gradient descent in logistic regression.
result Early-stopped gradient descent is well-calibrated and statistically consistent, while asymptotic gradient descent is not.
Feature selection is a standard approach to understanding and modeling high-dimensional classification data, but the corresponding statistical methods hinge on tuning parameters that are difficult to calibrate. In particular, existing calibration schemes in the logistic regression framework lack any finite sample guara…
A new method for multi-target regression robust to outliers.
problem Outliers in regression data affect the performance of ELM-based methods.
method Generalized Outlier Robust ELM (GOR-ELM) using norm and Elastic Net theory.
result GOR-ELM outperforms other ELM-based techniques in datasets with outliers.
Regularization of minimization for dealing with outliers and noise in Statistics and Signal Recoverymath.OC
We study the robustness properties of norm minimization for the classical linear regression problem with a given design matrix and contamination restricted to the dependent variable. We perform a fine error analysis of the estimator for measurements errors consisting of outliers coupled with noise. We…
We demonstrate that the primal-dual witness proof method may be used to establish variable selection consistency and -bounds for sparse regression problems, even when the loss function and/or regularizer are nonconvex. Using this method, we derive two theorems concerning support recovery and -…
Paper develops robust regression method for heavy-tailed errors.
problem High-dimensional robust regression with heavy-tailed errors.
method Iteratively reweighted -penalized adaptive Huber regression.
result Oracle convergence rate and variable selection consistency achieved.
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient penalty with hyperpriors.
result Reduces bias in sparse Bayesian learning.
Compressed Regressionstat.ML
Recent research has studied the role of sparsity in high dimensional regression and signal reconstruction, establishing theoretical limits for recovering sparse models from sparse data. This line of work shows that -regularized least squares regression can accurately estimate a sparse linear model from nois…