We investigate the learning rate of multiple kernel learning (MKL) with and elastic-net regularizations. The elastic-net regularization is a composition of an -regularizer for inducing the sparsity and an -regularizer for controlling the smoothness. We focus on a sparse setting where the total …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “ell p regularization”
This paper solves parameter estimation with ordered regularization using ADMM.
problem Scaling up ordered regularization for large-scale data.
method Alternating Direction Method of Multipliers (ADMM) for ordered regularization.
result ADMM-O outperforms or matches state-of-the-art methods in parameter estimation.
The paper analyzes -LinR for Ising model selection using statistical mechanics.
problem Model selection consistency of -LinR for Ising models.
method Replica method from statistical mechanics, -regularized linear regression (-LinR).
result Model selection consistency with sample complexity $M=\mathcal{O}\left(\log N
ight)$.
New method improves subspace clustering for large datasets.
problem Efficiently clustering large datasets with mixed regularization.
method Oracle-based active set algorithm for elastic net subspace clustering.
result Proves correct and scalable active set method for optimal coefficients.
Safe screening rules reduce computation time in logistic regression with regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
A new algorithm speeds up EEG source localization using regularization.
problem Challenging inverse problem in mapping EEG readings to brain activity.
method Formulated as a graphical generalized elastic net inverse problem, solved with a variable projected algorithm (VPAL).
result VPAL provides faster and more accurate EEG source localization compared to existing methods.
The paper analyzes optimization methods for high-dimensional linear regression.
problem Estimating sparse parameters from noisy observations in high-dimensional settings.
method Introduces and analyzes optimization methods for sparse estimation.
result Shows stable recovery properties and bounds for minimization and regularization methods.
Paper proposes efficient tensor kernels for solving nonparametric regularization.
problem Nonparametric extensions of regularization.
method Fast dual algorithm using tensor kernels.
result Efficient solution to nonparametric regularization problems.
This paper shows how to use regularization effectively in training sparse CNNs.
problem Why regularization hasn't been used in sparse deep learning models like CNNs.
method Demonstrated that SGD is not suitable for regularization and replaced it with a new training algorithm based on regularized dual averaging (RDA).
result Achieved state-of-the-art sparsity for CNNs using RDA with regularization, achieving 95% sparsity for ResNet18 on CIFAR-10.
In this paper, we propose -norm regularized models to seek near-optimal sparse portfolios. These sparse solutions reduce the complexity of portfolio implementation and management. Theoretical results are established to guarantee the sparsity of the second-order KKT points of the -norm regularized models…
A neural network solves logistic regression with regularization efficiently.
problem Efficiently solving logistic regression with regularization due to non-differentiability of norm.
method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
On -regularized estimation for nonlinear models that have sparse underlying linear structuresmath.ST
In a recent work (arXiv:0910.2517), for nonlinear models with sparse underlying linear structures, we studied the error bounds of -regularized estimation. In this note, we show that -regularized estimation in some important cases can achieve the same order of error bounds as those in the aforementioned …
Deep Encoders tackle sparse approximation using neural networks.
problem Nonconvex sparse approximation problem.
method Proposed Deep Encoders, modeled as feed-forward neural networks with novel neurons and pooling functions.
result Deep Encoders outperform conventional sparse coding solutions in terms of inference speed, learning capacity, and scalability.
Analytic method optimizes portfolio variance with asymmetric constraint.
problem Optimizing portfolio variance under budget and asymmetric constraints.
method Replica method from disordered systems theory.
result Regularization extends optimization interval and suppresses large sample fluctuations.
New neural network training method uses bi-fidelity data to reduce errors.
problem Training neural networks with limited high-fidelity data.
method Bi-fidelity -regularization strategies.
result Bi-fidelity -regularization reduces errors by one order of magnitude.
In this paper we consider the problem of grouped variable selection in high-dimensional regression using regularization (), which can be viewed as a natural generalization of the regularization (the group Lasso). The key condition is that the dimensionality can…
We speed up cross-validation in multinomial logistic regression with an -regularization formula.
problem Slow cross-validation in multinomial logistic regression with -regularization.
method Perturbative approach using large data size and model dimensionality.
result Significant reduction in computational time for cross-validation.
NACT improves tensor regression predictions with regularization.
problem Improving tensor regression predictions with structural information.
method Noise-Augmented regularization on Tucker decomposition.
result Achieves exact regularization on core tensor in linear and generalized linear tensor regression.
The paper analyzes how adding and regularization to a neural network affects its convergence.
problem Analyzing the convergence of two-layered neural networks with regularization.
method Extended convergence analysis of two-layered bias-free networks with output, incorporating and regularization.
result When is small, the weight vector converges to the optimal solution with high probability.
Study examines stability of image-reconstruction algorithms using variational regularization.
problem Stability and robustness of image-reconstruction algorithms in medical imaging.
method Review and novel stability results for -regularized linear inverse problems, focusing on .
result Guarantees Lipschitz continuity for small and Hölder continuity for larger in function spaces.
New harmonic functions show nodal sets can be topologically complex despite frequency and regularity constraints.
problem Understanding the topology of nodal sets of harmonic functions with bounded frequency and regularity.
method Constructing harmonic functions on the unit ball with specific properties.
result The Betti numbers of the nodal set can be arbitrarily large, contradicting previous topological bounds.
Paper tackles low-rank matrix recovery with column -norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
Proposes a filtering method for cluster analysis using -norm regularization.
problem Improving cluster analysis by filtering data.
method Minimizes a least squares function with a weighted -norm penalty, approximated by smooth non-convex functions.
result The proposed method can enhance existing clustering techniques.
Proposes new -based methods for low-rank sparse subspace clustering.
problem Clustering high-dimensional data points represented by low-dimensional subspaces.
method Introduces two quasi-norm based regularizations: GMC-LRSSC and -LRSSC. Solves resulting nonconvex optimization problems using alternating direction method of multipliers.
result Demonstrates effectiveness of proposed methods on synthetic and real-world datasets.
Regularized linear regression improves binary classification performance, especially with ridge and regularization.
problem Improving binary classification accuracy with noisy labels.
method Systematic study of regularization strengths on linear classifiers trained on noisy binary classification data.
result Ridge regression consistently improves classification error, while regularization can induce sparsity and regularization can concentrate weights to two values.
Proposes a method to adapt models in nonstationary environments using ℓ1 regularization.
problem Adapting models to nonstationary environments in machine learning.
method Integrates ℓ1 regularization of differences between source and target parameters.
result Effective balance of stability and plasticity in model adaptation.
Regularization of minimization for dealing with outliers and noise in Statistics and Signal Recoverymath.OC
We study the robustness properties of norm minimization for the classical linear regression problem with a given design matrix and contamination restricted to the dependent variable. We perform a fine error analysis of the estimator for measurements errors consisting of outliers coupled with noise. We…
Adversarially robust generalization theory via Jacobian regularization for deep neural networksstat.ML
This study connects Jacobian regularization to adversarial robustness and improves generalization.
problem Adversarial attacks make deep neural networks vulnerable.
method Developed a connection between Jacobian regularization and adversarial training, and established robust generalization gaps.
result Jacobian norms are related to both standard and robust generalization.
AdamW optimizes a constrained loss with norm constraint.
problem Understanding the optimization behavior of AdamW with norm constraint.
method Analyzing AdamW as a smoothed version of SignGD and connecting it to Frank-Wolfe optimization.
result AdamW implicitly performs constrained optimization with norm constraint.
We investigate the learning rate of multiple kernel leaning (MKL) with elastic-net regularization, which consists of an -regularizer for inducing the sparsity and an -regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non…
Approximate Leave-one-out Cross Validation for Regression with Regularizers (extended version)math.ST
The paper improves ALO for -regularized models.
problem Estimating out-of-sample error for -regularized models.
method Developed a novel theory for -regularized problems, bounding ALO error.
result For -regularized problems, ALO error goes to zero as p goes to infinity.
The paper analyzes the trade-off between smoothness and sparsity in GCN using lp-regularized learning.
problem Quantifying the trade-off between smoothness and sparsity in GCN.
method Proposes a novel SGD proximal algorithm for GCNs with an inexact operator to analyze the stability of the -regularized stochastic learning.
result Establishes an explicit theoretical understanding of GCN with -regularized stochastic learning.
Unified framework for accurate coresets in latent variable models and regularized regression.
problem Efficiently training models on large datasets.
method Unified framework for constructing accurate coresets for latent variable models and -regularized regression.
result Unified framework reduces coreset size for latent variable models and -regularized regression.
New regularization techniques improve stability of deep neural networks.
problem Improving stability of deep neural networks in high-dimensional data.
method Apply manifold regularization to develop new regularizers based on graph Laplacian sparsification.
result Empirically, networks achieve high stability in various perturbation models, including adversarial attacks.
Multi-task feature learning aims to identity the shared features among tasks to improve generalization. It has been shown that by minimizing non-convex learning models, a better solution than the convex alternatives can be obtained. Therefore, a non-convex model based on the capped- regularization wa…
Sparse Identification of Nonlinear Dynamical Systems via Reweighted -regularized Least Squaresstat.ML
Method improves SINDy for noisy nonlinear systems.
problem Recover nonlinear dynamical systems from noisy data.
method Reweighted -regularized least squares.
result Improved accuracy and robustness in noisy conditions.
Study supports recovery of PDEs from noisy data using a specific regularization method.
problem Support recovery of PDEs from a single noisy trajectory.
method Applying ℓ1-regularized Pseudo-Least Squares model to a given data set.
result Support of ℓ1-c coefficients asymptotically converges to the true signed-support of the PDE.
Two new regularization methods improve neural network performance and complexity control.
problem Improving neural network performance and complexity control with correlated or high-dimensional features.
method Two regularization strategies: covariance-aware ridge and covariance-aware lasso.
result Improves predictive performance and complexity control over standard penalties.
New model leads to optimal test loss in sparse linear regression.
problem Sparse linear regression with low test loss despite interpolating training data.
method Developed a new parametrization of the model that combines benefits of ℓ1 and ℓ2 norms.
result Training via gradient descent leads to an interpolator with near-optimal test loss.
Advances robust principal component analysis with transformed ℓ1 regularization.
problem Recovering low-rank structures from noisy, partially observed data corrupted by sparse outliers.
method Proposes transformed ℓ1 (TL1) regularization to improve approximations of rank and ℓ0 functional.
result Achieves higher accuracy in estimating low-rank and sparse components compared to classical convex models, especially under non-uniform sampling schemes.
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization. By training quantization-ready networks, our approach enables storing a single set of weights that can be quantized on-deman…
New algorithmic view of ℓ2 regularization using ODEs and path-following methods.
problem Optimizing convex loss functions with ℓ2 regularization.
method Established an equivalence between ℓ2-regularized solution paths and ODEs, proposing path-following algorithms based on homotopy methods and numerical ODE solvers.
result The solution path can be viewed as a hybrid of gradient descent and Newton method, providing novel schemes to choose grid points and reducing computational cost.
This paper proposes a method to select relevant features for multi-label learning.
problem Feature selection in multi-label learning to retain important information with minimal features.
method Random manifold sampling and joint sparse regularization to solve multicollinearity and obtain sparse feature sets.
result The proposed method outperforms other methods in selecting relevant features for multi-label learning.
FedElasticNet reduces communication costs and handles client drift in FL.
problem Expensive communication costs and client drift issues in federated learning.
method Leverages elastic net regularizers to sparsify local updates and limit client drift.
result FedElasticNet effectively resolves communication cost and client drift problems.
We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.
problem Analyzing coresets for regularized regression problems.
method Examined coresets for ridge regression and proposed a modified lasso problem.
result No coreset for regularized regression can be smaller than the unregularized version when .
Recently, Mahoney and Orecchia demonstrated that popular diffusion-based procedures to compute a quick \emph{approximation} to the first nontrivial eigenvector of a data graph Laplacian \emph{exactly} solve certain regularized Semi-Definite Programs (SDPs). In this paper, we extend that result by providing a statistica…
This paper addresses how well we can recover a data matrix when only given a few of its elements. We present a randomized algorithm that element-wise sparsifies the data, retaining only a few its elements. Our new algorithm independently samples the data using sampling probabilities that depend on both the squares ($\e…