We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “iterative regularization”
New approach uses hinge loss for iterative regularization in classification.
problem Improving classification accuracy through regularization.
method Develops an iterative regularization approach based on hinge loss.
result Proves convergence and rates of convergence for classification.
New iterative regularization method tackles non-smooth, non-strongly convex functionals.
problem Tackles non-smooth, non-strongly convex functionals in regularization problems.
method Primal-dual algorithm with convergence and stability analysis.
result First iterative regularization procedure for non-smooth, non-strongly convex functionals.
Study iterative regularization for linear models with convex bias, improving robust sparse recovery.
problem Improving robust sparse recovery with iterative regularization for linear models.
method Primal-dual gradient approach, analyzing convergence in presence of noise, combining regularization and optimization.
result Theoretical results show state-of-the-art performances with computational speed-ups.
Entropy regularized algorithms such as Soft Q-learning and Soft Actor-Critic, recently showed state-of-the-art performance on a number of challenging reinforcement learning (RL) tasks. The regularized formulation modifies the standard RL objective and thus generally converges to a policy different from the optimal gree…
Paper improves learning rates for GSC loss functions using iterated Tikhonov regularization.
problem Improving learning rates for GSC loss functions.
method Iterated Tikhonov regularization using proximal point method.
result Achieves fast and optimal rates for GSC loss functions.
IRKSN algorithm achieves sparse recovery with wider applicability conditions.
problem Sparse recovery challenges due to NP-hard nature and restrictive conditions.
method IRKSN algorithm based on -support norm regularizer.
result Achieves sparse recovery with explicit constants and standard linear rate.
New method averages SGD iterates to achieve adjustable regularization.
problem Overfitting in machine learning models.
method Averaging SGD iterates for regularized solutions.
result Obtain regularized solutions without tuning parameters.
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
problem Solving composite convex optimization problems with composite regularizers.
method Analyzed proximal stochastic gradient method and randomized incremental proximal method under relaxed variance assumptions.
result Proves convergence rate for last iterate of both algorithms under componentwise convexity and smoothness.
In this paper we consider regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…
New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
Computational Guarantees for Doubly Entropic Wasserstein Barycenters via Damped Sinkhorn Iterationsmath.OC
New algorithm for computing Wasserstein barycenters with guarantees.
problem Computing Wasserstein barycenters with varying regularization strengths.
method Damped Sinkhorn iterations followed by exact maximization/minimization steps.
result First non-asymptotic convergence guarantees for approximating Wasserstein barycenters.
Optimal ridge regularization computed iteratively from generative parameters.
problem Finding the optimal ridge regularization strength for linear regression.
method Iterative procedure to compute optimal regularization strength numerically.
result The proposed procedure attains near-optimal generalization across various conditions.
This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.
problem Solving Total Variation (TV) regularized problems with iterative algorithms.
method Unrolling proximal gradient descent solvers to learn their parameters.
result Two approaches to compute derivatives through proximal operators improve performance.
Within a statistical learning setting, we propose and study an iterative regularization algorithm for least squares defined by an incremental gradient method. In particular, we show that, if all other parameters are fixed a priori, the number of passes over the data (epochs) acts as a regularization parameter, and prov…
The paper analyzes SGD with dropout regularization in linear models, proving asymptotic properties and providing inference tools.
problem Analyzing the behavior of SGD with dropout regularization in linear models.
method Establishing geometric-moment contraction (GMC) and proving quenched central limit theorems (CLT).
result The existence of a unique stationary distribution and asymptotic normality results for SGD with dropout.
Many recent successful (deep) reinforcement learning algorithms make use of regularization, generally based on entropy or Kullback-Leibler divergence. We propose a general theory of regularized Markov Decision Processes that generalizes these approaches in two directions: we consider a larger class of regularizers, and…
Improved iterative hard thresholding for faster, sparser solutions.
problem Finding sparser solutions without sacrificing runtime.
method Adaptive regularization framework applied to iterative hard thresholding.
result Returns solutions with sparsity , improving over existing methods.
In this paper, we propose a stochastic Primal-Dual Hybrid Gradient (PDHG) approach for solving a wide spectrum of regularized stochastic minimization problems, where the regularization term is composite with a linear function. It has been recognized that solving this kind of problem is challenging since the closed-form…
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
RRPI improves offline RL by optimizing policies against worst-case dynamics.
problem Offline RL's performance degrades under distribution shift and transition uncertainty.
method Formulates offline RL as robust policy optimization, treating transition kernel as decision variable.
result RRPI achieves strong average performance on D4RL benchmarks, outperforming recent baselines.
Matrix factorization was used to generate investment recommendations for investors. An iterative conjugate gradient method was used to optimize the regularized squared-error loss function. The number of latent factors, number of iterations, and regularization values were explored. Overfitting can be addressed by either…
We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods. Rather than a uniform average of the iterates, we consider a weighted average, with weights decaying in a geometric fashion. In the context of linear least squares regression, we show that this …
Study compares dropout and l2 regularization in linear models.
problem Understanding the statistical behavior of dropout and l2 regularization in linear models.
method Derives non-asymptotic bounds for gradient descent iterates with dropout and compares them to l2 regularization.
result Indicates a more subtle relationship between dropout and l2 regularization, highlighting interactions between dynamics and randomness.
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
New algorithms solve complex minimax problems efficiently.
problem Nonconvex-strongly concave minimax problems in machine learning.
method Gradient norm regularized trust-region (GRTR) and Levenberg-Marquardt (LMNegCur) algorithms.
result Proved iteration complexities matching best known results.
Stochastic Proximal Gradient Algorithm with Minibatches. Application to Large Scale Learning Modelscs.LG
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size after iterations achieves suboptimality in expected quadratic distance.
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
KL regularization helps RL algorithms by implicitly averaging q-values.
problem Understanding why KL regularization improves RL performance.
method An approximate value iteration scheme, studying KL and entropy regularization.
result Strong performance bound combining linear horizon dependency and averaging effect of estimation errors.
KL-constrained API shows optimization issues and improved with regularization.
problem Optimization issues in KL-constrained API algorithms.
method Comparison of KL divergence as a constraint vs. regularizer, empirical evaluation.
result KL-constrained API is not guaranteed to converge and incurs linear regret.
EarlyStopping package helps prevent overfitting in iterative learning procedures.
problem Preventing overfitting in iterative learning procedures.
method Early stopping implemented in a Python package.
result Demonstrates how to use the EarlyStopping package to explore implicit regularization.
We establish a regularity theorem for the Harmonic - Einstein Equation. As a byproduct of the local regularity, we also have a compactness theorem on Harmonic - Einstein equation. The method is mainly the Moser iteration technique which has been used and developed by \cite{BKN89}, \cite{Tian90}, \cite{TV05a} and others…
Paper proposes a new sparse group k-max regularization for sparsity constraints.
problem Linear inverse problems with sparsity constraints are NP-hard.
method Sparse group k-max regularization, iterative soft thresholding algorithm.
result Approximates l0 norm more closely and enhances group-wise and in-group sparsity.
QMME balances cost and speed in convex optimization.
problem Slow convergence of first-order methods and high cost of second-order methods.
method Minimizing quadratic majorants with fixed curvature at each iteration.
result QMME framework achieves sequential convergence under standard assumptions.
We study the regularity properties for solutions of a class of Schrödinger equations on a stratified space endowed with an iterated edge metric. The focus is on obtaining optimal Hölder regularity of these solutions assuming fairly minimal conditions on the underlying metric and potential.
We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an l1-l2 norm regularizer. By systematically treating the non-differentiable regulariz…
New method for optimistic planning in MDPs using regularization.
problem Optimistic planning in infinite-horizon discounted MDPs.
method Regularized dynamic programming for approximate value iteration.
result Achieves near-optimal statistical guarantees in learning policies.
Proposes an exponentially increasing step-size for faster parameter estimation in statistical models.
problem Slow convergence of gradient descent in locally convex loss functions.
method Exponentially increasing step-size in gradient descent algorithm.
result Converges linearly to optimal solution under homogeneous assumptions.
Heuristic algorithm for 1D and 2D unfoldingphysics.data-an
A very simple heuristic approach to the unfolding problem will be described. An iterative algorithm starts with an empty histogram and every iteration aims to add one entry to this histogram. The entry to be added is selected according to a criteria which includes a test and a regularization. After a relatively s…
Sparse model selection is ubiquitous from linear regression to graphical models where regularization paths, as a family of estimators upon the regularization parameter varying, are computed when the regularization parameter is unknown or decided data-adaptively. Traditional computational methods rely on solving a set o…
New online method estimates OT distances from sample streams.
problem Computing OT distances between arbitrary distributions.
method Online Sinkhorn algorithm using iterative enrichment of non-parametric representation.
result Consistent estimation of true regularized OT distance with nearly-O(1/n) sample complexity.
Proposes batch version of Greenkhorn for multimarginal OT, proving convergence.
problem Optimal transport problems with multiple marginals.
method Batch Greenkhorn algorithm, iterative Bregman projections, greedy control.
result Global linear rate of convergence and explicit iteration complexity bounds.
Iteratively reweighted algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accele…
New algorithm reduces bias and variance in weighted least-squares solutions.
problem Inconsistent linear least-squares problems with rapidly decaying singular values.
method Regularized block Kaczmarz (ReBlocK) algorithm.
result ReBlocK outperforms RBK and minibatch SGD for inconsistent problems.
Paper proves suboptimal convergence rate of last iterate for SGDM.
problem Proves suboptimal convergence rate of last iterate for SGDM.
method Focuses on convergence rate of last iterate of SGDM, introduces Follow-The-Regularized-Leader-based algorithms.
result Shows optimal convergence rate of last iterate for unconstrained convex stochastic optimization problems.
The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computations of the proximity operators of the two functions separately. The paper investigates a stochastic version of the algorithm where both funct…
Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysismath.ST
New inequalities help optimize first-order algorithms for statistical risk analysis.
problem Optimizing first-order iterative algorithms for statistical risk analysis.
method Introducing basic inequalities that connect implicit and explicit regularization.
result The basic inequalities translate the number of iterations into an effective regularization coefficient.
We propose a family of relaxations of the optimal transport problem which regularize the problem by introducing an additional minimization step over a small region around one of the underlying transporting measures. The type of regularization that we obtain is related to smoothing techniques studied in the optimization…