A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New stability bounds for SGD on nonsmooth convex losses.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
Improved shuffling gradient methods converge faster for nonsmooth convex optimization.
New adaptive methods solve weakly convex stochastic optimization problems.
We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …
The paper relaxes assumptions for analyzing stochastic optimization algorithms.
In this paper, we consider the problem of minimizing the average of a large number of nonsmooth and convex functions. Such problems often arise in typical machine learning problems as empirical risk minimization, but are computationally very challenging. We develop and analyze a new algorithm that achieves robust linea…
New method solves nonsmooth low-rank matrix optimization problems efficiently.
Nesterov's extrapolation improves convergence in nonsmooth optimization.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
A new method solves convex optimization problems on manifolds efficiently.
New algorithm for fast nonsmooth optimization with applications in image processing and machine learning.
We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises …
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
This work establishes uniform convergence of subdifferentials in stochastic optimization.
In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter…
This work uses Lasry-Lions envelopes to solve nonconvex optimization problems.
Optimizes CM for stochastic convex optimization with progressive precision.
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and feasibility gap. Through theoretical analysis, we show how to update all the al…
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
Improved method reduces projection calls for nonsmooth convex optimization.
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
DoWG optimizer automatically adapts to convex and nonsmooth problems without tuning.
We propose an adaptive smoothing algorithm based on Nesterov's smoothing technique in \cite{Nesterov2005c} for solving "fully" nonsmooth composite convex optimization problems. Our method combines both Nesterov's accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriat…
Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for …
Improved analysis for clipped gradient methods in nonsmooth convex optimization under heavy-tailed noise.
Composite convex optimization problems which include both a nonsmooth term and a low-rank promoting term have important applications in machine learning and signal processing, such as when one wishes to recover an unknown matrix that is simultaneously low-rank and sparse. However, such problems are highly challenging t…
New algorithms reduce regret in online convex optimization with heavy-tailed gradients.
We introduce a geometrically transparent strict saddle property for nonsmooth functions. This property guarantees that simple proximal algorithms on weakly convex problems converge only to local minimizers, when randomly initialized. We argue that the strict saddle property may be a realistic assumption in applications…
Paper develops privacy-preserving federated learning for nonsmooth objectives.
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self…
We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving ERM problems with a nonsmooth regularization term. Current second-order and quasi-Newton methods for this problem either do not work well in the distributed setting or work only for specif…
Optimized method tackles convex optimization with heavy-tailed noise.
This paper tackles nonsmooth optimization in machine learning.
Due to their simplicity and excellent performance, parallel asynchronous variants of stochastic gradient descent have become popular methods to solve a wide range of large-scale optimization problems on multi-core architectures. Yet, despite their practical success, support for nonsmooth objectives is still lacking, ma…
On a Riemannian manifold, lower Ricci curvature bounds are known to be characterized by geodesic convexity properties of various entropies with respect to the Kantorovich-Rubinstein-Wasserstein square distance from optimal transportation. These notions also make sense in a (nonsmooth) metric measure setting, where they…
Study on Adam-family methods for nonsmooth optimization with convergence guarantees.
New algorithm solves complex optimization problems efficiently.
New methods help escape strict saddle points in nonsmooth optimization.
New saddle network architectures preserve convex-concave geometry in optimization problems.
Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.
Paper tackles efficient SVM classification over decentralized networks.
We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method…
New theory for nonsmooth systems helps optimize and control complex functions.
State-space smoothing has found many applications in science and engineering. Under linear and Gaussian assumptions, smoothed estimates can be obtained using efficient recursions, for example Rauch-Tung-Striebel and Mayne-Fraser algorithms. Such schemes are equivalent to linear algebraic techniques that minimize a conv…