A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New stability bounds for SGD on nonsmooth convex losses.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
Improved shuffling gradient methods converge faster for nonsmooth convex optimization.
We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …
In this paper, we consider the problem of minimizing the average of a large number of nonsmooth and convex functions. Such problems often arise in typical machine learning problems as empirical risk minimization, but are computationally very challenging. We develop and analyze a new algorithm that achieves robust linea…
New adaptive methods solve weakly convex stochastic optimization problems.
In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter…
The paper relaxes assumptions for analyzing stochastic optimization algorithms.
New method solves nonsmooth low-rank matrix optimization problems efficiently.
New algorithm for fast nonsmooth optimization with applications in image processing and machine learning.
We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises …
Nesterov's extrapolation improves convergence in nonsmooth optimization.
A new method solves convex optimization problems on manifolds efficiently.
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and feasibility gap. Through theoretical analysis, we show how to update all the al…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
This work establishes uniform convergence of subdifferentials in stochastic optimization.
Paper develops privacy-preserving federated learning for nonsmooth objectives.
This work uses Lasry-Lions envelopes to solve nonconvex optimization problems.
Improved method reduces projection calls for nonsmooth convex optimization.
We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPLA)---for sampling from a log concave distribution. Our method is a generalization of the Langevin algorithm to potentials expressed as the sum of one stochastic smooth term and multiple stochastic nonsmooth terms. In each iteration, our splitting t…
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self…
We propose an adaptive smoothing algorithm based on Nesterov's smoothing technique in \cite{Nesterov2005c} for solving "fully" nonsmooth composite convex optimization problems. Our method combines both Nesterov's accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriat…
Optimizes CM for stochastic convex optimization with progressive precision.
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
We propose a communication- and computation-efficient distributed optimization algorithm using second-order information for solving ERM problems with a nonsmooth regularization term. Current second-order and quasi-Newton methods for this problem either do not work well in the distributed setting or work only for specif…
A characterization of the proximal normal cone is obtained and a separation theorem for convex subsets of Riemannian manifolds is established. Moreover, the convexity of the distance function for a convex subset in the cases where the boundary of contains a geodesic segment, the boundary of is o…
Proximal methods avoid local minima in weakly convex problems.
Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.
On a Riemannian manifold, lower Ricci curvature bounds are known to be characterized by geodesic convexity properties of various entropies with respect to the Kantorovich-Rubinstein-Wasserstein square distance from optimal transportation. These notions also make sense in a (nonsmooth) metric measure setting, where they…
Unified algorithm solves convex optimization problems with optimal rates.
Improved analysis for clipped gradient methods in nonsmooth convex optimization under heavy-tailed noise.
DoWG optimizer automatically adapts to convex and nonsmooth problems without tuning.
Composite convex optimization problems which include both a nonsmooth term and a low-rank promoting term have important applications in machine learning and signal processing, such as when one wishes to recover an unknown matrix that is simultaneously low-rank and sparse. However, such problems are highly challenging t…
Due to their simplicity and excellent performance, parallel asynchronous variants of stochastic gradient descent have become popular methods to solve a wide range of large-scale optimization problems on multi-core architectures. Yet, despite their practical success, support for nonsmooth objectives is still lacking, ma…
Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for …
In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. …
New algorithms reduce regret in online convex optimization with heavy-tailed gradients.
New algorithm solves nonconvex-convex minimax problems efficiently.
New approach to gravity theory sacrifices smoothness for ellipticity.
Paper tackles efficient SVM classification over decentralized networks.
New theory for nonsmooth systems helps optimize and control complex functions.
We give several versions of local and global inverse mapping theorem for tame non necessarily smooth, mappings. Here tame mapping means a mapping which is subanalytic or, more generally, definable in some o-minimal structure. Our sufficient conditions are formulated in terms of various properties (convexity, positivity…
As surrogate functions of -norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…