Paper develops privacy-preserving federated learning for nonsmooth objectives.
problem Solving nonsmooth objective functions in a privacy-preserving manner.
method Zero-concentrated differential privacy (zCDP) with Gaussian noise, distributed ADMM, and approximation of augmented Lagrangian.
result The algorithm achieves a competitive privacy-accuracy trade-off and converges to the exact solution.
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
problem Nonsmooth and nonconvex objectives in statistical learning.
method Bregman-surrogate algorithm framework, including local linear approximation, mirror descent, iterative thresholding, DC programming.
result Global convergence rates for nonconvex and nonsmooth objectives in high dimensions.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
problem Nonconvex and nonsmooth optimization problems in statistics and machine learning.
method Projective Proximal Gradient Descent (PPGD) for solving a class of nonconvex and nonsmooth problems.
result PPGD achieves a fast convergence rate of O(1/k^2) for k ≥ k_0.
New theory for nonsmooth systems helps optimize and control complex functions.
problem Optimizing and controlling systems with nonsmooth functions.
method Higher-order averaging theory with nonsmooth near-identity transformation and lexicographic differentiation.
result Closed formula for nonsmooth first and second-order averaging.
Two new methods solve nonsmooth optimization on Riemannian Stiefel manifold.
problem Optimization over nonsmooth, non-differentiable functions on Riemannian manifolds.
method R-ProxSGD and R-ProxSPB, generalizing proximal SGD and SpiderBoost.
result R-ProxSPB finds ε-stationary points with IFO complexity of Ø(ε^(-3)) in online and Ø(n + √nε^(-2)) in finite-sum cases.
We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …
A new method solves complex optimization problems with nonconvex and nonsmooth components.
problem Nonconvex nonsmooth optimization problems with coupled functions.
method Successive difference-of-convex approximation method using Moreau envelopes.
result The method generates bounded sequences with stationary points as accumulation points.
New algorithm for large-scale nonsmooth convex optimization with robust convergence.
problem Minimizing the average of many nonsmooth and convex functions in machine learning.
method Developed a new algorithm called Randomized Smoothing SVRG that achieves robust linear convergence.
result Achieves robust linear convergence rate and superior time and gradient complexity compared to state-of-the-art methods.
A new method solves nonsmooth nonconvex optimization problems with noisy gradients.
problem Solving nonsmooth nonconvex optimization problems with noisy gradient information.
method Globalized stochastic semismooth Newton method combining semismooth Newton steps and proximal gradient steps.
result The method converges globally to stationary points in expectation and locally r-superlinearly.
The paper develops a convergence framework for inexact nonconvex and nonsmooth algorithms.
problem Tackles convergence of inexact nonconvex and nonsmooth algorithms.
method Promises pseudo sufficient descent and relative error conditions, and assumes continuity and Kurdyka-Lojasiewicz property.
result Proves the convergence of algorithms to critical points under specific conditions.
New algorithms solve large-scale low-rank and nonsmooth optimization problems efficiently.
problem Solving large-scale composite convex optimization problems with nonsmooth and low-rank terms.
method Stochastic optimization algorithms combining variance reduction and weak proximal oracle.
result First algorithm with nearly optimal sample complexity, single low-rank SVD per iteration, and log 1 / ε \log{1/ε} log 1/ ε thin-SVD computations. New method solves nonsmooth low-rank matrix optimization problems efficiently.
problem Nonsmooth and low-rank matrix optimization problems in statistics and machine learning.
method Low-rank Extragradient Method with warm-start initialization.
result The extragradient method converges to an optimal solution with rate O ( 1 / t ) O(1/t) O ( 1/ t ) and requires only two low-rank SVDs per iteration. ProxASAGA solves nonsmooth optimization problems faster than existing methods.
problem Lack of scalable parallel methods for nonsmooth optimization problems.
method ProxASAGA, a fully asynchronous sparse method inspired by SAGA.
result ProxASAGA achieves linear speedup with respect to sequential version under certain assumptions.
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and feasibility gap. Through theoretical analysis, we show how to update all the al…
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
problem Minimizing nonsmooth nonconvex functions with constraints.
method Developed a unified framework for Lagrangian-based methods using subgradient updates.
result Global convergence guarantees for the proposed framework under mild conditions.
A new method solves optimization problems over Stiefel manifold.
problem Optimization problems over the Stiefel manifold with smooth and nonsmooth components.
method Proximal gradient method based on retraction.
result The method globally converges to a stationary point and has iteration complexity.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
This work establishes uniform convergence of subdifferentials in stochastic optimization.
problem Understanding how empirical stationary points approximate population ones in nonsmooth, nonconvex stochastic optimization.
method Reduction principle for weakly convex stochastic objectives, focusing on subgradient convergence.
result Sharp uniform convergence rates for subdifferential mappings in stochastic convex-composite optimization.
New Frank-Wolfe algorithm solves nonsmooth convex problems quickly.
problem Nonsmooth convex optimization problems in machine learning.
method Deterministic Frank-Wolfe algorithm with coreset guarantees.
result Approximate solutions to various machine learning problems in sublinear time.
Nesterov's extrapolation improves convergence in nonsmooth optimization.
problem Improving convergence rate in nonsmooth convex optimization.
method Nesterov's extrapolation applied to projected subgradient methods.
result Nesterov's extrapolation optimizes individual convergence for nonsmooth problems.
New adaptive methods solve weakly convex stochastic optimization problems.
problem Solving weakly convex stochastic optimization problems.
method Adaptive first and zeroth-order methods using exponential moving averages.
result Established non-asymptotic convergence rates for nonsmooth and nonconvex problems.
Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.
problem Differentially private optimization of nonconvex, nonsmooth objectives.
method Proposes single-pass and multi-pass DP algorithms with improved sample complexity.
result Sample complexity bounds improved by factors of Ω ( d ) Ω(\sqrt{d}) Ω ( d ) and Ω ( d 3 / 4 ) Ω(d^{3/4}) Ω ( d 3/4 ) . New method improves convergence rates for convex optimization problems.
problem Convex optimization with nonsmooth objectives.
method Randomized coordinate descent with smoothing, acceleration, homotopy, and non-uniform sampling.
result First convergence rate guarantees for coordinate descent methods.
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self…
Develops a new method for solving nonsmooth nonconvex optimization problems.
problem Solving nonsmooth nonconvex composite optimization problems with noisy gradient information.
method Combines stochastic higher order steps and additional stochastic proximal gradient steps.
result Global convergence to stationary points in expectation with favorable performance on large-scale problems.
Cyclic coordinate descent identifies models in finite time and converges linearly.
problem Model identification in composite nonsmooth optimization problems.
method Cyclic coordinate descent for a wide class of functions.
result Explicit local linear convergence rates for coordinate descent.
Paper tackles efficient SVM classification over decentralized networks.
problem Efficiently classifying high-dimensional data over decentralized networks.
method Convolution-based smoothing technique for nonsmooth hinge loss function, combined with an efficient ADMM algorithm.
result Provable linear convergence of the ADMM algorithm and near-optimal statistical convergence of the sparse estimator.
In this paper, we study the stochastic gradient descent (SGD) method for the nonconvex nonsmooth optimization, and propose an accelerated SGD method by combining the variance reduction technique with Nesterov's extrapolation technique. Moreover, based on the local error bound condition, we establish the linear converge…
A new method solves convex optimization problems on manifolds efficiently.
problem Optimization on Hadamard manifolds with convex objectives.
method Intrinsic Riemannian proximal gradient method.
result Sublinear and linear convergence rates for convex and strongly convex problems, respectively.
Study on Adam-family methods for nonsmooth optimization with convergence guarantees.
problem Training nonsmooth neural networks with convergence guarantees.
method Two-timescale updating scheme and stochastic subgradient methods with gradient clipping.
result Convergence guarantees for various Adam-family methods in training nonsmooth neural networks.
This paper tackles nonsmooth optimization in machine learning.
problem Nonsmoothness in machine learning optimization problems.
method Identifying specific structures and leveraging them for practical applications.
result Compression, acceleration, and dimension reduction are possible with nonsmooth optimization.
Researchers found nonsmoothable actions of Z2 * Z2 on spin 4-manifolds.
problem Understanding nonsmoothable actions of Z2 * Z2 on spin 4-manifolds.
method Used an equivariant version of Furuta's 10/8 inequality to construct examples.
result Examples show that any proper subgroup of Z2 * Z2 is smoothable for some smooth structure.
Study on nonsmooth contractive SA with constant stepsize and Q-learning.
problem Understanding convergence and bias in nonsmooth contractive SA with different noise types.
method Proposed prelimit coupling technique for steady-state convergence and derived asymptotic bias.
result Asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize.
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence ra…
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size N N N after O ( 1 N ε ) \mathcal{O}(\frac{1}{Nε}) O ( N ε 1 ) iterations achieves ε − ε- ε − suboptimality in expected quadratic distance. Study compares nonsmooth spaces with integrable Ricci bounds.
problem Comparing geometric and functional inequalities on nonsmooth spaces.
method Localization method and one-dimensional comparison estimates.
result Extension of comparison principles to nonsmooth settings.
New stability bounds for SGD on nonsmooth convex losses.
problem Understanding stability of SGD on nonsmooth convex losses.
method Sharp upper and lower bounds for SGD and full-batch GD on nonsmooth convex losses.
result SGD can be less stable but still useful for generalization bounds.
New algorithm solves nonconvex-convex minimax problems efficiently.
problem Solving nonconvex-convex minimax problems with nonsmooth, nonconvex, and nonlinearity.
method Hybrid variance-reduced SGD algorithm combining smoothing and biased techniques.
result Achieves O(T^(-2/3)) convergence rate and best oracle complexity.
Improved Frank-Wolfe method tackles nonsmooth functions.
problem Efficiently solving large nonsmooth problems with sparse structures.
method Optimizes for approximation quality over all affine approximations.
result Overcomes issues with existing nonsmooth methods in low-rank matrix estimation.
Regularity results for geodesic X-ray transform on nonsmooth manifolds
problem Geodesic X-ray transform on nonsmooth simple manifolds
method Symbol smoothing arguments and pseudodifferential operators with low regularity symbols
result Improved injectivity results for L p L^p L p functions New methods help escape strict saddle points in nonsmooth optimization.
problem Escaping strict saddle points in nonsmooth optimization.
method An inexact stochastically perturbed gradient method applied to the Moreau envelope.
result A variety of algorithms for nonsmooth optimization can efficiently escape strict saddle points of the Moreau envelope.
A new method combines extrapolation and line search for solving nonconvex, nonsmooth optimization problems.
problem Nonconvex, nonsmooth optimization problems in machine learning and image processing.
method Proximal gradient method with extrapolation and line search (PGels).
result The method reduces to existing algorithms under proper parameter choices and converges to stationary points.
We extend the well-known BFGS quasi-Newton method and its memory-limited variant LBFGS to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: the local quadratic model, the identification of a descent direction, and the Wolfe …
We propose an algorithm for the non-negative factorization of an occurrence tensor built from heterogeneous networks. We use l0 norm to model sparse errors over discrete values (occurrences), and use decomposed factors to model the embedded groups of nodes. An efficient splitting method is developed to optimize the non…
Unified algorithm solves convex optimization problems with optimal rates.
problem Solving nonsmooth constrained convex optimization problems.
method Unified randomized block-coordinate primal-dual algorithm.
result Achieves optimal convergence rates of O ( n / k ) \mathcal{O}(n/k) O ( n / k ) and O ( n 2 / k 2 ) \mathcal{O}(n^2/k^2) O ( n 2 / k 2 ) . Proposes BMME for optimizing nonsmooth nonconvex problems with block structure.
problem Optimizing nonsmooth nonconvex problems with block structure.
method Block Alternating Bregman Majorization Minimization with Extrapolation (BMME).
result Subsequential convergence to a first-order stationary point under mild assumptions, global convergence under stronger conditions.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
Improved method reduces projection calls for nonsmooth convex optimization.
problem Optimizing nonsmooth convex functions with convex constraints.
method MOPES and MOLES methods combining Moreau-Yosida smoothing and accelerated first-order schemes.
result Achieves ε ε ε -suboptimality with significantly fewer projection calls.