Study currents from semi-convex functions, apply to Hessian measures.
problem Understanding currents from semi-convex functions.
method Analyze integer multiplicity rectifiable currents from subgradient graphs of semi-convex functions.
result Weak continuity theorem for currents with pointwise convergence.
Graph-dependent implicit regularisation improves Distributed SGD for convex problems.
problem Improving convergence rates in distributed stochastic subgradient descent.
method Graph-dependent implicit regularisation strategies for Distributed SGD.
result Established statistical learning rates retaining centralised guarantees.
New method bounds stochastic subgradient methods with heavy-tailed noise.
problem Bounding stochastic subgradient methods under heavy-tailed noise.
method Clipped version of projected stochastic subgradient method.
result Near optimal any-time and finite horizon bounds for averaging schemes.
The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.
problem Minimizing nonsmooth nonconvex functions with convergence guarantees.
method Developed a framework for stochastic subgradient methods with global stability guarantees.
result Iterates are uniformly bounded and asymptotically stabilize around the stable set of the differential inclusion.
Subgradient algorithm achieves optimal regret for both adversarial and i.i.d. costs on the simplex.
problem Achieving optimal regret for both adversarial and i.i.d. costs on the simplex.
method Demonstrates the universality of the Subgradient algorithm for online learning on the simplex.
result Shows simultaneous O ( N ) O(\sqrt N) O ( N ) regret for adversarial costs and O ( 1 ) O(1) O ( 1 ) pseudo-regret for i.i.d. costs. A distributed subgradient method tackles non-convex optimization problems in networks.
problem Solving non-convex optimization problems in distributed networks.
method Proposes a distributed stochastic subgradient method (stoDPSM) with theoretical guarantees.
result Global convergence of stoDPSM using Moreau envelope stationarity measure, and linear convergence under sharpness condition.
Improved subgradient method tackles ill-conditioned composite optimization problems.
problem Slow convergence of subgradient method for composite optimization problems.
method Preconditioned subgradient method with Levenberg-Marquardt approach.
result Linear convergence rate for composite optimization problems under mild conditions.
New Max-Plus neural network exploits subgradient sparsity for efficient training.
problem Training Max-Plus neural networks is challenging due to dense subgradients.
method Proposes a sparse subgradient algorithm tailored to Max-Plus models.
result Achieves more efficient updates while retaining theoretical guarantees.
The paper derives subgradient estimates for a specific nonlinear subparabolic equation on pseudo-Hermitian manifolds.
problem Deriving subgradient estimates for positive solutions to a nonlinear subparabolic equation on pseudo-Hermitian manifolds.
method Using the CR sub-Laplacian comparison property, the paper derives local subgradient estimates for positive solutions to the given equation.
result The paper establishes subgradient estimates for positive solutions to the nonlinear subparabolic equation.
New algorithm samples from complex non-convex distributions.
problem Sampling from non-convex, non-smooth distributions with superlinear gradients.
method Subgradient Tamed Unadjusted Langevin Algorithm (SG-TULA)
result Non-asymptotic convergence bounds in Wasserstein-2 distance for SG-TULA.
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
problem Proving convergence in multi-objective optimization problems.
method Wasserstein inverse reinforcement learning with projective subgradient method and gradient descent.
result Convergence of inverse reinforcement learning for multi-objective optimization.
Subgradient descent learns orthogonal dictionaries efficiently.
problem Sparse coding and dictionary learning for data representation.
method Subgradient descent algorithm with random initialization.
result Subgradient descent can recover orthogonal dictionaries under mild assumptions.
Inexact subgradient methods work well for semialgebraic functions with additive errors.
problem Approximate gradients in machine learning and optimization.
method Inexact subgradient methods with persistent additive errors in semialgebraic functions.
result Iterates eventually fluctuate near the critical set with a proximity of O ( ε ρ ) O(ε^ρ) O ( ε ρ ) , where ε ε ε is the magnitude of subgradient evaluation errors. A new GAN training method using primal-dual subgradient methods.
problem Training GANs to avoid mode collapse and generate diverse samples.
method Relating GANs to convex optimization via Lagrangian perspective and primal-dual subgradient methods.
result The proposed method resolves mode collapse and generates diverse samples.
Unified Lagrangian-based methods for nonsmooth nonconvex optimization.
problem Minimizing nonsmooth nonconvex functions with constraints.
method Developed a unified framework for Lagrangian-based methods using subgradient updates.
result Global convergence guarantees for the proposed framework under mild conditions.
New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.
problem Achieving high-probability parameter-free regret in online convex optimization with heavy-tailed data.
method Developed new regularization techniques to handle exponentially large iterates and heavy-tailed subgradients.
result Achieved regret bound of O ( ∥ u ∥ T 1 / p log ( 1 / δ ) ) O(\| \mathbf{u} \| T^{1/\mathfrak{p}} \log (1/δ)) O ( ∥ u ∥ T 1/ p log ( 1/ δ )) with high probability for subgradients with bounded p t h p^{th} p t h moments. Study proves convergence of subgradients for optimal transport-based objectives.
problem Ensuring statistical consistency and optimization stability in transport-based models.
method Proves graphical convergence of subdifferentials to the subdifferential of the population objective.
result Standard subgradient methods consistently approach stationary points of the population-level problem.
We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…
Develops methods for solving convex optimization problems with improved accuracy and convergence.
problem Solving stochastic convex optimization problems with improved accuracy and robustness.
method Approximate-proximal point (aProx) family, including stochastic subgradient, proximal point, and bundle methods.
result Improved models converge with probability 1 and enjoy optimal asymptotic normality results under weak assumptions.
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
problem Improving convergence rates of ISTA and FISTA for composite optimization.
method Improved proximal subgradient norm minimization using Lyapunov function.
result Convergence rates of ISTA and FISTA are accelerated.
Study on portfolio optimization and risk analysis, proving non-uniqueness and suggesting a method to resolve it.
problem Non-uniqueness in solution of portfolio optimization and risk analysis problems.
method Proof of non-uniqueness, introduction of Stainer point as a unique subgradient.
result Identification of a unique 'special' subgradient to resolve non-uniqueness in portfolio optimization and risk analysis.
New algorithms solve large-scale convex regression problems.
problem Large-scale convex regression with subgradient regularization.
method Active set type algorithm on dual QP, approximate optimization, randomized augmentation.
result Solves problems with n=10^5 and d=10 in minutes.
Study robust recovery of low-rank matrices from corrupted measurements without rank prior.
problem Robust recovery of low-rank matrices from corrupted Gaussian measurements with unknown rank.
method Subgradient method with diminishing stepsizes for nonconvex nonsmooth problem.
result Subgradient method converges to exact low-rank solution at sublinear rate under RDPP condition.
New theory accelerates stochastic optimization by leveraging local growth rate.
problem First-order stochastic convex optimization convergence rate.
method Developed two accelerated stochastic subgradient methods.
result Optimal iteration complexity of O ( 1 / ε 2 ( 1 − θ ) ) O(1/ε^{2(1-θ)}) O ( 1/ ε 2 ( 1 − θ ) ) for achieving ε ε ε -optimal solution. We describe a new technique for computing lower-bounds on the minimum energy configuration of a planar Markov Random Field (MRF). Our method successively adds large numbers of constraints and enforces consistency over binary projections of the original problem state space. These constraints are represented in terms of …
In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to…
Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampli…
Stochastic subgradient descent avoids critical points in definable functions.
problem Finding local minima in definable functions.
method Stochastic subgradient descent with density-like perturbation.
result SGD converges to a local minimum in definable functions.
Parameter-free online convex optimization with sub-exponential noise achieves optimal regret.
problem Online convex optimization with sub-exponential noise, especially when subgradients are unbounded.
method Designing a novel parameter-free algorithm BANCO via a reduction to betting on noisy coins.
result BANCO achieves the optimal regret rate in the problem of unconstrained online convex optimization with sub-exponential noise.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
New estimator avoids overfitting in convex regression.
problem Overfitting near boundaries in convex regression.
method Minimizes sum of squared errors with a penalty on subgradient.
result Uniform almost sure consistency and convergence rates established.
In this paper, we define the geometric median of a probability measure on a Riemannian manifold, give its characterization and a natural condition to ensure its uniqueness. In order to calculate the median in practical cases, we also propose a subgradient algorithm and prove its convergence as well as estimating the er…
Given a convex optimization problem and its dual, there are many possible first-order algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality, and implies notably that for certain problems, such…
Paper proves equivalences in portfolio optimization with new risk measures.
problem Portfolio optimization with novel risk measures.
method Derive subgradients and gradients for negative expectile and omega ratio.
result Negative expectile can be used as a portfolio optimization objective.
This work establishes uniform convergence of subdifferentials in stochastic optimization.
problem Understanding how empirical stationary points approximate population ones in nonsmooth, nonconvex stochastic optimization.
method Reduction principle for weakly convex stochastic objectives, focusing on subgradient convergence.
result Sharp uniform convergence rates for subdifferential mappings in stochastic convex-composite optimization.
New algorithms optimize spectral risk measures, improving interpolation between average and worst-case performance.
problem Optimizing spectral risk measures for learning systems.
method Developed stochastic algorithms to optimize spectral risk measures by characterizing their subdifferential and addressing challenges like biasedness of subgradient estimates and non-smoothness.
result Our approach outperforms out-of-the-box stochastic subgradient and dual averaging methods in optimizing spectral risk measures.
Paper presents a robust estimator for density ratio estimation that trims outliers.
problem Vulnerability of density ratio estimation to corrupted data points.
method Automatically identifies and trims outliers in density ratio estimation; uses convex formulation and subgradient descent.
result Global optimum can be obtained via subgradient descent; parameter estimation error analyzed under high-dimensional settings.
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…
A new method solves convex optimization on curved spaces.
problem Optimization on curved spaces with non-smooth functions.
method Convex bundle method on Riemannian manifolds.
result The method converges to a minimizer under mild conditions.
New method estimates sparse mean from noisy data without knowing sparsity level.
problem Sparse mean estimation under adversarial corruptions.
method Incremental learning approach to nonconvex optimization.
result Achieves optimal statistical rate under moderate signal-to-noise ratio.
Efficient algorithms for clustered Lasso and OSCAR reduce computational costs.
problem High dimensional regression with feature clustering.
method Efficient path algorithms for clustered Lasso and OSCAR, reducing computational costs.
result Proposed algorithms are more efficient than existing methods in numerical experiments.
RSG method reduces subgradient method's complexity for convex optimization.
problem Finding optimal solutions for convex optimization problems efficiently.
method Periodically restarts the standard subgradient method to reduce complexity.
result RSG method finds ε ε ε -optimal solutions with lower complexity than SG. Parallel algorithm finds sparse solutions for nonconvex problems.
problem Nonconvex sparsity-regularized rank minimization.
method Parallel best-response algorithm with exact line search.
result Guaranteed convergence to a stationary point.
Paper proposes fast, robust methods for low-rank matrix recovery.
problem Estimating low-rank matrices from incomplete or corrupted data.
method Scaled subgradient methods for nonsmooth, nonconvex formulations.
result Methods converge almost dimension-free and condition-number independent.
In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FS ε _\varepsilon ε ) and least squares boosting (LS-Boost($…
Study on tensor nuclear norm's decomposability and subdifferential.
problem Understanding tensor nuclear norm in higher-order tensors.
method Showed decomposability over specific subspaces, derived subdifferential inclusions, and studied subgradients.
result Established the statistical performance of tensor robust principal component analysis.
New algorithm reduces privacy loss in SGD without learning rate tuning.
problem Locally differentially private stochastic optimization with high privacy loss.
method BANCO (Betting Algorithm for Noisy COins) for ε ε ε -LDP SGD. result Matches convergence rate of tuned SGD without learning rate tuning.
New method for optimization on Hadamard manifolds with curvature-independent guarantees.
problem Curvature-dependent complexity in geodesic convex optimization.
method Introducing horospherical convexity and developing algorithms for optimization.
result Curvature-independent convergence of subgradient descent and Nesterov's method.