The paper examines hyperbolicity in bounded strongly minimally convex domains in R^d.
problem Investigating hyperbolicity in bounded strongly minimally convex domains.
method Analyzing the minimal metric and Hilbert metric in convex domains.
result Every bounded strongly minimally convex domain is Gromov hyperbolic.
Sharp estimates for Finsler metrics in convex domains.
problem Estimating distances in Finsler metrics near convex points.
method Sharp estimates for intrinsic distances of Finsler metrics.
result Characterization of k-quasi hyperbolic metric in convex geometry. We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex prob…
Improved CBCD-type methods converge faster for strongly convex minimization problems.
problem Achieving faster convergence for strongly convex minimization problems using CBCD-type methods.
method Proposed an improved convergence analysis for CBCD-type methods, showing a tighter iteration complexity.
result Iteration complexity of O(log2(p)⋅log(1/ε)) matches GD methods up to a log2(p) factor. New method solves non-strongly convex optimization problems without quadratic regularization.
problem Minimizing the sum of an average of smooth convex components and a non-differentiable convex function.
method Accelerated randomized mirror descent algorithm without strongly convex assumption.
result Performance of algorithms improved without quadratic regularization.
New method improves mini-batch efficiency for convex optimization problems.
problem Efficiently solving convex regularized empirical risk minimization with mini-batches.
method Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method with double acceleration and variance reduction.
result Significantly reduces the number of mini-batches needed for optimal performance.
New algorithms minimize dynamic regret for strongly convex losses.
problem Minimizing dynamic regret for strongly convex losses.
method Developed Strongly Adaptive algorithms exploiting KKT conditions.
result Achieved near optimal dynamic regret of O(d1/3n1/3extTV[u1:n]2/3∨d). In this paper, a correspondence via duality is established between the set of locally strongly convex symmetric equiaffine hyperspheres and the set of minimal symmetric Lagrangian submanifolds in a certain complex space form. By using this correspondence theorem, we are able to provide an alternative proof of the class…
Optimizes CM for stochastic convex optimization with progressive precision.
problem Stochastic nature of objective function in convex optimization.
method Iterative coordinate minimization with optimal precision control.
result Order-optimal regret performance for strongly convex and nonsmooth functions.
A novel distributed method tracks gradients for convex optimization over networks.
problem Distributed optimization of strongly-convex functions over a network.
method S-AB algorithm using auxiliary variables and row/column stochastic weights.
result Linear convergence to a neighborhood of the global minimizer.
The paper proves Gromov hyperbolicity of certain metrics using isoperimetric inequalities.
problem Investigating Gromov hyperbolicity of specific metrics.
method Using isoperimetric inequalities to characterize Gromov hyperbolicity.
result Characterization of domains where these metrics are Gromov hyperbolic.
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…
New method gives high confidence bounds for stochastic convex optimization with minimal overhead.
problem Rare high probability guarantees in stochastic convex optimization.
method ProxBoost algorithm combining robust distance estimation and proximal point method.
result Wide class of stochastic optimization algorithms can achieve high confidence bounds with logarithmic and polylogarithmic overhead.
Drago optimizes DRO problems with faster convergence.
problem Distributionally robust optimization with closed, convex uncertainty sets.
method Primal-dual coupled variance reduction algorithm with cyclic and randomized updates.
result Achieves state-of-the-art linear convergence rate on strongly convex-strongly concave problems.
New methods accelerate gradient descent for convex and strongly convex functions.
problem Improving convergence rates of gradient-based optimization methods.
method Formulated two classes of first-order algorithms with Lyapunov analyses and Hamiltonian assisted gradient method.
result Achieved accelerated convergence rates matching Nesterov's methods in strongly and general convex settings.
Detecting convexity in polynomials over boxes is NP-hard even for degree 3.
problem Detecting convexity in polynomials over compact regions, especially boxes.
method Proof by reduction to the NP-hard problem of testing global convexity of polynomials of degree four.
result The problem of testing convexity over a box is strongly NP-hard even for polynomials of degree 3.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.
Dual-based algorithms optimize distributed convex problems over networks.
problem Optimizing distributed convex problems over network constraints.
method Dual formulation of primal problem, distributed algorithms achieving optimal rates.
result Achieves optimal rates similar to centralized algorithms with additional cost related to network spectral properties.
Almost all local minima in neural networks are strongly convex.
problem The prevalence of strongly convex neighborhoods around local minima in neural network optimization landscapes.
method Rigorous analysis of shallow neural networks with analytic activation functions, dividing parameter space into efficient and redundant domains.
result For shallow neural networks on the efficient domain, almost all local minima are strongly convex.
New methods reduce variance in stochastic dual averaging for sparse solutions.
problem Regularized empirical risk minimization problems in machine learning.
method Stochastic dual averaging with variance reduction for sparser solutions.
result Achieve best known convergence rates for both strongly and non-strongly convex regularizers.
New averaging strategy achieves optimal convergence rate with high probability.
problem Optimizing convergence rate for strongly-convex functions.
method Simple non-uniform averaging strategy combined with Freedman's inequality.
result Achieves optimal O(1/T) convergence rate with high probability. Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.
problem Solving stochastic min-max problems with strong convexity and strong concavity.
method Epoch-wise stochastic gradient descent ascent method (Epoch-GDA) without additional assumptions.
result Achieves the optimal rate of O(1/T) for the duality gap of general SCSC min-max problems. Investigates properties of a pseudometric on domains in Euclidean space, linking it to hyperbolic geometry.
problem Defines and analyzes a pseudometric on domains in Rn to understand their hyperbolic properties. method Introduces a pseudometric based on conformal harmonic discs and studies its properties and conditions for hyperbolicity.
result Characterizes domains as hyperbolic based on their geometric properties and provides sufficient conditions for hyperbolicity.
Paper closes convergence gap for SGD without replacement.
problem Establishing convergence rate for SGD without replacement.
method Analyzes convergence rates for strongly convex and smooth functions.
result Achieves a rate of O(1/T^2 + n^2/T^3) for quadratic sums.
Paper analyzes regret bounds for unconstrained online optimization.
problem Minimizing regret in dynamic online learning for strongly convex and smooth functions.
method Preconditioned OGD, Online Optimistic Newton (OON), multiple gradient queries.
result Achieves O(C2,T∗) regret bound with one gradient query per round. Characterizes symmetric Bernoulli distributions with minimal convex sums.
problem Understanding minimal dependence among Bernoulli random vectors.
method Geometric and algebraic representations of multivariate symmetric Bernoulli distributions.
result Characterizes extremal negative dependence and builds minimal dependence copulas.
Strongly convex bodies can be approximated by smooth ones.
problem Approximating strongly convex bodies with smooth ones.
method Using C2 locally strongly convex bodies. result Smooth approximations of strongly convex bodies exist and can be controlled in terms of Hausdorff distance.
Let U⊆Rn be open and convex. We show that every (not necessarily Lipschitz or strongly) convex function f:U→R can be approximated by real analytic convex functions, uniformly on all of U. In doing so we provide a technique which transfers results on uniform approximation on bounded …
Study optimizes zero-order strongly convex function minimization with higher order smoothness.
problem Optimizing a strongly convex function with noisy evaluations.
method Randomized approximation of projected gradient descent with smoothing kernel.
result Upper bounds and minimax lower bounds for the algorithm, showing near-optimality.
Paper solves robust convex problems with heavy-tailed noise.
problem Solving convex compositional problems with heavy-tailed noise.
method Sub-Gaussian confidence bounds under weak heavy-tailed noise assumptions, using boosting strategy.
result Achieves nearly optimal high probability convergence result.
Improved dynamic regret analysis for strongly convex and smooth functions.
problem Analyzing dynamic regret for online learning algorithms.
method Improved analysis of the Online Multiple Gradient Descent (OMGD) algorithm.
result Achieved a best-of-three-worlds guarantee for dynamic regret.
Paper develops algorithms for solving non-convex non-concave problems with applications in GAN training.
problem Solving non-convex non-concave min-max saddle-point problems.
method Inexact proximal point method with strongly monotone mappings.
result First-order convergence to a nearly stationary solution of the original min-max problem.
Improved SHB method for faster convergence on strongly-convex quadratics.
problem Understanding and improving the theoretical and practical advantages of SHB.
method Noise-adaptive multi-stage algorithm for SHB with accelerated convergence.
result SHB can achieve accelerated convergence with larger mini-batch sizes.
New algorithm solves complex optimization problems with two regularization terms efficiently.
problem Complex optimization problems with two regularization terms, especially composed with linear functions.
method Stochastic Primal-Dual Proximal ExtraGradient descent (SPDPEG) for convex and strongly convex objectives.
result Converges with rates matching best first-order stochastic algorithms.
The paper simplifies strongly convex problems to simplicial structures.
problem Strongly convex problems with Cr smoothness. method Generic linear perturbations and singularity theory.
result Strongly convex C1 problems are C0 simplicial. Extends geometric descent method for convex composite problems.
problem Nonsmooth and strongly convex composite problems.
method Geometric Proximal Gradient Method (GeoPG)
result Achieves optimal linear convergence rate of (1-1/\sqrtκ).
Paper improves privacy in ERM with faster algorithms and broader applicability.
problem Privacy-preserving machine learning with empirical risk minimization.
method Develops faster algorithms for differentially private ERM in various settings.
result Achieves optimal or near-optimal utility bounds with less gradient complexity.
The paper characterizes complex Finsler metrics that are projectively flat or dually flat.
problem Characterizing complex Finsler metrics with specific geometric properties.
method Proving conditions for projective flatness and dually flatness in terms of Minkowski metrics.
result Strongly convex complex Finsler metrics are projectively flat or dually flat if and only if they come from Minkowski metrics.
New algorithm optimally minimizes convex functions with noisy gradients.
problem Minimizing strongly convex, smooth functions with noisy gradient estimates.
method A multistage accelerated stochastic gradient method with restarts.
result Achieves optimal convergence rate in deterministic and stochastic cases.
Push-SAGA is a decentralized algorithm for directed graphs that converges linearly.
problem Finite-sum minimization over directed graphs with stochastic gradients.
method Combines variance reduction, gradient tracking, and consensus algorithms.
result Achieves linear convergence for smooth and strongly convex problems.
This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…
Two new algorithms optimize decentralized convex optimization with reduced communication rounds.
problem Decentralized minimization of smooth strongly convex functions in a network.
method Proposes two new algorithms based on accelerated Forward Backward methods.
result First algorithm is optimal in terms of communication rounds and gradient computations.
VR-SGD is a simple method for machine learning that uses larger learning rates and averages.
problem Efficiently solving machine learning problems with large datasets.
method A simple variant of SVRG with specific averaging and update rules.
result VR-SGD achieves linear convergence for strongly convex problems and similar performance to momentum methods for non-strongly convex problems.
Improved AG methods for noisy convex optimization.
problem Designing robust algorithms for minimizing smooth strongly convex functions with noisy gradients.
method Developed AG methods with improved robustness to gradient errors.
result AG methods achieve better robustness and faster convergence to optimal solutions.
Paper extends SMM to weakly convex and multi-convex surrogates for non-convex optimization.
problem Non-convex optimization with weakly convex or multi-convex surrogates.
method Stochastic majorization-minimization with proximal regularization or block-minimization.
result Convergence rates for empirical and expected losses under non-i.i.d. data.
New algorithms achieve optimal performance in online convex optimization with strong convexity and squared ℓ2 norms.
problem Optimal performance in online convex optimization with specific cost structures.
method Proposed and analyzed new algorithms (G-OBD, R-OBD) with theoretical guarantees.
result G-OBD and R-OBD achieve optimal competitive ratios of O(m−1/2) under specific conditions. New algorithms exploit data's strong convexity for fast linear convergence without explicit regularization.
problem Empirical risk minimization with convex loss functions.
method Primal-dual first-order algorithms that exploit data's strong convexity.
result Adaptive primal-dual algorithms achieve linear convergence without explicit regularization.
Universal algorithm minimizes adaptive regret for various convex functions.
problem Minimizing adaptive regret in changing environments for multiple convex functions.
method Borrowing MetaGrad's idea of multiple learning rates and using sleeping experts.
result First universal algorithm for minimizing adaptive regret of convex functions.