Paper introduces ICGNs to model convex gradients.
problem Modeling convex gradients efficiently.
method Integrates Jacobian-vector product in a neural network.
result Single layer ICGN outperforms single layer ICNN in fitting.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “Convex gradients”
Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
We show that domains, that allow for convex functions with unbounded gradient at their boundary, are convex.
Adaptive methods improve gradient descent and proximal gradient for convex optimization.
problem Improving efficiency of gradient descent and proximal gradient methods.
method Adaptive versions of GD and ProxGD using local curvature information.
result Proved convergence with local Lipschitz gradient assumptions.
Stochastic gradient algorithms estimate the gradient based on only one or a few samples and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems. However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates, due to t…
Gradient flow converges to a minimal convex structure.
problem Finding the minimal convex structure in hyperbolic manifolds.
method Weil-Petersson gradient vector field of renormalized volume.
result The flow converges to the structure with minimum convex core volume.
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descentcs.LG
Unified framework for analyzing neural networks trained by gradient descent.
problem Lack of generalizable guarantees for neural networks trained by gradient descent.
method Proxy convexity and proxy Polyak-Lojasiewicz inequalities.
result Unified guarantees for neural networks trained by gradient descent.
New algorithm improves convergence for non-convex problems with boundaries.
problem Optimizing non-convex problems with constraints.
method Reflected Gradient Langevin Dynamics with probabilistic representation.
result Promising convergence rates, faster than existing methods.
Improves bandit convex optimization with gradient variations.
problem Bandit Convex Optimization with Gradient Variations.
method Refined analysis of non-consecutive gradient variation.
result Improved dimension dependence for convex and strongly convex functions.
This paper optimizes functions of probability measures using particle gradient descent for displacement convex functions.
problem Optimizing functions of probability measures with displacement convex properties.
method Particle gradient descent applied to displacement convex functions with theoretical guarantees.
result Finite number of particles and computations are sufficient to find optimal solutions for displacement convex functions.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.
Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimizationcs.LG
New algorithms ensure reproducibility and optimal convergence in convex optimization.
problem Trade-off between reproducibility and convergence rate in convex optimization.
method Regularization-based algorithms for smooth convex minimization and minimax optimization.
result Achieves optimal reproducibility and near-optimal gradient complexity for various oracle settings.
A new method solves convex optimization problems on manifolds efficiently.
problem Optimization on Hadamard manifolds with convex objectives.
method Intrinsic Riemannian proximal gradient method.
result Sublinear and linear convergence rates for convex and strongly convex problems, respectively.
Accelerated first-order methods for convex optimization with locally Lipschitz continuous gradientmath.OC
New methods for convex optimization with locally Lipschitz gradient, achieving faster convergence.
problem Optimization problems with locally Lipschitz continuous gradient.
method Accelerated proximal gradient (APG) methods and proximal augmented Lagrangian method.
result Achieved faster convergence rates for convex optimization problems with locally Lipschitz gradient.
In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…
SGD and stochastic gradient descent converge at optimal rates for certain non-convex functions.
problem Optimal convergence rates for non-convex functions under gradient noise.
method Geometric interpretation of the PL-condition to analyze convergence rates.
result Convergence rates of SGD and stochastic gradient descent match those of strongly convex quadratics.
ICCNLS models complex relationships as convex and concave components.
problem Complex input-output relationships with affine ambiguity.
method Sub-gradient constrained affine functions, global orthogonality constraints, L1, L2, and elastic net regularisation.
result Improved predictive accuracy and model simplicity compared to conventional methods.
New methods accelerate gradient descent for convex and strongly convex functions.
problem Improving convergence rates of gradient-based optimization methods.
method Formulated two classes of first-order algorithms with Lyapunov analyses and Hamiltonian assisted gradient method.
result Achieved accelerated convergence rates matching Nesterov's methods in strongly and general convex settings.
SGD converges to global minimum for certain non-convex functions.
problem Theoretical challenges in optimizing non-convex functions in machine learning.
method Perturbed SGD on a broad class of non-convex functions.
result SGD converges to global minimum for certain non-convex functions.
Optimal neural network approximation for Wasserstein gradient direction via convex optimization.
problem Approximating Wasserstein gradient direction with limited data.
method Two-layer networks with squared-ReLU activations, SDP relaxation.
result Optimal approximation of Wasserstein gradient direction in two-layer networks.
dboost optimizes prediction models for convex cone problems.
problem Optimizing prediction models for decision-making.
method Gradient boosting with implicit differentiation for convex quadratic cone programming.
result dboost reduces out-of-sample decision regret.
Novel BSG method for efficient stochastic optimization.
problem Efficient optimization of non-convex surfaces in stochastic settings.
method Binary search combined with first order gradient optimization.
result BSG produces more promising results and better generalization than other methods.
Simple rules ensure gradient descent adapts to local geometry, converging for convex and nonconvex problems.
problem Minimizing convex and nonconvex functions efficiently.
method Two rules: don't increase stepsize too fast and don't overstep local curvature.
result Method converges for convex and nonconvex problems, even with infinite global smoothness.
We consider the convex-concave saddle point problem where is smooth and convex and is smooth and strongly convex. We prove that if the coupling matrix has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if is not stron…
A new algorithm improves convergence rates for convex optimization problems.
problem Convex optimization problems with finite-sum structure.
method Nesterov Accelerated Shuffling Gradient (NASG) integrating Nesterov's acceleration with different shuffling schemes.
result Improved convergence rate of O(1/T) for unified shuffling schemes.
No accelerated gradient method for hyperbolic convex functions.
problem Existence of accelerated gradient methods for geodesically convex functions on hyperbolic spaces.
method Analysis of volume growth in negatively curved spaces.
result No-go theorem for accelerated gradient methods on hyperbolic plane.
New adaptive step-size method for convex optimization without tuning.
problem Optimizing convex functions efficiently with stochastic gradients.
method Adapted Adaptive Gradient Descent Without Descent to stochastic setting.
result Stochastic gradient descent converges under various assumptions.
This work accelerates gradient descent with anytime convergence guarantees.
problem Improving the convergence rate of gradient descent methods.
method Proposes a stepsize schedule for gradient descent that achieves anytime convergence rates.
result Gradient descent can achieve convergence rates of for any stopping time .
Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexitymath.OC
New method circumvents non-convexity in bilevel RL via hyper-gradient.
problem Non-convexity in lower-level RL problems in bilevel reinforcement learning.
method Characterizing hyper-gradient via fully first-order information, circumventing convexity assumption.
result Developed model-based and model-free algorithms with convergence rate .
We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex prob…
We propose a projected semi-stochastic gradient descent method with mini-batch for improving both the theoretical complexity and practical performance of the general stochastic gradient descent method (SGD). We are able to prove linear convergence under weak strong convexity assumption. This requires no strong convexit…
New method finds near-optimal solutions for non-convex optimization problems.
problem Finding near-optimal solutions for non-convex optimization problems.
method Riemannian stochastic recursive momentum method
result Achieves a near-optimal complexity of .
This paper introduces Jensen, an easily extensible and scalable toolkit for production-level machine learning and convex optimization. Jensen implements a framework of convex (or loss) functions, convex optimization algorithms (including Gradient Descent, L-BFGS, Stochastic Gradient Descent, Conjugate Gradient, etc.), …
Gradient flow expands curves to round shapes.
problem Expanding curves to round shapes.
method Steepest descent L2-gradient flow of entropy.
result Flow converges to a round expanding circle for various initial curves.
Stochastic gradient descent is the method of choice for large scale optimization of machine learning objective functions. Yet, its performance is greatly variable and heavily depends on the choice of the stepsizes. This has motivated a large body of research on adaptive stepsizes. However, there is currently a gap in o…
Constructs retractions of CAT(1) spaces to convex subsets.
problem Geometric description of an analytic tool.
method Gradient flow of time-dependent locally Lipschitz semiconcave functions.
result Existence of gradient flows proved for independent interest.
Preconditioned non-convex gradient descent improves noisy matrix estimation.
problem Estimating low-rank matrices from noisy measurements.
method Preconditioned non-convex gradient descent for noisy measurements.
result Preconditioned method converges to minimax optimal estimate at a linear rate.
Lower bounds on queries needed for finding stationary points in non-convex optimization.
problem Finding -stationary points in non-convex stochastic optimization.
method Proving lower bounds on the number of queries required by stochastic first-order methods.
result Lower bounds on the number of queries required to find -stationary points are tight and optimal.
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimizationmath.OC
Improved shuffling gradient methods converge faster for nonsmooth convex optimization.
problem Improving convergence rates for nonsmooth convex optimization problems.
method Analysis of shuffling gradient methods, focusing on Random Reshuffle and Single Shuffle strategies.
result Shuffling gradient methods, particularly Random Reshuffle and Single Shuffle, converge faster than Proximal Gradient Descent for nonsmooth convex optimization.
Accelerated gradient method's stability deteriorates exponentially with steps.
problem Algorithmic stability of Nesterov's accelerated gradient method.
method Analysis of two notions of algorithmic stability for Nesterov's accelerated gradient method.
result Stability of Nesterov's accelerated method deteriorates exponentially with the number of gradient steps.
We generalize the theory of gradient flows of semi-convex functions on CAT(0)-spaces, developed by Mayer and Ambrosio--Gigli--Savaré, to CAT(1)-spaces. The key tool is the so-called "commutativity" representing a Riemannian nature of the space, and all results hold true also for metric spaces satisfying the commutativi…
AEGD optimizes non-convex functions with dynamic energy updates.
problem Optimizing non-convex functions efficiently and robustly.
method Adaptive Gradient Descent (AEGD) with a dynamically updated energy variable.
result AEGD achieves energy-dependent convergence rates for both convex and non-convex objectives.
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimizationmath.OC
Accelerated gradient (AG) methods are breakthroughs in convex optimization, improving the convergence rate of the gradient descent method for optimization with smooth functions. However, the analysis of AG methods for non-convex optimization is still limited. It remains an open question whether AG methods from convex o…
Improved convergence for Polyak steps with momentum in smooth convex optimization.
problem Optimizing smooth strongly convex functions with limited information.
method Polyak steps with momentum, derived for accelerated gradient method.
result Convergence guarantees for accelerated gradient method with Polyak steps and momentum.
The paper tackles safe reinforcement learning with convex regularization.
problem Safe reinforcement learning in complex, high-dimensional settings with safety constraints.
method Doubly-regularized RL framework combining reward and parameter regularization, formulated as a convex regularized objective with parametrized policies on an infinite-dimensional statistical manifold.
result Exponential convergence guarantees under sufficient regularization, robust theoretical insights and guarantees for safe RL.
Generalized gradient flows in Hadamard manifolds and convex optimization on entanglement polytopesmath.OC
Paper addresses optimization on Hadamard manifolds, generalizing gradient flow.
problem Optimization of convex functions on Hadamard manifolds.
method Introduces a generalized gradient flow to minimize .
result Gradient flow attains infimum in limit for basic manifolds.
Unified analysis for shuffling-type gradient methods in optimization.
problem Optimization of finite-sum problems using shuffling strategies.
method Unified convergence analysis for various shuffling methods.
result Improved convergence rates for nonconvex problems and matching rates for convex problems.
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole objective function is strongly convex. Such problems often arise in machine lea…