arXiv research

Geodesic loops escape from balls at a sublinear rate imply virtually abelian fundamental group.

Open manifolds with nonnegative Ricci curvature have virtually abelian fundamental groups if they escape from bounded balls at a small rate.

SGD's escape rate depends on log loss barrier, not linear loss barrier.

Stochastic (sub)gradient methods require step size schedule tuning to perform well in practice. Classical tuning strategies decay the step size polynomially and lead to optimal sublinear rates on (strongly) convex problems. An alternative schedule, popular in nonconvex optimization, is called \emph{geometric step decay…

The Dirichlet random walk on manifolds has a positive escape rate if the cover is non-amenable.

New result on group actions in CAT(0) spaces with vanishing escape rate.

The paper explores curvature-free effects in manifolds with volume growth and ends-counting.

A new method helps escape saddle points in non-convex optimization.

New methods help escape strict saddle points in nonsmooth optimization.

We consider minimizing a nonconvex, smooth function $f$ on a Riemannian manifold $\mathcal{M}$. We show that a perturbed version of Riemannian gradient descent algorithm converges to a second-order stationary point (and hence is able to escape saddle points on the manifold). The rate of convergence depends as $1/ε^2$ o…

The paper analyzes how noise geometry influences the performance of SGD in machine learning.

The paper addresses pricing interest rate derivatives in markets with volatility uncertainty.

Improves training GANs by escaping limit cycles.

Study on optimal rates for sequential probability assignment using smoothed analysis.

This paper shows that a perturbed form of gradient descent converges to a second-order stationary point in a number iterations which depends only poly-logarithmically on dimension (i.e., it is almost "dimension-free"). The convergence rate of this procedure matches the well-known convergence rate of gradient descent to…

Paper generalizes extragradient methods for solving equations and inclusions with improved convergence rates.

We present the first tree-based regressor whose convergence rate depends only on the intrinsic dimension of the data, namely its Assouad dimension. The regressor uses the RPtree partitioning procedure, a simple randomized variant of k-d trees.

Gradient EM converges globally for over-parameterized Gaussian mixtures.

GP-UCB resolves sublinear regret for kernelized bandits.

New algorithms reduce communication for sparse mean estimation in noisy distributed systems.

Stochastic algorithm achieves sublinear convergence for bi-objective optimization.

This paper explains why Adam generalizes worse than SGD by analyzing its components.

Single-timescale actor-critic finds globally optimal policy.

We analyze the variance of stochastic gradients along negative curvature directions in certain non-convex machine learning models and show that stochastic gradients exhibit a strong component along these directions. Furthermore, we show that - contrary to the case of isotropic noise - this variance is proportional to t…

New algorithms tackle machine learning problems using manifold proximal point methods.

New convergence rates found for PnP methods using MMSE denoisers.

A new method solves convex optimization problems on manifolds efficiently.

The alternating gradient descent (AGD) is a simple but popular algorithm which has been applied to problems in optimization, machine learning, data ming, and signal processing, etc. The algorithm updates two blocks of variables in an alternating manner, in which a gradient step is taken on one block, while keeping the …

SALR improves deep learning generalization by dynamically adjusting learning rates.

In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties of this method, both in the exact and inexact setting, in the case when the obje…

The three operator splitting scheme was recently proposed by [Davis and Yin, 2015] as a method to optimize composite objective functions with one convex smooth term and two convex (possibly non-smooth) terms for which we have access to their proximity operator. In this short note we provide an alternative proof for the…

Classifies conformal transformations in spacetimes without observer horizons.

New algorithms reduce private bandit regret to nearly non-private levels.

The paper analyzes neural network dynamics after weights escape the origin.

This paper proposes a stochastic variant of a classic algorithm---the cubic-regularized Newton method [Nesterov and Polyak 2006]. The proposed algorithm efficiently escapes saddle points and finds approximate local minima for general smooth, nonconvex functions in only $\mathcal{\tilde{O}}(ε^{-3.5})$ stochastic gradien…

We shortly review the statistical properties of the escape times, or hitting times, for stock price returns by using different models which describe the stock market evolution. We compare the probability function (PF) of these escape times with that obtained from real market data. Afterwards we analyze in detail the ef…

Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.

New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.

Algorithm maximizes revenue-risk by estimating price impact kernel and optimizing control problems.

Gradient descent can take exponentially long to escape saddle points in 2D.

Federated learning algorithm improves with intermittent client availability.

The paper shows how sublinear biLipschitz equivalences affect Morse boundaries of metric spaces.

Local LMO optimizes constrained problems using local linear minimization.

In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general noncon…

New algorithm helps escape saddle points in optimization problems.

We consider the non-stochastic version of the (cooperative) multi-player multi-armed bandit problem. The model assumes no communication at all between the players, and furthermore when two (or more) players select the same action this results in a maximal loss. We prove the first $\sqrt{T}$-type regret guarantee for th…

The paper proves Zimmer's conjecture for non-uniform lattices by controlling mass escape and Lyapunov exponents.

Nesterov's accelerated gradient descent (AGD), an instance of the general family of "momentum methods", provably achieves faster convergence rate than gradient descent (GD) in the convex setting. However, whether these methods are superior to GD in the nonconvex setting remains open. This paper studies a simple variant…