arXiv research

New findings show Bregman proximal algorithms can get stuck near non-stationary points.

We analyze stochastic gradient algorithms for optimizing nonconvex problems. In particular, our goal is to find local minima (second-order stationary points) instead of just finding first-order stationary points which may be some bad unstable saddle points. We show that a simple perturbed version of stochastic recursiv…

New method finds points for approximating distributions faster.

Polyconvex energies with conformal invariance have smooth stationary points outside a discrete set.

Develops a deep non-stationary kernel for non-stationary spatio-temporal point processes.

New method finds stationary points in bilevel optimization problems.

New algorithm finds approximate stationary points faster under differential privacy constraints.

This paper tackles the computational complexity of finding approximate stationary points in non-convex optimization.

The paper tackles finding stationary points in stochastic convex optimization problems.

Neural networks' weights don't converge to stationary points but training loss stabilizes.

Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima…

In this article we study the regularity of stationary points of the knot energies $E^α$ introduced by O'Hara in the range $α\in (2,3)$. In a first step we prove that $E^α$ is $C^1$ on the set of all regular embedded closed curves belonging to $H^{(α+1)/2,2}$ and calculate its derivative. After that we use the structure…

Paper introduces a neural network-based non-stationary influence kernel for complex event data.

Variance reduction techniques like SVRG provide simple and fast algorithms for optimizing a convex finite-sum objective. For nonconvex objectives, these techniques can also find a first-order stationary point (with small gradient). However, in nonconvex optimization it is often crucial to find a second-order stationary…

Green functions on stationary varifolds established with inequalities and convergence results.

In this paper, we obtain the finite-horizon and infinite-horizon ruin probability asymptotics for risk processes with claims of subexponential tails for non-stationary arrival processes that satisfy a large deviation principle. As a result, the arrival process can be dependent, non-stationary and non-renewal. We give t…

Two classification results for stationary surfaces of least moment of inertia.

Improved method finds second-order stationary points privately with better efficiency.

New algorithms find near-stationary points in convex optimization.

Gradient descent and its variants are widely used in machine learning. However, oracle access of gradient may not be available in many applications, limiting the direct use of gradient descent. This paper proposes a method of estimating gradient to perform gradient descent, that converges to a stationary point for gene…

Characterizes critical points in convex double and triple bubbles.

Computing Nash equilibrium (NE) of multi-player games has witnessed renewed interest due to recent advances in generative adversarial networks. However, computing equilibrium efficiently is challenging. To this end, we introduce the Gradient-based Nikaido-Isoda (GNI) function which serves: (i) as a merit function, vani…

For a sequence of immersed connected closed Hamiltonian stationary Lagrangian submaniolds in $\mathbb{C}^{n}$ with uniform bounds on their volumes and the total extrinsic curvatures, we prove that a subsequence converges either to a point or to a Hamiltonian stationary Lagrangian $n$-varifold locally uniformly in $C^{k…

This paper shows that a perturbed form of gradient descent converges to a second-order stationary point in a number iterations which depends only poly-logarithmically on dimension (i.e., it is almost "dimension-free"). The convergence rate of this procedure matches the well-known convergence rate of gradient descent to…

Bayesian optimization with Gaussian processes speeds up searches for stationary points.

ELBO of VAEs converges to a sum of three entropies.

Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.

New algorithm finds approximate stationary points in non-convex optimization.

Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, even for simple control problems solvable by standard dynamic programming techniques, policy gradient algorithms face non-convex optimization pro…

Paper shows no spurious local minima in a specific matrix factorization problem.

ELBO converges to a sum of entropies for many generative models.

In this paper we study general Schatten-$p$ quasi-norm (SPQN) regularized matrix minimization problems. In particular, we first introduce a class of first-order stationary points for them, and show that the first-order stationary points introduced in [11] for an SPQN regularized $vector$ minimization problem are equiva…

Memory-based models can learn to approximate Bayes-optimal predictors for non-stationary data.

New algorithms detect changes in non-stationary MABs for better performance.

Generative models' ELBOs converge to entropy sums, proving for various models.

Given a heterogeneous time-series sample, the objective is to find points in time (called change points) where the probability distribution generating the data has changed. The data are assumed to have been generated by arbitrary unknown stationary ergodic distributions. No modelling, independence or mixing assumptions…

Extends Euler's problem to Lorentz-Minkowski plane.

Paper improves compressed SGD to reach second-order stationary points.

The paper extends Euler's problem to hyperbolic and spherical planes.

Stochastic gradient Langevin dynamics (SGLD) is a fundamental algorithm in stochastic optimization. Recent work by Zhang et al. [2017] presents an analysis for the hitting time of SGLD for the first and second order stationary points. The proof in Zhang et al. [2017] is a two-stage procedure through bounding the Cheege…

Reinforcement learning (RL) methods learn optimal decisions in the presence of a stationary environment. However, the stationary assumption on the environment is very restrictive. In many real world problems like traffic signal control, robotic applications, one often encounters situations with non-stationary environme…

We study Hamiltonian stationary Lagrangian surfaces in C^2, i.e. Lagrangian surfaces in C^2 which are stationary points of the area functional under smooth Hamiltonian variations. Using loop groups, we propose a formulation of the equation as a completely integrable system. We construct a Weierstrass type representatio…

Gradient descent (GD) and stochastic gradient descent (SGD) are the workhorses of large-scale machine learning. While classical theory focused on analyzing the performance of these methods in convex optimization problems, the most notable successes in machine learning have involved nonconvex optimization, and a gap has…

New theory for area of Legendrian surfaces, proving smoothness and variational results.

Study reveals how travel times on cylindrical boundaries can identify spacetime structure.

The paper proves that certain stationary hypersurfaces in high dimensions are essentially flat.

The paper studies Gauss maps of space-like stationary surfaces in Lorentz-Minkowski space, focusing on ramification and unicity.

Improved algorithm finds second-order stationary points in non-convex optimization.