arXiv research

Proves convergence of PSGLA for sampling non-convex potentials.

New sampling algorithm for non-smooth potentials.

We prove quantitative convergence rates at which discrete Langevin-like processes converge to the invariant distribution of a related stochastic differential equation. We study the setup where the additive noise can be non-Gaussian and state-dependent and the potential function can be non-convex. We show that the key p…

Study of non-convex potential functions in deep learning with Poincaré inequality.

In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…

Accelerates convergence in global non-convex optimization with reversible diffusion.

Shielded LMC samples from non-convex spaces with repulsive drift.

New method improves sampling from non-convex distributions using HFHR dynamics.

Recent studies have illustrated that stochastic gradient Markov Chain Monte Carlo techniques have a strong potential in non-convex optimization, where local and global convergence guarantees can be shown under certain conditions. By building up on this recent theory, in this study, we develop an asynchronous-parallel s…

This paper tackles global Nash equilibrium in non-convex multi-player games.

LMC algorithm achieves efficient sampling from complex distributions with specific tail behaviors.

Learning with a {\it convex loss} function has been a dominating paradigm for many years. It remains an interesting question how non-convex loss functions help improve the generalization of learning with broad applicability. In this paper, we study a family of objective functions formed by truncating traditional loss f…

Neural networks are a powerful class of functions that can be trained with simple gradient descent to achieve state-of-the-art performance on a variety of applications. Despite their practical success, there is a paucity of results that provide theoretical guarantees on why they are so effective. Lying in the center of…

Paper tackles fast convergence for non-convex strongly-concave min-max problems.

This study analyzes AdaGrad's stability and convergence in non-convex optimization.

New PAC-Bayesian bounds for online learning with data streams.

We lower bound the complexity of finding $ε$-stationary points (with gradient norm at most $ε$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (i…

New optimizer G-AdaGrad improves upon AdaGrad for non-convex machine learning problems.

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to non-cooperative approaches in the strongly-convex setting. More re…

Neural nets solve electric field in non-convex microfluidic devices.

This thesis tackles non-convex Bayesian learning via scalable dynamic importance sampling algorithms.

A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.

The paper provides gradient estimates for Neumann semigroups on manifolds with boundary under unbounded curvature conditions.

A new ODE model explains gradient descent dynamics near edge of stability.

We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the \textit{quasi-potential}. We analytically construct the quasi-potential function in the case when the loss function is convex and admits only one global minimum point. We show in this case th…

Unified distributed SGD improves non-convex optimization for large datasets.

Optimizes binary regression models with gradient ascent-descent methods.

Study on convergence of exponential probability measures with applications to maximum entropy models and SGLD.

Neural networks can represent complex piecewise functions efficiently.

This work builds the connection between the regularity theory of optimal transportation map, Monge-Ampère equation and GANs, which gives a theoretic understanding of the major drawbacks of GANs: convergence difficulty and mode collapse. According to the regularity theory of Monge-Ampère equation, if the support of the …

Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. …

Improved reSGLD accelerates convergence in non-convex learning problems.

Paper analyzes convergence of decentralized algorithms with noise and bias.

New algorithms improve sampling from complex distributions.

Compact, non-convex curve flows are created.

Integrates estimation and optimization for uncertain parameters.

We propose a new algorithm to learn a one-hidden-layer convolutional neural network where both the convolutional weights and the outputs weights are parameters to be learned. Our algorithm works for a general class of (potentially overlapping) patches, including commonly used structures for computer vision tasks. Our a…

We present Zeno, a technique to make distributed machine learning, particularly Stochastic Gradient Descent (SGD), tolerant to an arbitrary number of faulty workers. Zeno generalizes previous results that assumed a majority of non-faulty nodes; we need assume only one non-faulty worker. Our key idea is to suspect worke…

Paper studies early-stopped mirror descent for noisy sparse phase retrieval.

Paper introduces LR to generate synthetic data with privacy protection.

This paper explores optimising acquisition functions in Bayesian optimisation.

First order methods can take extremely long to find global minima of non-convex functions.

New algorithm improves convergence for non-convex problems with boundaries.

Adaptive algorithms improve performance in non-convex optimization across various scenarios.

ECP optimizes expensive functions without knowing Lipschitz constant.

CoolMomentum combines momentum and Simulated Annealing for deep learning optimization.

Adaptive step-size improves optimization in complex geometries.

Improved FDAM algorithms for heterogeneous data with constant communication complexity.