arXiv research

Paper tackles non-convex inf-projection problems with stochastic optimization.

The paper tackles finding stationary points in stochastic convex optimization problems.

Stochastic gradient algorithms estimate the gradient based on only one or a few samples and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems. However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates, due to t…

New methods optimize complex optimization problems with improved efficiency.

The paper relaxes assumptions for analyzing stochastic optimization algorithms.

Study shows Stochastic Mirror Descent optimizes convex problems with infinite noise variance.

Accelerates stochastic optimization for convex and strongly convex problems.

New adaptive methods solve weakly convex stochastic optimization problems.

Optimized method tackles convex optimization with heavy-tailed noise.

Study on convex ordering in stochastic control for swing contracts, proving value function convexity.

Paper solves optimization problems with convex expectation constraints using a new algorithm.

New algorithms solve convex-concave problems faster than previous methods.

Recently, many variance reduced stochastic alternating direction method of multipliers (ADMM) methods (e.g.\ SAG-ADMM, SDCA-ADMM and SVRG-ADMM) have made exciting progress such as linear convergence rates for strongly convex problems. However, the best known convergence rate for general convex problems is O(1/T) as opp…

New method achieves optimal performance without needing problem parameters.

New algorithm improves convergence rates for convex optimization problems.

New algorithm solves complex non-convex problems efficiently.

This paper considers online convex optimization (OCO) with stochastic constraints, which generalizes Zinkevich's OCO over a known simple fixed set by introducing multiple stochastic functional constraints that are i.i.d. generated at each round and are disclosed to the decision maker only after the decision is made. Th…

A distributed subgradient method tackles non-convex optimization problems in networks.

RONM method reduces regret in stochastic convex bandits with decreasing noise.

New method gives high confidence bounds for stochastic convex optimization with minimal overhead.

Two new methods solve large-scale stochastic convex problems with linear constraints.

New stochastic algorithms solve DC functions and non-convex problems efficiently.

In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )}$ is the inner function, and $F_i(\cdot)$ is the outer function. We explore the variance reduction based met…

We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. Roughly, the n…

Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.

Paper solves robust convex problems with heavy-tailed noise.

Bayesian optimization tackles non-convex, two-stage stochastic problems efficiently.

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …

NSGLD improves SGLD for non-convex optimization problems.

Paper tackles private optimization for non-smooth objectives efficiently.

Optimizes stochastic and online optimization methods based on problem geometry.

Adaptive algorithm AMSGrad converges for weakly convex constrained optimization problems.

New algorithm learns optimal stepsizes for SGD in noisy non-convex optimization.

In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal stationary points. However, existing results on stochastic non-convex optimizatio…

The paper analyzes reg-SGD for convex problems, proving convergence and quantifying the rate of convergence.

We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex prob…

We propose a new stochastic L-BFGS algorithm and prove a linear convergence rate for strongly convex and smooth functions. Our algorithm draws heavily from a recent stochastic variant of L-BFGS proposed in Byrd et al. (2014) as well as a recent approach to variance reduction for stochastic gradient descent from Johnson…

New method solves stochastic optimization problems with affine constraints.

SGD converges to global minimum for certain non-convex functions.

Study optimal portfolio management with periodic evaluations in stochastic models, considering convex constraints.

Improved privacy-preserving methods for convex optimization with heavy-tailed data.

New algorithm solves saddle point problems in Banach spaces.

New algorithm finds local minima in non-convex problems efficiently.

Paper develops probabilistic bounds for a stochastic gradient algorithm in non-convex problems.

Develops methods for solving convex optimization problems with improved accuracy and convergence.

Better models make stochastic optimization more stable and robust.

SVRG fails for deep learning due to non-convexity.