arXiv research

The paper examines hyperbolicity in bounded strongly minimally convex domains in R^d.

Sharp estimates for Finsler metrics in convex domains.

We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex prob…

We consider the problem of minimizing the sum of an average function of a large number of smooth convex components and a general, possibly non-differentiable, convex function. Although many methods have been proposed to solve this problem with the assumption that the sum is strongly convex, few methods support the non-…

New algorithms minimize dynamic regret for strongly convex losses.

In this paper, a correspondence via duality is established between the set of locally strongly convex symmetric equiaffine hyperspheres and the set of minimal symmetric Lagrangian submanifolds in a certain complex space form. By using this correspondence theorem, we are able to provide an alternative proof of the class…

Optimizes CM for stochastic convex optimization with progressive precision.

In this paper, we develop a new accelerated stochastic gradient method for efficiently solving the convex regularized empirical risk minimization problem in mini-batch settings. The use of mini-batches is becoming a golden standard in the machine learning community, because mini-batch settings stabilize the gradient es…

We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimization (ERM), and in particular linear least-squares regression, across a wide range of problem settings. To achieve this, we establish a framewo…

The paper proves Gromov hyperbolicity of certain metrics using isoperimetric inequalities.

Drago optimizes DRO problems with faster convergence.

New methods accelerate gradient descent for convex and strongly convex functions.

It has recently been shown that the problem of testing global convexity of polynomials of degree four is {strongly} NP-hard, answering an open question of N.Z. Shor. This result is minimal in the degree of the polynomial when global convexity is of concern. In a number of applications however, one is interested in test…

We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging method with variance reduction. Our methods generate a sparser solution than the…

AGNES accelerates gradient descent with noisy gradients.

The cyclic block coordinate descent-type (CBCD-type) methods, which performs iterative updates for a few coordinates (a block) simultaneously throughout the procedure, have shown remarkable computational performance for solving strongly convex minimization problems. Typical applications include many popular statistical…

We study dual-based algorithms for distributed convex optimization problems over networks, where the objective is to minimize a sum $\sum_{i=1}^{m}f_i(z)$ of functions over in a network. We provide complexity bounds for four different cases, namely: each function $f_i$ is strongly convex and smooth, each function is ei…

Almost all local minima in neural networks are strongly convex.

Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.

Investigates properties of a pseudometric on domains in Euclidean space, linking it to hyperbolic geometry.

Paper closes convergence gap for SGD without replacement.

Paper analyzes regret bounds for unconstrained online optimization.

Strongly convex bodies can be approximated by smooth ones.

Characterizes symmetric Bernoulli distributions with minimal convex sums.

Let $U\subseteq\mathbb{R}^{n}$ be open and convex. We show that every (not necessarily Lipschitz or strongly) convex function $f:U\to\mathbb{R}$ can be approximated by real analytic convex functions, uniformly on all of $U$. In doing so we provide a technique which transfers results on uniform approximation on bounded …

Study optimizes zero-order strongly convex function minimization with higher order smoothness.

Paper solves robust convex problems with heavy-tailed noise.

Improved dynamic regret analysis for strongly convex and smooth functions.

We consider stochastic gradient descent algorithms for minimizing a non-smooth, strongly-convex function. Several forms of this algorithm, including suffix averaging, are known to achieve the optimal $O(1/T)$ convergence rate in expectation. We consider a simple, non-uniform averaging strategy of Lacoste-Julien et al. …

Improved SHB method for faster convergence on strongly-convex quadratics.

The paper simplifies strongly convex problems to simplicial structures.

In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in mach…

In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph. Assuming that each agent has access to a stochastic first-order oracle ($\mathcal{SFO}$), we propose a novel distribut…

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …

This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…

Push-SAGA is a decentralized algorithm for directed graphs that converges linearly.

Two new algorithms optimize decentralized convex optimization with reduced communication rounds.

In this paper, we prove that a strongly convex complex Finsler metric $F$ on a domain $D\subset\mathbb{C}^n$ is projectively flat (resp. dually flat) if and only if $F$ comes from a strongly convex complex Minkowski metric.

Paper extends SMM to weakly convex and multi-convex surrogates for non-convex optimization.

In this work we establish the first linear convergence result for the stochastic heavy ball method. The method performs SGD steps with a fixed stepsize, amended by a heavy ball momentum term. In the analysis, we focus on minimizing the expected loss and not on finite-sum minimization, which is typically a much harder p…

The Bergman kernel's minimal point determines domain properties.

New method estimates minimizer and minimum value of a regression function.

We prove, under a certain boundedness condition at infinity on the $(\bar{X}^{\top}, \bar{X}^{\bot})$-component of the second fundamental form, the vanishing of the essential spectrum of a complete minimal $\bar{X}$-bounded and $\bar{X}$-properly immersed submanifold on a Riemannian manifold endowed with a strongly con…

Improved COCO algorithms with better constraint control.

The paper tackles minimax optimality in continuum contextual bandits with Hölder continuity.

RR algorithm improves convergence rate without strong convexity assumptions.

To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the setting of online convex optimization, several algorithms have been successfully developed to minimize the adaptive regret. However, existi…

In this paper, we consider the problem of minimizing the average of a large number of nonsmooth and convex functions. Such problems often arise in typical machine learning problems as empirical risk minimization, but are computationally very challenging. We develop and analyze a new algorithm that achieves robust linea…