arXiv research

Paper finds smooth convex solutions to curvature problem.

Smooth even solutions found for a generalized convex geometry problem.

Smooths out complex shapes into simpler forms.

Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

Proves existence of smooth convex solutions to capillary curvature equations.

Local minimizers are convex and close to Wulff shapes.

Generalizes smoothness conditions for optimization methods.

Study on convex capillary hypersurfaces with Lp curvature in half-space.

Estimates convex hulls of smooth function images with error bounds.

Expanding FCCO to non-smooth weakly-convex problems, improving deep learning performance.

New methods improve convergence in non-convex non-smooth learning problems.

Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxat…

We consider the problem of minimizing the sum of an average function of a large number of smooth convex components and a general, possibly non-differentiable, convex function. Although many methods have been proposed to solve this problem with the assumption that the sum is strongly convex, few methods support the non-…

New sampling algorithm for non-smooth potentials.

We propose an adaptive smoothing algorithm based on Nesterov's smoothing technique in \cite{Nesterov2005c} for solving "fully" nonsmooth composite convex optimization problems. Our method combines both Nesterov's accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriat…

Paper tackles private optimization for non-smooth objectives efficiently.

We consider the problem of finding local minimizers in non-convex and non-smooth optimization. Under the assumption of strict saddle points, positive results have been derived for first-order methods. We present the first known results for the non-smooth case, which requires different analysis and a different algorithm…

We focus on the robust principal component analysis (RPCA) problem, and review a range of old and new convex formulations for the problem and its variants. We then review dual smoothing and level set techniques in convex optimization, present several novel theoretical results, and apply the techniques on the RPCA probl…

New algorithm solves complex optimization problems efficiently.

We consider the fundamental problem in non-convex optimization of efficiently reaching a stationary point. In contrast to the convex case, in the long history of this basic problem, the only known theoretical results on first-order non-convex optimization remain to be full gradient descent that converges in $O(1/\varep…

OSGM uses online learning to adapt stepsize for faster convergence.

We investigate online convex optimization in changing environments, and choose the adaptive regret as the performance measure. The goal is to achieve a small regret over every interval so that the comparator is allowed to change over time. Different from previous works that only utilize the convexity condition, this pa…

Proves existence and uniqueness of solutions to the Lp Gaussian Minkowski problem.

Proves rigidity for specific initial data sets under the dominant energy condition.

This paper focuses on convex constrained optimization problems, where the solution is subject to a convex inequality constraint. In particular, we aim at challenging problems for which both projection into the constrained domain and a linear optimization under the inequality constraint are time-consuming, which render …

This work accelerates gradient descent with anytime convergence guarantees.

We consider the problem of finding critical points of functions that are non-convex and non-smooth. Studying a fairly broad class of such problems, we analyze the behavior of three gradient-based methods (gradient descent, proximal update, and Frank-Wolfe update). For each of these methods, we establish rates of conver…

Step decay schedules improve convergence in non-convex optimization.

We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the …

New algorithms reduce dynamic regret for convex and smooth functions in non-stationary environments.

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and strongly convex. We prove that if the coupling matrix $A$ has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if $f$ is not stron…

We consider forward-backward greedy algorithms for solving sparse feature selection problems with general convex smooth functions. A state-of-the-art greedy method, the Forward-Backward greedy algorithm (FoBa-obj) requires to solve a large number of optimization problems, thus it is not scalable for large-size problems…

Optimal private ERM and SCO with subquadratic gradient complexity.

Unified flow solves $L^p$ Christoffel-Minkowski problem for $p>1$.

New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.

New SPS variant improves non-smooth optimization without small gradients.

Two new methods solve large-scale stochastic convex problems with linear constraints.

The paper explores different smooth map notions on convex sets and their relationships.

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …

We investigate the theoretical limits of pipeline parallel learning of deep learning architectures, a distributed setup in which the computation is distributed per layer instead of per example. For smooth convex and non-convex objective functions, we provide matching lower and upper complexity bounds and show that a na…

In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [8,11] for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov's technique developed in [8] for analyzing the RBCD method for minimizing a smooth convex functio…

Optimized method tackles convex optimization with heavy-tailed noise.

We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(ε^{-4/5})$ iteration complexity, breaking the $O(ε^{-1})$ barrier so far present for previous methods. We arrive at a similar rate fo…

A new method solves convex optimization on curved spaces.

New algorithm solves complex minimax problems efficiently.

A new optimization method, BPM, converges linearly in non-convex, non-smooth problems.

Paper investigates curvature problems and existence of solutions.