arXiv research

New algorithm expands FTRL framework with improved worst-case regret bounds.

Establishes geometric convergence of iterative optimization algorithms.

Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA). Although uniform sampling can guarantee that the sampled stochastic quantity is an unbiased estim…

New algorithm reduces regret and constraint violation in online convex optimization with complex constraints.

Paper develops efficient algorithms for robust optimization across multiple groups.

A new framework optimizes model transfer across domains with labeled data.

Sample efficiency is critical in solving real-world reinforcement learning problems, where agent-environment interactions can be costly. Imitation learning from expert advice has proved to be an effective strategy for reducing the number of interactions required to train a policy. Online imitation learning, which inter…

Characterizes minimizing curves in Riemannian manifolds.

Defines weak geodesics on specific subsets of manifolds.

The paper proves properties of curves in Riemannian manifolds.

We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, an…

This monograph presents the main complexity theorems in convex optimization and their corresponding algorithms. Starting from the fundamental theory of black-box optimization, the material progresses towards recent advances in structural optimization and stochastic optimization. Our presentation of black-box optimizati…

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.

We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the …

In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…

We provide tight upper and lower bounds on the complexity of minimizing the average of $m$ convex functions using gradient and prox oracles of the component functions. We show a significant gap between the complexity of deterministic vs randomized optimization. For smooth functions, we show that accelerated gradient de…

Stochastic gradient algorithms estimate the gradient based on only one or a few samples and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems. However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates, due to t…

We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective. Inspired by the classical prox methods, we develop a novel algorithmic framework for GANs via an infinite-dimensional two-player game and prove rigorous convergence rates to the mixed NE, reso…

We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method…

This paper studies first order methods for solving smooth minimax optimization problems $\min_x \max_y g(x,y)$ where $g(\cdot,\cdot)$ is smooth and $g(x,\cdot)$ is concave for each $x$. In terms of $g(\cdot,y)$, we consider two settings -- strongly convex and nonconvex -- and improve upon the best known rates in both. …

New method predicts and optimizes matrix recovery from noisy measurements.

Develops a gradient flow for Muon optimizer, a method for optimization.

Two algorithms find optimal points in decentralized optimization.

Optimized method tackles convex optimization with heavy-tailed noise.

In this paper, we propose a simple variant of the original stochastic variance reduction gradient (SVRG), where hereafter we refer to as the variance reduced stochastic gradient descent (VR-SGD). Different from the choices of the snapshot point and starting point in SVRG and its proximal variant, Prox-SVRG, the two vec…

Method solves nonconvex constrained optimization problems with a new augmented Lagrangian approach.

We study \emph{TV regularization}, a widely used technique for eliciting structured sparsity. In particular, we propose efficient algorithms for computing prox-operators for $\ell_p$-norm TV. The most important among these is $\ell_1$-norm TV, for whose prox-operator we present a new geometric analysis which unveils a …

Mirror flows converge to a limiting flow with a convex potential.

New mirror maps improve PMD performance in reinforcement learning.

We describe mirror symmetry on higher dimensional tori, paying special attention to the behaviour of D-branes under mirror symmetry. To find the mirror D-branes the description of mirror symmetry on D-branes due to Ooguri, Oz en Yin is used. This method allows us to deal with the coisotropic D-branes recently introduce…

Derives Mirror Descent from gradient flow on a Riemannian manifold.

A new method solves l1-regularized optimization problems efficiently and sparsely.

To make deep neural networks feasible in resource-constrained environments (such as mobile devices), it is beneficial to quantize models by using low-precision weights. One common technique for quantizing neural networks is the straight-through gradient method, which enables back-propagation through the quantization ma…

Motivated by Strominger-Yau-Zaslow's mirror symmetry proposal and Kontsevich's homological mirror symmetry conjecture, we study mirror phenomena (in A-model) of certain results from Donaldson-Thomas theory for Calabi-Yau 4-folds.

Study homological mirror symmetry for Hirzebruch surfaces using Morse homotopy.

This paper deforms complex tori and their mirrors using gerbes.

Constructs mirror pairs for solvmanifolds using Lie groups.

It has been recently shown that a large class of balanced graph cuts allows for an exact relaxation into a nonlinear eigenproblem. We review briefly some of these results and propose a family of algorithms to compute nonlinear eigenvectors which encompasses previous work as special cases. We provide a detailed analysis…

We study mirror symmetry of type II strings on manifolds with the exceptional holonomy groups $G_2$ and Spin(7). Our central result is a construction of mirrors of Spin(7) manifolds realized as generalized connected sums. In parallel to twisted connected sum $G_2$ manifolds, mirrors of such Spin(7) manifolds can be fou…

Homological mirror symmetry for toric Fano surfaces using Morse homotopy.

New analysis shows GMD can converge linearly under PL-like conditions.

In this article we explore some finer properties of equi-areal mirrors and introduce techniques for developing new mirror surfaces that simultaneously minimize angular and areal distortion.

Mirror flow optimizes separable data problems, converging to a maximum margin classifier.

Study connects mirror symmetry invariants to K-stability for toric manifolds.

Reparameterizes mirror descent as gradient descent for efficient sparse learning.

This paper focuses on a topological version on the Strominger-Yau-Zaslow mirror symmetry conjecture. Roughly put, the SYZ conjecture suggests that mirror pairs of Calabi-Yau manifolds are related by the existence of dual special Lagrangian torus fibrations. We explore this conjecture without reference to the special La…

NGMs create mirrored features to assess neural network feature importance.