arXiv research

Ancient Lagrangian flows get limited convex solutions.

Developed a theory of local convexity for second order differential equations on Lie algebroids.

We consider a square-integrable semimartingale and investigate the convex order relations between its discrete, continuous and predictable quadratic variation. As the main results, we show that if the semimartingale has conditionally independent increments and symmetric jump measure, then its discrete realized variance…

Novel approximation hierarchy for sparse quadratic programs.

A new algorithm for solving constrained convex optimization problems efficiently.

In this work we study convex relaxations of quadratic optimisation problems over permutation matrices. While existing semidefinite programming approaches can achieve remarkably tight relaxations, they have the strong disadvantage that they lift the original $n {\times} n$-dimensional variable to an $n^2 {\times} n^2$-d…

Local SGD outperforms minibatch SGD for quadratic objectives.

Note on the computational complexity of Gromov-Wasserstein distance.

Study on regularity of optimal transport maps on convex domains with quadratic cost.

OMGD algorithm optimizes online convex optimization with switching costs and delayed gradients.

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…

Quadratic memory is essential for optimal convex optimization queries.

Accelerated gradient method's stability deteriorates exponentially with steps.

Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.

We consider the problem of estimating the phases of K mixed complex signals from a multichannel observation, when the mixing matrix and signal magnitudes are known. This problem can be cast as a non-convex quadratically constrained quadratic program which is known to be NP-hard in general. We propose three approaches t…

In this paper we study a continuous-time stochastic linear quadratic control problem arising from mathematical finance. We model the asset dynamics with random market coefficients and portfolio strategies with convex constraints. Following the convex duality approach, we show that the necessary and sufficient optimalit…

We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…

In this paper, we study the problem of escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of the problem, if the convex set $\mathcal{C}$ is simple for a quadratic objectiv…

New algorithm extends LMC to more complex potentials.

We note that known methods achieving the optimal oracle complexity for first order convex optimization require quadratic memory, and ask whether this is necessary, and more broadly seek to characterize the minimax number of first order queries required to optimize a convex Lipschitz function subject to a memory constra…

New method uses DC functions for piecewise linear regression.

We derive a Bernstein type result for the special Lagrangian equation, namely, any global convex solution must be quadratic. In terms of minimal surfaces, the result says that any global minimal Lagrangian graph with convex potential must be a hyper-plane.

FedExProx's performance is no better than GD for quadratic optimization.

The paper develops a convex parameterization for robust RNNs ensuring stability and robustness.

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.

Proposes r2SGLD for efficient constrained exploration in non-convex learning.

The DANE algorithm is an approximate Newton method popularly used for communication-efficient distributed machine learning. Reasons for the interest in DANE include scalability and versatility. Convergence of DANE, however, can be tricky; its appealing convergence rate is only rigorous for quadratic objective, and for …

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…

A new method for exponentially weighted moving models using approximations.

New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.

New method improves efficiency of non-convex matrix reconstruction.

In a seminal paper "Volumen und Oberfläche" (1903), Minkowski introduced the basic notion of mixed volumes and the corresponding inequalities that lie at the heart of convex geometry. The fundamental importance of characterizing the extremals of these inequalities was already emphasized by Minkowski himself, but has to…

Improved SVRG for quadratic functions achieves better performance and running times.

Paper tackles multivariate shape-constrained convex regression problems.

Signal estimation problems with smoothness and sparsity priors can be naturally modeled as quadratic optimization with $\ell_0$-"norm" constraints. Since such problems are non-convex and hard-to-solve, the standard approach is, instead, to tackle their convex surrogates based on $\ell_1$-norm relaxations. In this paper…

Paper closes convergence gap for SGD without replacement.

New framework for RL with linear-convex models reduces performance gap.

Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance. This paper concerns the problem of learning control policies for unknown linear dynamical systems so as to maximize…

In this paper, we analyze a real-valued reflected backward stochastic differential equation (RBSDE) with an unbounded obstacle and an unbounded terminal condition when its generator $f$ has quadratic growth in the $z$-variable. In particular, we obtain existence, comparison, and stability results, and consider the opti…

In this paper, we propose the first computationally efficient projection-free algorithm for bandit convex optimization (BCO). We show that our algorithm achieves a sublinear regret of $O(nT^{4/5})$ (where $T$ is the horizon and $n$ is the dimension) for any bounded convex functions with uniformly bounded gradients. We …

Reconstructing polytopes with fixed facet directions from support function evaluations.

Improved SHB method for faster convergence on strongly-convex quadratics.

This paper considers the recovery of a rank $r$ positive semidefinite matrix $X X^T\in\mathbb{R}^{n\times n}$ from $m$ scalar measurements of the form $y_i := a_i^T X X^T a_i$ (i.e., quadratic measurements of $X$). Such problems arise in a variety of applications, including covariance sketching of high-dimensional data…

We propose the convex factorization machine (CFM), which is a convex variant of the widely used Factorization Machines (FMs). Specifically, we employ a linear+quadratic model and regularize the linear term with the $\ell_2$-regularizer and the quadratic term with the trace norm regularizer. Then, we formulate the CFM o…

We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…

We study computational and statistical consequences of problem geometry in stochastic and online optimization. By focusing on constraint set and gradient geometry, we characterize the problem families for which stochastic- and adaptive-gradient methods are (minimax) optimal and, conversely, when nonlinear updates -- su…

We study a robust maximization problem from terminal wealth and consumption under a convex constraints on the portfolio. We state the existence and the uniqueness of the consumption-investment strategy by studying the associated quadratic backward stochastic differential equation (BSDE in short). We characterize the op…