arXiv research

In this work we study convex relaxations of quadratic optimisation problems over permutation matrices. While existing semidefinite programming approaches can achieve remarkably tight relaxations, they have the strong disadvantage that they lift the original $n {\times} n$-dimensional variable to an $n^2 {\times} n^2$-d…

Note on the computational complexity of Gromov-Wasserstein distance.

Quadratic memory is essential for optimal convex optimization queries.

OMGD algorithm optimizes online convex optimization with switching costs and delayed gradients.

In this paper we study a continuous-time stochastic linear quadratic control problem arising from mathematical finance. We model the asset dynamics with random market coefficients and portfolio strategies with convex constraints. Following the convex duality approach, we show that the necessary and sufficient optimalit…

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…

We consider the problem of estimating the phases of K mixed complex signals from a multichannel observation, when the mixing matrix and signal magnitudes are known. This problem can be cast as a non-convex quadratically constrained quadratic program which is known to be NP-hard in general. We propose three approaches t…

In this paper, we study the problem of escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of the problem, if the convex set $\mathcal{C}$ is simple for a quadratic objectiv…

We derive a Bernstein type result for the special Lagrangian equation, namely, any global convex solution must be quadratic. In terms of minimal surfaces, the result says that any global minimal Lagrangian graph with convex potential must be a hyper-plane.

We note that known methods achieving the optimal oracle complexity for first order convex optimization require quadratic memory, and ask whether this is necessary, and more broadly seek to characterize the minimax number of first order queries required to optimize a convex Lipschitz function subject to a memory constra…

Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.

Ancient Lagrangian flows get limited convex solutions.

New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.

Developed a theory of local convexity for second order differential equations on Lie algebroids.

Paper solves optimal portfolio deleveraging with cross asset impacts.

We consider a square-integrable semimartingale and investigate the convex order relations between its discrete, continuous and predictable quadratic variation. As the main results, we show that if the semimartingale has conditionally independent increments and symmetric jump measure, then its discrete realized variance…

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…

New algorithm extends LMC to more complex potentials.

In this paper, we analyze a real-valued reflected backward stochastic differential equation (RBSDE) with an unbounded obstacle and an unbounded terminal condition when its generator $f$ has quadratic growth in the $z$-variable. In particular, we obtain existence, comparison, and stability results, and consider the opti…

Paper solves graph matching problem using convex relaxation to the simplex.

We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…

We propose the convex factorization machine (CFM), which is a convex variant of the widely used Factorization Machines (FMs). Specifically, we employ a linear+quadratic model and regularize the linear term with the $\ell_2$-regularizer and the quadratic term with the trace norm regularizer. Then, we formulate the CFM o…

Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance. This paper concerns the problem of learning control policies for unknown linear dynamical systems so as to maximize…

Novel approximation hierarchy for sparse quadratic programs.

A new method for exponentially weighted moving models using approximations.

We study computational and statistical consequences of problem geometry in stochastic and online optimization. By focusing on constraint set and gradient geometry, we characterize the problem families for which stochastic- and adaptive-gradient methods are (minimax) optimal and, conversely, when nonlinear updates -- su…

Signal estimation problems with smoothness and sparsity priors can be naturally modeled as quadratic optimization with $\ell_0$-"norm" constraints. Since such problems are non-convex and hard-to-solve, the standard approach is, instead, to tackle their convex surrogates based on $\ell_1$-norm relaxations. In this paper…

New method improves efficiency of non-convex matrix reconstruction.

dboost optimizes prediction models for convex cone problems.

This paper considers the recovery of a rank $r$ positive semidefinite matrix $X X^T\in\mathbb{R}^{n\times n}$ from $m$ scalar measurements of the form $y_i := a_i^T X X^T a_i$ (i.e., quadratic measurements of $X$). Such problems arise in a variety of applications, including covariance sketching of high-dimensional data…

FedExProx's performance is no better than GD for quadratic optimization.

We study a robust maximization problem from terminal wealth and consumption under a convex constraints on the portfolio. We state the existence and the uniqueness of the consumption-investment strategy by studying the associated quadratic backward stochastic differential equation (BSDE in short). We characterize the op…

In this paper, we propose the first computationally efficient projection-free algorithm for bandit convex optimization (BCO). We show that our algorithm achieves a sublinear regret of $O(nT^{4/5})$ (where $T$ is the horizon and $n$ is the dimension) for any bounded convex functions with uniformly bounded gradients. We …

New framework for RL with linear-convex models reduces performance gap.

We consider the problem of minimizing the sum of an average function of a large number of smooth convex components and a general, possibly non-differentiable, convex function. Although many methods have been proposed to solve this problem with the assumption that the sum is strongly convex, few methods support the non-…

Study on regularity of optimal transport maps on convex domains with quadratic cost.

Improved SVRG for quadratic functions achieves better performance and running times.

We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…

We study the minimal surface equation in the Heisenberg space, Nil_3. A geometric proof of non existence of minimal graphs over non convex, bounded and unbounded domains is achieved (our proof holds in the Euclidean space as well). We solve the Dirichlet problem for the minimal surface equation over bounded and unbound…

Graph alignment problem solved with convex relaxations for correlated matrices.

Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.

Paper develops methods for non-quadratic loss low-rank matrix recovery.

We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions. For such a problem, we construct a sequence of surrogate convex constrained optimiza…

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and strongly convex. We prove that if the coupling matrix $A$ has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if $f$ is not stron…

Constrained second-order convex optimization algorithms are the method of choice when a high accuracy solution to a problem is needed, due to their local quadratic convergence. These algorithms require the solution of a constrained quadratic subproblem at every iteration. We present the \emph{Second-Order Conditional G…

Accelerated gradient method's stability deteriorates exponentially with steps.

This paper establishes a statistical versus computational trade-off for solving a basic high-dimensional machine learning problem via a basic convex relaxation method. Specifically, we consider the {\em Sparse Principal Component Analysis} (Sparse PCA) problem, and the family of {\em Sum-of-Squares} (SoS, aka Lasserre/…