arXiv research

Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.

New algorithm solves minimax games with linear constraints.

Develops efficient method for nonconvex problems using Regula Falsi.

Safe reinforcement learning with nonconvex constraints using convex approximations.

OLLA framework efficiently samples from constrained distributions with nonconvex constraints.

Paper proposes a method to find approximate SOSP for nonconvex conic optimization problems.

New ADMM algorithm tackles nonconvex constraints from GANs for faster learning.

Unified framework for constrained diffusion models on nonconvex sets with efficient landing mechanism.

Improves logistic regression performance with nonconvex programming.

Paper extends LfC to distributed learning with asynchronous optimization.

Constrained least squares regression is an essential tool for high-dimensional data analysis. Given a partition $\mathcal{G}$ of input variables, this paper considers a particular class of nonconvex constraint functions that encourage the linear model to select a small number of variables from a small number of groups …

Unified Lagrangian-based methods for nonsmooth nonconvex optimization.

New algorithms optimize constrained problems faster, avoiding full set optimization.

Method solves nonconvex constrained optimization problems with a new augmented Lagrangian approach.

This paper explores the nonconvexity of push-forward constraints in machine learning.

New method solves sparse PCA and CCA with guaranteed convergence.

We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…

Study optimization landscapes for overcomplete representations, showing benign geometric structures.

Classifiers and rating scores are prone to implicitly codifying biases, which may be present in the training data, against protected classes (i.e., age, gender, or race). So it is important to understand how to design classifiers and scores that prevent discrimination in predictions. This paper develops computationally…

In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, econometrics, etc. The problem is formulated to minimize the least squares loss with a sparsity-induci…

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian b…

Paper proposes a new method to find approximate SOSP for nonconvex constrained optimization problems.

Nonconvex and nonsmooth optimization problems are frequently encountered in much of statistics, business, science and engineering, but they are not yet widely recognized as a technology in the sense of scalability. A reason for this relatively low degree of popularity is the lack of a well developed system of theory an…

We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regres…

A new method tackles nonconvex optimization with penalties and proximal terms.

CRPO solves challenging SRL problems with convergence guarantee.

New algorithms reduce complexity for solving nonconvex optimization problems with stochastic objectives and constraints.

New method solves sparse deconvolution problems with theoretical guarantees and practical applications.

Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …

This paper concerns the problem of recovering an unknown but structured signal $x \in R^n$ from $m$ quadratic measurements of the form $y_r=|<a_r,x>|^2$ for $r=1,2,...,m$. We focus on the under-determined setting where the number of measurements is significantly smaller than the dimension of the signal ($m<<n$). We for…

Proposes a new PCA method that balances Euclidean and angle distances.

Proposes a generalized XGBoost method for nonconvex loss functions.

In this paper, we focus on solving an important class of nonconvex optimization problems which includes many problems for example signal processing over a networked multi-agent system and distributed learning over networks. Motivated by many applications in which the local objective function is the sum of smooth but po…

New algorithm for nonconvex optimization on constrained Riemannian manifolds converges quickly.

We analyze the performance of alternating minimization for loss functions optimized over two variables, where each variable may be restricted to lie in some potentially nonconvex constraint set. This type of setting arises naturally in high-dimensional statistics and signal processing, where the variables often reflect…

Symmetric nonnegative matrix factorization has found abundant applications in various domains by providing a symmetric low-rank decomposition of nonnegative matrices. In this paper we propose a Frank-Wolfe (FW) solver to optimize the symmetric nonnegative matrix factorization problem under a simplicial constraint, whic…

New method solves complex constrained optimization problems.

VRSGT algorithm reduces orthogonality constraints in decentralized optimization.

Paper learns Markov models from data with low-rank optimization.

SONATA algorithm converges to solutions of nonconvex smooth functions with KL property.

New method solves nonconvex optimization problems without explicit gradients.

Several fundamental problems that arise in optimization and computer science can be cast as follows: Given vectors $v_1,\ldots,v_m \in \mathbb{R}^d$ and a constraint family ${\cal B}\subseteq 2^{[m]}$, find a set $S \in \cal{B}$ that maximizes the squared volume of the simplex spanned by the vectors in $S$. A motivatin…

Study on predicting sequences with Gaussian constraints, linking to intrinsic volumes and metric complexity.

Paper shows Burer-Monteiro method can solve SDPs in polynomial time under smoothed analysis.

We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm receives any parameter value and returns: local minimum, second-order stationary point, or a strict descent direction. The presence of $M$ data p…

New methods solve complex optimization problems with fewer function queries.

We study the problem of recovery of matrices that are simultaneously low rank and row and/or column sparse. Such matrices appear in recent applications in cognitive neuroscience, imaging, computer vision, macroeconomics, and genetics. We propose a GDT (Gradient Descent with hard Thresholding) algorithm to efficiently r…

We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the …