A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
Paper proposes an algorithm to solve complex minimax problems efficiently.
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
Schedule-free SGD is optimal for nonconvex optimization problems.
New algorithms solve nonconvex-concave minimax problems without parameter knowledge.
This paper analyzes OGDA and EG methods for nonconvex minimax problems.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
New framework explains why nonconvex methods work well in low-rank matrix estimation.
As surrogate functions of -norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…
New algorithm solves nonconvex-convex minimax problems efficiently.
New algorithms solve complex minimax problems without needing derivatives.
AGDA and variance-reduced methods solve nonconvex-nonconcave minimax problems globally and faster.
Paper analyzes nonconvex bandit problems with improved adaptive methods.
Develops efficient method for nonconvex problems using Regula Falsi.
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
Survey of tractable nonconvex problems using symmetry.
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
PAGE optimizes nonconvex problems with optimal convergence rates.
Introduces PPMM algorithm for nonconvex robust regression problems.
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
TiAda adapts adaptive gradient methods for nonconvex minimax optimization.
Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving opti…
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form . Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and s…
A fast sketching algorithm solves regularized least squares problems efficiently.
This work addresses the issue of large covariance matrix estimation in high-dimensional statistical analysis. Recently, improved iterative algorithms with positive-definite guarantee have been developed. However, these algorithms cannot be directly extended to use a nonconvex penalty for sparsity inducing. Generally, a…
We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…
We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of nonconvex -smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into subproblems, and utilizes an augmented Lagrangian b…
A new method solves a complex optimization problem efficiently.
PAGE is a simple gradient estimator for nonconvex optimization problems.
We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…
In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, econometrics, etc. The problem is formulated to minimize the least squares loss with a sparsity-induci…
While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private…
Within the unmanageably large class of nonconvex optimization, we consider the rich subclass of nonsmooth problems that have composite objectives---this already includes the extensively studied convex, composite objective problems as a special case. For this subclass, we introduce a powerful, new framework that permits…
Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.
Efficient solver for nonconvex tensor regularization reduces computational cost.
The paper analyzes PPM for nonconvex-nonconcave problems, identifying three regions with varying convergence guarantees.
The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…
We study the safe reinforcement learning problem with nonlinear function approximation, where policy optimization is formulated as a constrained optimization problem with both the objective and the constraint being nonconvex functions. For such a problem, we construct a sequence of surrogate convex constrained optimiza…
Paper reviews advances in solving sparsest vector problem in subspaces.
We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient (SVRG) methods for them. SVRG and related methods have recently surged into prominence for convex optimization given their edge over stochastic gradient descent (SGD); but their theoretical analysis almost exclusively assumes convex…
New algorithm solves structured nonconvex-nonconcave min-max problems.
DS-GDA solves nonconvex-nonconcave problems without regularity conditions.
We provide theoretical analysis of the statistical and computational properties of penalized -estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …
In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and di…
We study nonconvex optimization landscapes for learning overcomplete representations, including learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictionaries, where these unsupervised learning problems find many applications in high-dimensional data analysis. Despite the empirical success of …
Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.
Innovative method solves nonconvex optimization on manifolds.