A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Unified framework for constructing nonconvex sparse recovery methods.
Improved SGD methods converge faster for nonconvex optimization.
Schedule-free SGD is optimal for nonconvex optimization problems.
We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…
PPGD solves nonconvex nonsmooth optimization problems without KL property.
New framework explains why nonconvex methods work well in low-rank matrix estimation.
Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.
While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private…
In the paper, we study the stochastic alternating direction method of multipliers (ADMM) for the nonconvex optimizations, and propose three classes of the nonconvex stochastic ADMM with variance reduction, based on different reduced variance stochastic gradients. Specifically, the first class called the nonconvex stoch…
This paper analyzes OGDA and EG methods for nonconvex minimax problems.
Simple DP algorithms find approximate solutions for nonconvex ERM.
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…
Unified parametric assumption improves convergence guarantees for nonconvex optimization.
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general noncon…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
This work addresses the issue of large covariance matrix estimation in high-dimensional statistical analysis. Recently, improved iterative algorithms with positive-definite guarantee have been developed. However, these algorithms cannot be directly extended to use a nonconvex penalty for sparsity inducing. Generally, a…
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
As surrogate functions of -norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…
With the large rising of complex data, the nonconvex models such as nonconvex loss function and nonconvex regularizer are widely used in machine learning and pattern recognition. In this paper, we propose a class of mini-batch stochastic ADMMs (alternating direction method of multipliers) for solving large-scale noncon…
Paper proposes a new method for training nonconvex models.
PAGE optimizes nonconvex problems with optimal convergence rates.
Paper proposes an algorithm to solve complex minimax problems efficiently.
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
New algorithm solves nonconvex-convex minimax problems efficiently.
Develops efficient method for nonconvex problems using Regula Falsi.
New algorithms solve nonconvex-concave minimax problems without parameter knowledge.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
Paper analyzes nonconvex bandit problems with improved adaptive methods.
New nonconvex penalty smooths at origin for deep learning.
Unified framework for nonconvex matrix completion with linearly parameterized factors.
Study uncovers statistical optimality of nonconvex tensor completion methods.
Innovative method solves nonconvex optimization on manifolds.
The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…
This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…
Sharp estimates for heat flow on nonconvex domains.
We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.
TiAda adapts adaptive gradient methods for nonconvex minimax optimization.
Two-Timescale EM Methods improve EM for nonconvex models.
Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem. Recently, various algorithms using nonconvex penalties have been proposed, among whic…
New algorithms solve complex minimax problems without needing derivatives.
Paper develops methods for statistical inference with SGD in nonconvex optimization.
PAGE is a simple gradient estimator for nonconvex optimization problems.
We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…
Within the unmanageably large class of nonconvex optimization, we consider the rich subclass of nonsmooth problems that have composite objectives---this already includes the extensively studied convex, composite objective problems as a special case. For this subclass, we introduce a powerful, new framework that permits…
Apollo improves nonconvex stochastic optimization efficiency.