A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Improved SGD methods converge faster for nonconvex optimization.
This paper analyzes OGDA and EG methods for nonconvex minimax problems.
PAGE optimizes nonconvex problems with optimal convergence rates.
Unified analysis of SGD variants for nonconvex federated optimization.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.
We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient (SVRG) methods for them. SVRG and related methods have recently surged into prominence for convex optimization given their edge over stochastic gradient descent (SGD); but their theoretical analysis almost exclusively assumes convex…
New algorithm for privacy-preserving nonconvex optimization.
Unified analysis for shuffling-type gradient methods in optimization.
Schedule-free SGD is optimal for nonconvex optimization problems.
New SGD analysis for nonconvex optimization finds optimal rates.
A new method tackles nonconvex optimization with penalties and proximal terms.
Paper proposes a new method for training nonconvex models.
SGHMC improves sampling and optimization under local conditions.
We provide theoretical analysis of the statistical and computational properties of penalized -estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …
In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and di…
The paper addresses nonconvex penalized LAD estimation in partial linear models using DNNs.
Matrix completion has attracted much interest in the past decade in machine learning and computer vision. For low-rank promotion in matrix completion, the nuclear norm penalty is convenient due to its convexity but has a bias problem. Recently, various algorithms using nonconvex penalties have been proposed, among whic…
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…
Paper proposes an algorithm to solve complex minimax problems efficiently.
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
Paper proposes robust tensor regression method for tensor data analysis.
Survey of tractable nonconvex problems using symmetry.
We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…
This work addresses the issue of large covariance matrix estimation in high-dimensional statistical analysis. Recently, improved iterative algorithms with positive-definite guarantee have been developed. However, these algorithms cannot be directly extended to use a nonconvex penalty for sparsity inducing. Generally, a…
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form . Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and s…
Studied SGD convergence under weak conditions.
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
A faster ADMM method for nonconvex optimization with improved complexity.
Paper proposes a new method to separate low rank and sparse matrices without bias.
Unified framework for nonconvex matrix completion with linearly parameterized factors.
New algorithm for nonconvex optimization on constrained Riemannian manifolds converges quickly.
This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
TiAda adapts adaptive gradient methods for nonconvex minimax optimization.
Two-Timescale EM Methods improve EM for nonconvex models.
We analyse a linear regression problem with nonconvex regularization called smoothly clipped absolute deviation (SCAD) under an overcomplete Gaussian basis for Gaussian random data. We propose an approximate message passing (AMP) algorithm considering nonconvex regularization, namely SCAD-AMP, and analytically show tha…
Adaptive gradient methods are workhorses in deep learning. However, the convergence guarantees of adaptive gradient methods for nonconvex optimization have not been thoroughly studied. In this paper, we provide a fine-grained convergence analysis for a general class of adaptive gradient methods including AMSGrad, RMSPr…
Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative …
In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathemati…
Paper develops methods for statistical inference with SGD in nonconvex optimization.
Study optimization landscapes for overcomplete representations, showing benign geometric structures.
Smooth finite-sum optimization has been widely studied in both convex and nonconvex settings. However, existing lower bounds for finite-sum optimization are mostly limited to the setting where each component function is (strongly) convex, while the lower bounds for nonconvex finite-sum optimization remain largely unsol…
Improved analysis for nonconvex SGD methods with flexible sampling.
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
Paper analyzes SGLD for nonconvex optimization with local conditions.
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data. Both problems can be formulated as an optimization problem with nonsmooth objective and nonconvex constraints. Sinc…
New algorithms solve complex minimax problems without needing derivatives.