New method estimates large covariance matrices using nonconvex penalties.
problem Estimating large covariance matrices in high-dimensional data.
method Developed a first-order algorithm using generalized nonconvex penalties.
result Positive-definite covariance estimators using nonconvex penalties.
New framework explains why nonconvex methods work well in low-rank matrix estimation.
problem Nonconvex low-rank matrix estimation problems in machine learning.
method Developed a theoretical framework revealing a benign regularizer.
result Nonconvex procedures can behave well due to a disguised convexity.
Sharp estimates for heat flow on nonconvex domains.
problem Quantitative estimates for heat flow on nonconvex domains.
method Sharp gradient and transport estimates with novel dependence on time.
result Equivalent characterization of lower bound on second fundamental form.
The paper explores nonconvex penalties using Bernstein functions for sparse estimation.
problem Sparse estimation in high-dimensional problems.
method Nonconvex penalties based on Bernstein functions, with coordinate descent and proximal alternating linearized minimization methods.
result The Bernstein penalty leads to effective sparse estimation and classification.
Paper develops methods for statistical inference with SGD in nonconvex optimization.
problem Statistical inference for nonconvex optimization problems.
method Proposes two online inferential procedures combining SGD and bootstrap techniques.
result Establishes error convergence rates and asymptotically valid bootstrap confidence intervals.
Study uncovers statistical optimality of nonconvex tensor completion methods.
problem Estimating a low-rank tensor from incomplete and corrupted observations.
method Two-stage estimation algorithm for nonconvex optimization.
result Nonconvex tensor completion achieves optimal ℓ2 accuracy. New hybrid SGD algorithms improve stochastic nonconvex optimization complexity.
problem Solving stochastic nonconvex optimization problems efficiently.
method Hybrid SARAH-SGD algorithm combining biased and unbiased estimators.
result Achieves better complexity bound for ε-stationary points. Unified framework for nonconvex low-rank matrix estimation using gradient descent.
problem Estimating low-rank matrices in noisy and noiseless settings.
method Gradient descent algorithm applied to nonconvex optimization.
result Unified framework guarantees linear convergence to the unknown low-rank matrix with optimal statistical error.
PAGE is a simple gradient estimator for nonconvex optimization problems.
problem Nonconvex optimization problems in machine learning.
method PAGE is a probabilistic gradient estimator that uses vanilla SGD with probability and a small adjustment with probability 1-p.
result PAGE achieves optimal convergence rates for nonconvex finite-sum and online problems.
New algorithm solves high-dimensional regression and precision matrix estimation problems efficiently.
problem High-dimensional multivariate regression and precision matrix estimation.
method Gradient descent with hard thresholding for nonconvex optimization.
result Algorithm achieves optimal statistical rate with provable convergence.
Paper proposes a nonconvex approach for sparse reduced rank regression.
problem Sparse reduced rank regression model estimation problem.
method Formulated as a nonconvex optimization problem with alternating minimization method.
result Nonconvex function leads to better estimation accuracy and efficiency.
Quantum annealing improves EM algorithm's performance in nonconvex optimization.
problem EM algorithm's tendency to get stuck in local optima for nonconvex problems.
method Integrates quantum fluctuations into EM algorithm to induce tunnel effect and avoid local optima.
result Quantum annealing EM algorithm converges and performs better in nonconvex optimization problems.
We establish Evans-Krylov estimates for certain nonconvex fully nonlinear elliptic and parabolic equations by exploiting partial Legendre transformations. The equations under consideration arise in part from the study of the "pluriclosed flow" introduced by the first author and Tian
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
Unified framework for nonconvex matrix completion with linearly parameterized factors.
problem Matrix completion with improved accuracy using linearly parameterized factors.
method Unified nonconvex optimization framework with Correlated Parametric Factorization condition.
result Uniform upper bounds for low-rank estimation at any local minimum.
We present a unified framework for low-rank matrix estimation with nonconvex penalties. We first prove that the proposed estimator attains a faster statistical rate than the traditional low-rank matrix estimator with nuclear norm penalty. Moreover, we rigorously show that under a certain condition on the magnitude of t…
We provide theoretical analysis of the statistical and computational properties of penalized M-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …
Paper proposes a new optimizer for faster nonconvex optimization.
problem Optimizing nonconvex objectives efficiently and quickly.
method Integrates stochastic and biased gradient estimation with a hyper-parameter.
result The hyper-parameter can be configured to improve convergence rate.
The MM algorithm improves robust penalized estimation for outlier-contaminated data.
problem Outliers in data affect the reliability of penalized estimation.
method Innovative MM algorithm for both convex and nonconvex loss functions.
result Established convergence theory for MM algorithm with various loss functions.
Sparse feature selection has been demonstrated to be effective in handling high-dimensional data. While promising, most of the existing works use convex methods, which may be suboptimal in terms of the accuracy of feature selection and parameter estimation. In this paper, we expand a nonconvex paradigm to sparse group …
A new hybrid algorithm reduces stochastic gradient evaluations for nonconvex optimization.
problem Solving stochastic composite nonconvex optimization problems efficiently.
method Proposes a new hybrid variance-reduced proximal gradient method with a stochastic gradient estimator.
result Achieves optimal stochastic oracle complexity bound with one less gradient evaluation.
The paper addresses nonconvex penalized LAD estimation in partial linear models using DNNs.
problem Challenges in nonconvex penalized LAD estimation with DNNs in partial linear models.
method Parameterizes nonparametric term with DNNs, formulates penalized LAD problem, introduces proximal subgradient method.
result Establishes consistency, convergence rate, and asymptotic normality of the estimator.
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular l1 technique suffers from both selection inconsisten…
The paper introduces ADMM methods with variance reduction for nonconvex optimization.
problem Nonconvex optimization problems.
method Stochastic ADMM with variance reduction.
result Iteration complexity bound of O(1/ε) for obtaining an ε-stationary solution. In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathemati…
Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.
problem Nonconvex-concave minimax optimization problems.
method Two shuffling gradient-based algorithms for nonconvex-linear and nonconvex-strongly concave settings.
result Achieves state-of-the-art oracle complexity in nonconvex optimization and best-known complexity bounds for nonconvex-strongly concave setting.
Paper proposes a method to make nonconvex optimization more efficient.
problem Nonconvex regularizers improve performance but are harder to optimize.
method Redistribute nonconvexity from regularizer to loss, making it convex.
result Optimization with convexified regularizer is faster and more efficient.
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
New algorithm completes noisy tensors quickly and accurately.
problem Reconstructing low-rank tensors from incomplete and noisy data.
method Two-stage nonconvex gradient descent algorithm.
result Achieves near-optimal statistical guarantees and linear time complexity.
Paper analyzes SGLD for nonconvex optimization with local conditions.
problem Analyzing sampling algorithms for nonconvex optimization.
method Non-asymptotic estimates for SGLD under local conditions.
result Establishes error bounds for expected excess risk.
Safe reinforcement learning with nonconvex constraints using convex approximations.
problem Safe reinforcement learning with nonlinear function approximation.
method Constructing surrogate convex constrained optimization problems by replacing nonconvex functions with convex quadratic functions.
result Solutions to surrogate problems converge to a stationary point of the original nonconvex problem.
Paper learns Markov models from data with low-rank optimization.
problem Learning Markov models from a single trajectory with latent structure.
method Two maximum likelihood estimation methods: convex with nuclear-norm regularization and nonconvex with rank constraint. Novel DC programming algorithm for nonconvex estimator.
result Accurate estimation of full transition model with trajectory length proportional to state space.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
New method reduces complexity for nonconvex optimization problems.
problem Minimizing composite functions with random or finite sum inner mappings.
method Stochastic composite gradient method with incremental variance reduction.
result Achieves complexity similar to best first-order methods for expected-value and finite-sum nonconvex functions.
New algorithm framework solves stochastic composite nonconvex optimization problems efficiently.
problem Solving stochastic composite nonconvex optimization problems.
method ProxSARAH framework using SARAH estimator with proximal gradient and averaging steps.
result Achieves best-known complexity bounds with constant and adaptive step-sizes.
Proposes a new method to improve estimation in Gaussian graphical models.
problem Optimal estimation in high-dimensional Gaussian graphical models.
method Graphical nonconvex optimization, approximated by a sequence of convex programs.
result Achieves the oracle rate of convergence and outperforms other methods.
The paper analyzes spectral initialization for nonconvex estimation, revealing phase transitions and computational complexities.
problem Estimating signals in nonconvex settings with spectral initialization.
method Arbitrary generalized linear sensing models, high-dimensional limit analysis.
result Spectral method performance has phase transitions and computational complexity depends on sample-to-signal dimension ratio.
New algorithm converges to optimal phase retrieval estimator with misspecified link functions.
problem High-dimensional sparse phase retrieval with incorrect model specification.
method Simple variant of thresholded Wirtinger flow algorithm, linear convergence for optimal accuracy.
result Linear convergence to optimal estimator for a broad family of unknown link functions.
We provide novel theoretical results regarding local optima of regularized M-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…
New algorithm speeds up LVGGM estimation by solving nonconvex optimization.
problem Estimating the latent variable Gaussian graphical model with sparse and low-rank components.
method Sparsity constrained maximum likelihood estimator with alternating gradient descent and hard thresholding.
result Our algorithm converges linearly to the optimal components up to statistical precision.
We give a lower estimate of the gap of the first two eigenvalues of the Schrodinger operator with a nonconvex potential in terms of a distance associated with the potential. The results here can be applied to the double well potential.
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
Paper proposes a new method to separate low rank and sparse matrices without bias.
problem Recovering low rank and sparse matrices from measurements.
method Uses nonconvex regularizers and alternating proximal gradient descent.
result Error bounds for the algorithm applied to sparse optimization, matrix completion, and robust PCA.
New algorithm solves nonconvex-convex minimax problems efficiently.
problem Solving nonconvex-convex minimax problems with nonsmooth, nonconvex, and nonlinearity.
method Hybrid variance-reduced SGD algorithm combining smoothing and biased techniques.
result Achieves O(T^(-2/3)) convergence rate and best oracle complexity.
Paper proposes estimating gradients for zeroth-order nonconvex optimization.
problem Oracle access of gradients is limited in many applications.
method Develops a gradient descent method using estimated gradients.
result Algorithm finds second-order stationary points efficiently.
Gradient descent implicitly regularizes nonconvex problems, achieving near-optimal results.
problem Statistical estimation problems like phase retrieval, matrix completion, and blind deconvolution.
method Gradient descent without explicit regularization.
result Gradient descent achieves near-optimal statistical and computational guarantees.
Sparse principal component analysis (PCA) involves nonconvex optimization for which the global solution is hard to obtain. To address this issue, one popular approach is convex relaxation. However, such an approach may produce suboptimal estimators due to the relaxation effect. To optimally estimate sparse principal su…
Paper improves understanding of noisy matrix completion using convex relaxation and nonconvex optimization.
problem Estimating a low-rank matrix from noisy partial entries.
method Combining convex relaxation and the nonconvex Burer-Monteiro approach.
result Convex relaxation achieves near-optimal estimation errors for noisy matrix completion.