Compact, non-convex curve flows are created.
problem Creating compact, non-convex ancient solutions for curve shortening flow.
method Constructed an ancient solution asymptotic to Yin-Yang curve.
result Compact, non-convex ancient solutions for curve shortening flow are demonstrated.
Paper uses integer programming for non-convex boosting in classification.
problem Improving classification performance using non-convex optimization.
method Non-convex boosting via integer programming.
result Results comparable to or better than state-of-the-art.
First order methods can take extremely long to find global minima of non-convex functions.
problem Finding global minimizers of non-convex functions.
method Designing a family of non-convex functions and using statistical lower bounds for parameter estimation.
result First order methods can take exponential time to converge to a global minimizer.
New algorithm improves convergence for non-convex problems with boundaries.
problem Optimizing non-convex problems with constraints.
method Reflected Gradient Langevin Dynamics with probabilistic representation.
result Promising convergence rates, faster than existing methods.
New algorithms achieve high probability second-order convergence in non-convex optimization.
problem Stochastic non-convex optimization with high probability second-order convergence.
method Proposed NCG-S updating step and two algorithms.
result First algorithms with high probability second-order convergence and almost linear time complexity.
New diffusions help globally optimize non-convex functions.
problem Optimizing non-convex functions globally.
method Euler discretization of Langevin diffusion.
result Different diffusions optimize different convex and non-convex functions.
New algorithm finds local minima in non-convex, non-smooth problems.
problem Finding local minimizers in non-convex and non-smooth optimization.
method Perturbed Proximal Descent, tailored for non-smooth cases.
result First known results for non-smooth optimization.
We introduce a new local regret framework for non-convex models in dynamic environments.
problem Challenges in online forecasting for non-convex models with frequent updates and concept drift.
method We propose a novel local regret framework and a time-smoothed gradient update rule.
result Our approach yields more stable, robust, and computationally efficient forecasting compared to state-of-the-art methods.
A new non-convex method improves robust PCA with features.
problem Robust Principal Component Analysis with prior feature information.
method A novel non-convex optimization approach for decomposition.
result Exact recovery guarantees with low computational complexity.
New algorithm tackles non-convex matrix completion in semi-random settings.
problem Matrix completion in semi-random environments with varying observation probabilities.
method Proposes a pre-processing step to re-weight semi-random input, followed by a nearly-linear time algorithm.
result Recovering ground-truth matrix using non-convex local minima after pre-processing.
Efficiently minimizes regret in non-convex games with gradient-based methods.
problem Computational intractability of standard regret minimization in non-convex games.
method Defining a new notion of regret and using gradient-based optimization methods.
result Achieves optimal regret, leading to convergence to equilibrium.
In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…
This work shows neural networks can solve non-convex constraints problems.
problem Training neural networks under non-convex constraints.
method Project stochastic gradient descent with no-regret analysis of online learning.
result Overparameterized neural networks achieve near-optimal and near-feasible solutions.
New algorithms optimize non-smooth, non-convex objectives with improved complexity.
problem Optimizing non-smooth, non-convex stochastic objectives.
method Reduction to online learning, applying optimistic online learning techniques.
result Improved complexity for finding ( δ , ε ) (δ,ε) ( δ , ε ) -stationary points. SGD's uncertainty quantified in non-convex learning problems.
problem Uncertainty quantification in non-convex learning problems.
method Asymptotic normality of SGD iterates and bias characterization.
result SGD iterates are asymptotically normally distributed around the expected value of the invariant distribution.
Deep neural networks with multiple branches are less non-convex, improving performance.
problem Improving neural network performance through multi-branch architectures.
method Quantitative measurement of duality gap for neural networks with multi-branches and various activation functions.
result The duality gap of multi-branch neural networks decreases as the number of branches increases, leading to less non-convex optimization problems.
Optimizers find approximate global minima in non-convex problems.
problem Understanding why local methods solve non-convex optimization problems.
method Formalizing the hypothesis that many local minima are approximately global minima.
result Most local minima of practical non-convex objectives are approximately global minima.
SGHMC uses noise to find global minima in non-convex learning.
problem Finding global minima in non-convex optimization problems.
method SGHMC with momentum and Gaussian noise for non-asymptotic convergence.
result Non-asymptotic convergence analysis for non-convex optimization.
New insights into using momentum for non-convex optimization.
problem Improving training of non-convex models like deep neural networks.
method Developed a Lyapunov analysis of SGD with momentum using stochastic primal averaging.
result Precise conditions under which SGD+M outperforms SGD and optimal hyper-parameter schedules.
Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…
Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
This work explores the non-convex optimization in compressive learning and the performance of heuristics.
problem The challenge of learning from compressed representations in compressive learning.
method Numerical simulations of the non-convex optimization landscape and heuristic performance.
result Properties of the non-convex optimization landscape and heuristic performance are explored.
New method finds near-optimal solutions for non-convex optimization problems.
problem Finding near-optimal solutions for non-convex optimization problems.
method Riemannian stochastic recursive momentum method
result Achieves a near-optimal complexity of i l d e O ( ε − 3 ) ilde{\mathcal{O}}(ε^{-3}) i l d e O ( ε − 3 ) . This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
We study how gradient convergence speeds up in non-convex learning tasks.
problem Understanding the convergence of gradients in non-convex learning problems.
method We propose vector-valued Rademacher complexities to derive uniform convergence bounds for gradients in non-convex learning problems.
result We show that for non-convex models, gradient convergence can be dimension-independent under certain distributional assumptions.
This monograph explores non-convex optimization techniques for machine learning.
problem Capturing complex learning and prediction problems with non-convex optimization.
method Analyzes various non-convex optimization techniques and their applications.
result Direct non-convex optimization methods often outperform relaxation-based techniques.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
Paper shows non-convex loss functions can be optimized efficiently.
problem Optimizing non-convex loss functions is challenging.
method Uses stochastic variance reduction methods to find global optimal solutions.
result Stochastic variance reduction methods converge to global optimal with linear rate.
Paper proposes a working set algorithm for non-convex sparse regression with provable convergence.
problem Estimating sparse linear models from high-dimensional data using non-convex regularizers.
method FireWorks algorithm based on non-convex reformulation and leveraging residual geometry.
result Convergence to a stationary point of the full problem with provable guarantees.
FTPL achieves optimal regret in online non-convex learning.
problem Online non-convex learning with non-convex losses.
method Follow the Perturbed Leader (FTPL) algorithm.
result FTPL achieves optimal regret rate of O ( T − 1 / 2 ) O(T^{-1/2}) O ( T − 1/2 ) . Improved convergence analysis for decentralized non-convex optimization.
problem Minimizing a sum of smooth non-convex functions over a network.
method Gradient tracking in decentralized stochastic gradient descent (GT-DSGD).
result GT-DSGD achieves network-independent performances matching centralized SGD under certain conditions.
Paper introduces risk measures for non-convex portfolios.
problem Risk measurement in non-convex transaction costs models.
method Analyzes all portfolio selections to find acceptable positions.
result Properties and examples of non-convex portfolio risk measures.
Improved optimization guarantees for deep learning models with Nesterov acceleration.
problem Optimization in non-convex deep learning landscapes.
method Analysis of Nesterov acceleration in benignly non-convex landscapes.
result Identical guarantees can be obtained in optimization problems with weak geometric assumptions, especially in overparametrized deep learning.
Adaptive momentum method solves non-convex min-max problems.
problem Non-convex min-max optimization problems in training generative adversarial networks.
method Proposes an adaptive momentum algorithm for non-convex min-max optimization.
result Establishes non-asymptotic convergence rates for the proposed algorithm.
This study improves graph signal denoising for vector-valued data with non-convex penalties.
problem Denoising piecewise smooth graph signals with varying smoothness levels.
method Extended graph trend filtering with non-convex penalties and ADMM algorithm.
result Non-convex penalties outperform convex ones in recovery performance.
Study non-convex manifolds' rigidity properties without convexity assumption.
problem Rigidity properties of non-convex manifolds.
method Analyzes boundary and lens rigidity on non-convex domains, proving rigidity for simply connected and non-trapping surfaces.
result Injectivity of X-ray transform on tensors for non-convex boundaries and non-trapping surfaces.
Non-convex extremal length found in surface metrics.
problem Extremal length functions on surfaces are not always convex.
method Used harmonic maps to R \mathbb{R} R -trees and minimal surfaces in R n \mathbb{R}^n R n . result Found measured foliations with non-convex extremal length functions.
Quantum annealing outperforms classical in solving non-convex optimization problems crucial for machine learning.
problem Solving non-convex optimization problems efficiently.
method Designing a classical energy function and adding a quantum transverse field to facilitate tunneling.
result Quantum annealing converges efficiently to optimal solutions in a wide class of non-convex problems, unlike classical thermal annealing.
Study on equilibrium with non-convex preferences.
problem Existence of equilibrium in non-convex preference settings.
method Provided a necessary and sufficient condition for equilibrium existence.
result Standard equilibrium theory cannot be applied to non-convex preferences.
Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.
problem Estimating differences in multi-attribute Gaussian graphical models with similar structure.
method Penalized D-trace loss function with non-convex (log-sum and SCAD) penalties, proximal gradient descent methods.
result Theoretical analysis and numerical examples support consistency in support recovery and estimation.
Paper solves curvature equations in Minkowski space for non-convex domains.
problem Solving curvature equations in non-convex domains of Minkowski space.
method Existence theorem proved via \emph{a priori} estimates and Serrin-type condition.
result Existence of solutions for curvature equations in non-convex domains.
Online SGD from random init solves non-smooth, non-convex phase retrieval.
problem Solving phase retrieval with non-smooth, non-convex loss functions.
method Online stochastic gradient descent (SGD) with constant step size, starting from arbitrary initialization.
result SGD converges from arbitrary initializations for the amplitude squared loss objective.
Paper tackles non-convex inf-projection problems with stochastic optimization.
problem Non-convex and possibly non-smooth inf-projection minimization problems.
method Developed stochastic algorithms for finding (nearly) stationary solutions.
result Established first-order convergence for non-convex inf-projection problems.
This paper accelerates gradient methods to find local minima in non-convex optimization.
problem Finding local minima in non-convex optimization problems.
method Polyak's Heavy Ball method and Nesterov's Accelerated Gradient method for extracting negative curvature.
result A new AG algorithm converges to second-order stationary points with improved iteration complexity.
A fast method for decentralized non-convex optimization over networks.
problem Decentralized non-convex optimization problems over a network of nodes.
method GT-SAGA, a randomized incremental gradient method that evaluates one component gradient per node per iteration.
result GT-SAGA achieves almost sure and mean-squared convergence to a first-order stationary point for general smooth non-convex problems.
SGD method converges to minima for non-convex functions.
problem Non-convex optimization problems in machine learning.
method Stochastic gradient descent method for non-convex objective functions.
result Estimates on the rate of convergence to minima.
Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning settings. However, solving the non-convex optimization problems associated with non-conv…
Asynchronous L-BFGS speeds up non-convex optimization.
problem Non-convex optimization challenges in machine learning.
method Asynchronous stochastic L-BFGS algorithm for non-convex optimization.
result Achieves an ergodic convergence rate of O ( 1 / N ) {\cal O}(1/\sqrt{N}) O ( 1/ N ) and linear speedup.