Study non-convex manifolds' rigidity properties without convexity assumption.
problem Rigidity properties of non-convex manifolds.
method Analyzes boundary and lens rigidity on non-convex domains, proving rigidity for simply connected and non-trapping surfaces.
result Injectivity of X-ray transform on tensors for non-convex boundaries and non-trapping surfaces.
Manifolds uniquely identified by boundary distance differences.
problem Identifying Riemannian manifolds by their boundary distances.
method Distance difference representation on non-convex boundaries without restrictions.
result Complete Riemannian manifolds uniquely determined by their boundary distances.
We show that any open subset of a contact manifold of dimension greater than three contains a certain non-convex hypersurface violating the Thurston-Bennequin inequality.
Geodesic convex optimization extends convex optimization to manifolds.
problem Optimizing non-convex functions on manifolds.
method Introducing geodesic convexity on manifolds.
result Certain non-convex problems can be formulated as geodesically convex optimization problems.
Accelerated method finds critical points faster on manifolds.
problem Optimization on non-convex manifolds.
method Accelerated gradient methods on Riemannian manifolds.
result Find approximate first-order critical points faster than regular gradient descent.
New method finds near-optimal solutions for non-convex optimization problems.
problem Finding near-optimal solutions for non-convex optimization problems.
method Riemannian stochastic recursive momentum method
result Achieves a near-optimal complexity of ildeO(ε−3). Let $L=\DD+Z$ for a C1 vector field Z on a complete Riemannian manifold possibly with a boundary. By using the uniform distance, a number of transportation-cost inequalities on the path space for the (reflecting) L-diffusion process are proved to be equivalent to the curvature condition $\Ric-\nn Z\ge - K$ and t…
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
ODCGM solves non-convex optimization on manifolds with simpler projections.
problem Minimizing non-convex functions over smooth manifolds.
method Orthogonal Directions Constrained Gradient Method (ODCGM) that projects onto a vector space.
result ODCGM converges to the manifold with near-optimal oracle complexities.
Study non-convex matrix factorization using Riemannian geometry.
problem Matrix completion via non-convex optimization.
method Optimization over a Grassmannian manifold, analyzing principal angles.
result Geodesically convex region in matrix completion cost function.
The paper provides gradient estimates for Neumann semigroups on manifolds with boundary under unbounded curvature conditions.
problem Gradient estimates for Neumann semigroups on manifolds with boundary under unbounded curvature conditions.
method Establishes Bismut-type formulas and gradient estimates for Feynman--Kac semigroups on Riemannian manifolds with boundary, under geometric conditions formulated in terms of Ricci curvature and second fundamental form.
result Derives pointwise gradient estimates for the Neumann semigroup under variable, possibly unbounded, lower curvature bounds.
SGD converges almost surely in non-convex problems, avoiding saddle points and accelerating convergence.
problem Understanding convergence of SGD in non-convex optimization problems.
method Analysis of SGD trajectories, focusing on boundedness, convergence to strict saddle points, and rate of convergence.
result SGD converges almost surely to a minimizer in non-convex problems, avoiding strict saddle points.
New method certifies generative models' robustness.
problem Certifying generative models' robustness is challenging due to non-convex sets.
method ApproxLine, a scalable certification method capturing infinite sets or distributions over them.
result ApproxLine provides sound deterministic and probabilistic guarantees.
Study finds eigenvalue bounds for non-convex domains using cohomology.
problem Eigenvalue bounds for non-convex domains.
method Cohomology, Poincaré-type inequalities, Cheeger-McGowan gluing lemma.
result Established geometric lower bounds for eigenvalues in non-convex domains.
Compact, non-convex curve flows are created.
problem Creating compact, non-convex ancient solutions for curve shortening flow.
method Constructed an ancient solution asymptotic to Yin-Yang curve.
result Compact, non-convex ancient solutions for curve shortening flow are demonstrated.
Paper uses integer programming for non-convex boosting in classification.
problem Improving classification performance using non-convex optimization.
method Non-convex boosting via integer programming.
result Results comparable to or better than state-of-the-art.
First order methods can take extremely long to find global minima of non-convex functions.
problem Finding global minimizers of non-convex functions.
method Designing a family of non-convex functions and using statistical lower bounds for parameter estimation.
result First order methods can take exponential time to converge to a global minimizer.
New algorithm improves convergence for non-convex problems with boundaries.
problem Optimizing non-convex problems with constraints.
method Reflected Gradient Langevin Dynamics with probabilistic representation.
result Promising convergence rates, faster than existing methods.
New method avoids saddle points without gradients.
problem Optimizing non-convex functions efficiently.
method Zero-order derivative-free algorithm using only function evaluations.
result Converges to second-order stationary points efficiently.
New algorithms achieve high probability second-order convergence in non-convex optimization.
problem Stochastic non-convex optimization with high probability second-order convergence.
method Proposed NCG-S updating step and two algorithms.
result First algorithms with high probability second-order convergence and almost linear time complexity.
New diffusions help globally optimize non-convex functions.
problem Optimizing non-convex functions globally.
method Euler discretization of Langevin diffusion.
result Different diffusions optimize different convex and non-convex functions.
A new depth measure for non-convex data supports, faster than halfspace depth.
problem Non-convex data supports in multivariate statistics.
method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).
result The new depth measure is consistent and can be computed faster.
New algorithm finds local minima in non-convex, non-smooth problems.
problem Finding local minimizers in non-convex and non-smooth optimization.
method Perturbed Proximal Descent, tailored for non-smooth cases.
result First known results for non-smooth optimization.
We introduce a new local regret framework for non-convex models in dynamic environments.
problem Challenges in online forecasting for non-convex models with frequent updates and concept drift.
method We propose a novel local regret framework and a time-smoothed gradient update rule.
result Our approach yields more stable, robust, and computationally efficient forecasting compared to state-of-the-art methods.
A new non-convex method improves robust PCA with features.
problem Robust Principal Component Analysis with prior feature information.
method A novel non-convex optimization approach for decomposition.
result Exact recovery guarantees with low computational complexity.
New algorithm tackles non-convex matrix completion in semi-random settings.
problem Matrix completion in semi-random environments with varying observation probabilities.
method Proposes a pre-processing step to re-weight semi-random input, followed by a nearly-linear time algorithm.
result Recovering ground-truth matrix using non-convex local minima after pre-processing.
Efficiently minimizes regret in non-convex games with gradient-based methods.
problem Computational intractability of standard regret minimization in non-convex games.
method Defining a new notion of regret and using gradient-based optimization methods.
result Achieves optimal regret, leading to convergence to equilibrium.
In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…
This work shows neural networks can solve non-convex constraints problems.
problem Training neural networks under non-convex constraints.
method Project stochastic gradient descent with no-regret analysis of online learning.
result Overparameterized neural networks achieve near-optimal and near-feasible solutions.
New algorithms optimize non-smooth, non-convex objectives with improved complexity.
problem Optimizing non-smooth, non-convex stochastic objectives.
method Reduction to online learning, applying optimistic online learning techniques.
result Improved complexity for finding (δ,ε)-stationary points. SGD's uncertainty quantified in non-convex learning problems.
problem Uncertainty quantification in non-convex learning problems.
method Asymptotic normality of SGD iterates and bias characterization.
result SGD iterates are asymptotically normally distributed around the expected value of the invariant distribution.
Deep neural networks with multiple branches are less non-convex, improving performance.
problem Improving neural network performance through multi-branch architectures.
method Quantitative measurement of duality gap for neural networks with multi-branches and various activation functions.
result The duality gap of multi-branch neural networks decreases as the number of branches increases, leading to less non-convex optimization problems.
Optimizers find approximate global minima in non-convex problems.
problem Understanding why local methods solve non-convex optimization problems.
method Formalizing the hypothesis that many local minima are approximately global minima.
result Most local minima of practical non-convex objectives are approximately global minima.
SGHMC uses noise to find global minima in non-convex learning.
problem Finding global minima in non-convex optimization problems.
method SGHMC with momentum and Gaussian noise for non-asymptotic convergence.
result Non-asymptotic convergence analysis for non-convex optimization.
New insights into using momentum for non-convex optimization.
problem Improving training of non-convex models like deep neural networks.
method Developed a Lyapunov analysis of SGD with momentum using stochastic primal averaging.
result Precise conditions under which SGD+M outperforms SGD and optimal hyper-parameter schedules.
Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…
Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
This work explores the non-convex optimization in compressive learning and the performance of heuristics.
problem The challenge of learning from compressed representations in compressive learning.
method Numerical simulations of the non-convex optimization landscape and heuristic performance.
result Properties of the non-convex optimization landscape and heuristic performance are explored.
This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
Paper proposes a new method for MRI data recovery using bi-linear modeling.
problem Recovering high-fidelity MRI data from dynamic sequences.
method Bi-linear modeling framework for manifold learning and sparse approximation.
result The method improves MRI data recovery over existing techniques.
We study how gradient convergence speeds up in non-convex learning tasks.
problem Understanding the convergence of gradients in non-convex learning problems.
method We propose vector-valued Rademacher complexities to derive uniform convergence bounds for gradients in non-convex learning problems.
result We show that for non-convex models, gradient convergence can be dimension-independent under certain distributional assumptions.
This monograph explores non-convex optimization techniques for machine learning.
problem Capturing complex learning and prediction problems with non-convex optimization.
method Analyzes various non-convex optimization techniques and their applications.
result Direct non-convex optimization methods often outperform relaxation-based techniques.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
Paper shows non-convex loss functions can be optimized efficiently.
problem Optimizing non-convex loss functions is challenging.
method Uses stochastic variance reduction methods to find global optimal solutions.
result Stochastic variance reduction methods converge to global optimal with linear rate.
Paper proposes a working set algorithm for non-convex sparse regression with provable convergence.
problem Estimating sparse linear models from high-dimensional data using non-convex regularizers.
method FireWorks algorithm based on non-convex reformulation and leveraging residual geometry.
result Convergence to a stationary point of the full problem with provable guarantees.
FTPL achieves optimal regret in online non-convex learning.
problem Online non-convex learning with non-convex losses.
method Follow the Perturbed Leader (FTPL) algorithm.
result FTPL achieves optimal regret rate of O(T−1/2). Improved convergence analysis for decentralized non-convex optimization.
problem Minimizing a sum of smooth non-convex functions over a network.
method Gradient tracking in decentralized stochastic gradient descent (GT-DSGD).
result GT-DSGD achieves network-independent performances matching centralized SGD under certain conditions.
Paper introduces risk measures for non-convex portfolios.
problem Risk measurement in non-convex transaction costs models.
method Analyzes all portfolio selections to find acceptable positions.
result Properties and examples of non-convex portfolio risk measures.