New global section found for geodesic flows on convex hypersurfaces.
problem Finding global sections for geodesic flows on convex hypersurfaces.
method Constructing a global hypersurface of section with an isometric involution.
result Generalized Birkhoff annulus to higher dimensions.
First order methods can take extremely long to find global minima of non-convex functions.
problem Finding global minimizers of non-convex functions.
method Designing a family of non-convex functions and using statistical lower bounds for parameter estimation.
result First order methods can take exponential time to converge to a global minimizer.
SGD converges to global minimum for certain non-convex functions.
problem Theoretical challenges in optimizing non-convex functions in machine learning.
method Perturbed SGD on a broad class of non-convex functions.
result SGD converges to global minimum for certain non-convex functions.
AGGLIO optimizes non-convex functions with local convexity guarantees.
problem Optimizing non-convex functions with local convexity.
method Stage-wise, graduated optimization technique for locally convex functions.
result Global convergence to the global optimum for non-convex and locally convex objectives.
Convex curves evolve into circles over time.
problem Deforming convex curves into circles.
method Generalized length-preserving flow for convex curves.
result Convex curves evolve into circles over time.
This work is devoted to a systematic study of symplectic convexity for integrable Hamiltonian systems with elliptic and focus-focus singularities. A distinctive feature of these systems is that their base spaces are still smooth manifolds (with boundary and corners), similarly to the toric case, but their associated in…
Proof shows local convexity implies global convexity in special geometric spaces.
problem Proving convexity in CAT(0) cubed complexes from local convexity.
method Analyzes vertex link structures to determine convexity.
result Local combinatorial properties determine global convexity.
New algorithm improves convergence for non-convex problems with boundaries.
problem Optimizing non-convex problems with constraints.
method Reflected Gradient Langevin Dynamics with probabilistic representation.
result Promising convergence rates, faster than existing methods.
We introduce a notion of probabilistic convexity and generalize some classical globalization theorems in Alexandrov geometry. A weighted Alexandrov's lemma is developed as a basic tool.
Optimizers find approximate global minima in non-convex problems.
problem Understanding why local methods solve non-convex optimization problems.
method Formalizing the hypothesis that many local minima are approximately global minima.
result Most local minima of practical non-convex objectives are approximately global minima.
This paper finds a global surface of section in dynamically convex L(p,p-1) using ECH.
problem Finding a global surface of section in dynamically convex L(p,p-1).
method Using Embedded Contact Homology (ECH).
result Relates periods of the surface of section to the first ECH spectrum.
An Euler discretization of the Langevin diffusion is known to converge to the global minimizers of certain convex and non-convex optimization problems. We show that this property holds for any suitably smooth diffusion and that different diffusions are suitable for optimizing different classes of convex and non-convex …
Study extends convexity in curved spaces using fractional integrals.
problem Extending convexity to curved spaces with nonpositive curvature.
method Introducing (geodesically) h-convex functions and using Katugampola's fractional integrals. result Essentially sharp estimate involving squared distance mappings.
Accelerates convergence in global non-convex optimization with reversible diffusion.
problem Global non-convex optimization challenges.
method Utilizes reversible diffusion processes with adaptive diffusion coefficients.
result Accelerated convergence with reduced discretization error.
We solve the optimization of two-layer ReLU networks using convex math.
problem Optimizing two-layer ReLU neural networks.
method Exact characterization of optimal solutions via convex optimization.
result We prove that all globally optimal solutions can be found via convex optimization.
Paper develops exact convex optimization for neural networks with polynomial activations.
problem Training two-layer neural networks with nonlinear polynomial activations.
method Exact convex optimization using semidefinite programming.
result Global optimization of neural networks is polynomial-time computable.
Proves flows of two-convex Lagrangians are regular, global, and converge.
problem Proves regularity, global existence, and convergence of Lagrangian mean curvature flows in the two-convex case.
method Uses a newly discovered monotone quantity to control two-convexity.
result Proves results for the mean curvature flow of area-decreasing Lagrangian submanifolds.
The paper constructs Levi flat structures using structure sheaves and differential complexes.
problem Global solvability and regularity of Levi flat structures.
method Employing formal integrability and differential complexes, the paper constructs a resolution for the structure sheaf.
result Global exactness and Sobolev regularity of the differential complex for Levi flat structures.
Geodesic convexity generalizes the notion of (vector space) convexity to nonlinear metric spaces. But unlike convex optimization, geodesically convex (g-convex) optimization is much less developed. In this paper we contribute to the understanding of g-convex optimization by developing iteration complexity analysis for …
New methods optimize functions on hyperbolic and spherical spaces, matching Euclidean rates up to logarithmic factors.
problem Optimizing functions on non-Euclidean spaces like hyperbolic and spherical geometries.
method Introduced accelerated global first-order methods for L-smooth and geodesically convex functions on hyperbolic and spherical spaces. result Achieved the same rates as accelerated gradient descent in Euclidean space, up to logarithmic factors.
We derive a Bernstein type result for the special Lagrangian equation, namely, any global convex solution must be quadratic. In terms of minimal surfaces, the result says that any global minimal Lagrangian graph with convex potential must be a hyper-plane.
We propose convex relaxations for convolutional neural nets with one hidden layer where the output weights are fixed. For convex activation functions such as rectified linear units, the relaxations are convex second order cone programs which can be solved very efficiently. We prove that the relaxation recovers the glob…
New method trains quantized neural networks to global optimality.
problem Training optimal quantized neural networks is intractable due to combinatorial non-convex optimization.
method Convex optimization strategy using hidden convexity, semidefinite lifting, and Grothendieck's identity.
result Quantized NN problems can be solved to global optimality in polynomial-time.
This paper proposes a new global optimization algorithm using deep learning.
problem Developing efficient algorithms for global optimization of non-convex functions.
method Two-phase approach: minimization phase with model-driven deep learning, escaping phase with reinforcement learning.
result The proposed algorithm significantly outperforms classical optimization methods and handles ill-posed functions.
This paper tackles global Nash equilibrium in non-convex multi-player games.
problem Challenges in finding global Nash equilibrium due to non-convexity.
method Conjugate transformation and variational inequality formulation to prove existence and design algorithms.
result Designs an ODE-based algorithm with exponential convergence rate and proves its effectiveness in practical scenarios.
ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.
problem Non-convex optimization challenges in machine learning.
method Energy Conserving Descent (ECD) algorithm, stochastic ECD dynamics (sECD), quantum ECD Hamiltonian (qECD).
result ECD and its quantum version achieve exponential speedup over gradient descent.
Unified framework reveals regularization mechanism in deep ReLU networks via convex optimization.
problem Understanding the success of deep neural networks.
method Developed a unified framework using convex optimization to reveal regularization mechanisms.
result ReLU networks can be globally optimized via convex programs, enforcing sparsity.
Paper proves global optimality of a simple optimization scheme for learning DAG models.
problem Learning acyclic directed graphical models from data.
method Path-following optimization scheme for bivariate setting.
result Simple optimization scheme globally converges to global minimum.
Paper solves optimal portfolio deleveraging with cross asset impacts.
problem Maximize equity while meeting debt/equity requirement with cross asset price impacts.
method Developed successive convex optimization (SCO) and an effective global algorithm integrating SCO, convex relaxation, and branch-and-bound.
result Proposed algorithms find global optimal solutions efficiently.
Bayesian method clusters data and selects variables with shrinkage priors.
problem Sparse convex clustering with limited data accuracy issues.
method Bayesian approach using global-local shrinkage priors and Gibbs sampling.
result Improved estimation accuracy in sparse convex clustering.
The TREX is a recently introduced method for performing sparse high-dimensional regression. Despite its statistical promise as an alternative to the lasso, square-root lasso, and scaled lasso, the TREX is computationally challenging in that it requires solving a non-convex optimization problem. This paper shows a remar…
ProGO optimizes non-convex functions without gradients, outperforming existing methods.
problem Challenges in global optimization, especially with non-convex functions and limited gradient information.
method Probabilistic approach using multidimensional integration and latent slice sampler.
result ProGO converges to global optima efficiently and outperforms existing methods.
While optimizing convex objective (loss) functions has been a powerhouse for machine learning for at least two decades, non-convex loss functions have attracted fast growing interests recently, due to many desirable properties such as superior robustness and classification accuracy, compared with their convex counterpa…
Global convergence for robust regression problems via IRLS with enhancements.
problem Global convergence for robust regression problems.
method Augmentations to IRLS to ensure global recovery and improved robustness.
result Global recovery guarantees for robust regression problems, outperforming state-of-the-art algorithms.
Locally convex compact immersed hypersurfaces in Finsler-Hadamard manifolds with bounded T-curvature are considered. We prove that such hypersurfaces are embedded as the boundary of convex body under certain conditions on the normal curvatures
Let U⊆Rd be open and convex. We prove that every (not necessarily Lipschitz or strongly) convex function f:U→R can be approximated by real analytic convex functions, uniformly on all of U. We also show that C0-fine approximation of convex functions by smooth (or real analytic) conv…
Techniques involving factorization are found in a wide range of applications and have enjoyed significant empirical success in many fields. However, common to a vast majority of these problems is the significant disadvantage that the associated optimization problems are typically non-convex due to a multilinear form or…
We prove that any complete immersed globally orientable uniformly 2-convex translating soliton Σ⊂Rn+1 for the mean curvature flow is locally strictly convex. It follows that a uniformly 2-convex entire graphical translating soliton in Rn+1,n≥3 is the axisymmetric "bowl soliton…
We consider globally hyperbolic maximal anti de Sitter 3-manifolds M with a closed Cauchy surface S of genus greater than one and prove that any pair of hyperbolic metrics on S can be realized as the boundary metrics of the convex core of a maximal globally hyperbolic anti de Sitter 3-manifold structure on M. T…
Paper proposes a new algorithm combining gradient descent and Langevin dynamics.
problem Gradient descent can get stuck in local minima, while Langevin dynamics can explore but is slow.
method Replica exchange mechanism swaps positions if Langevin yields a lower objective function.
result New algorithm converges to global minimum linearly with high probability.
Polynomial-time convex optimization for CNNs with ReLU activations.
problem Training Convolutional Neural Networks (CNNs) with ReLU activations.
method Developed a convex analytic framework using semi-infinite duality to formulate equivalent convex optimization problems for CNN architectures.
result Proved that two-layer CNNs can be globally optimized via an ℓ2 norm regularized convex program. Convex clustering solves a stable optimization problem for clustering.
problem Clustering with stable and scalable solutions.
method Solving a convex optimization problem with a single tuning parameter.
result The optimization problem has a unique global minimizer stable to inputs.
New framework for DNN training guarantees convergence to global minimum.
problem Training deep neural networks to converge to global minimum.
method Reformulated minimization problem with recursive algorithmic framework, using bounded style assumptions.
result Convergence to an ε-(global) minimum with O(1/ε^3) gradient computations.
We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…
We give a variational proof of the existence and uniqueness of a convex cap with the given upper boundary. The proof uses the concavity of the total scalar curvature functional on the space of generalized convex caps. As a byproduct, we prove that generalized convex caps with the fixed boundary are globally rigid, that…
This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by…
Under a convexity assumption on the boundary we solve a local inverse problem, namely we show that the geodesic X-ray transform can be inverted locally in a stable manner; one even has a reconstruction formula. We also show that under an assumption on the existence of a global foliation by strictly convex hypersurfaces…
Over-parameterization makes optimization easier for simple neural networks, even with minor extra neurons.
problem Understanding the impact of over-parameterization on optimization landscapes of shallow neural networks.
method Analyzing a simple ReLU neural network with Gaussian inputs, focusing on optimization properties and landscape changes.
result Over-parameterization makes the objective function one-point strongly convex in most directions, aiding optimization.