The paper proves Zimmer's conjecture for non-uniform lattices by controlling mass escape and Lyapunov exponents.
problem Proving Zimmer's conjecture for non-uniform lattices in higher-rank semisimple Lie groups.
method Establishes finiteness of low-dimensional actions, introduces novel techniques to control mass escape and Lyapunov exponents.
result Proves Zimmer's conjecture for many non-uniform lattices, improving previous results.
In this paper we study the ergodic theory of the geodesic flow on negatively curved geometrically finite manifolds. We prove that the measure theoretic entropy is upper semicontinuous when there is no loss of mass. In case we are losing mass, the critical exponents of parabolic subgroups of the fundamental group have a…
Open manifolds with nonnegative Ricci curvature have virtually abelian fundamental groups if they escape from bounded balls at a small rate.
problem Understanding the fundamental groups of open manifolds with nonnegative Ricci curvature.
method Analyzing the escape rate of minimal geodesic loops and relating it to the fundamental group's properties.
result If an open manifold has a small escape rate, its fundamental group is virtually abelian.
Bayesian analysis predicts properties of proton-emitting nuclei beyond the proton drip line.
problem Predicting properties of unstable nuclei in the proton-rich region.
method Bayesian Gaussian processes and mass models corrected with statistical emulators.
result Quantified predictions for separation energies and probabilities of proton emission.
SGD's escape rate depends on log loss barrier, not linear loss barrier.
problem Understanding the escape rate of SGD from local minima.
method Derived a stochastic differential equation (SDE) with additive noise from SGD's multiplicative noise property.
result The log loss barrier determines the escape rate of SGD, not the linear loss barrier.
Classifies conformal transformations in spacetimes without observer horizons.
problem Understanding conformal transformations in spacetimes without observer horizons.
method Proves classification of conformal transformations into two types: escaping and non-escaping.
result Conformal transformations of Einstein's static universe are classified.
The paper analyzes neural network dynamics after weights escape the origin.
problem Understanding gradient flow dynamics of neural networks after the origin.
method Analyzes gradient flow of homogeneous neural networks with locally Lipschitz gradients.
result Characterizes the first saddle point encountered after escaping the origin.
We shortly review the statistical properties of the escape times, or hitting times, for stock price returns by using different models which describe the stock market evolution. We compare the probability function (PF) of these escape times with that obtained from real market data. Afterwards we analyze in detail the ef…
Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.
problem Finding safe zones in policy Markov Decision Processes to limit trajectory escape.
method Bi-criteria approximation learning algorithm with polynomial sample complexity.
result Achieves almost 2 approximation for both escape probability and safe zone size.
Gradient descent can take exponentially long to escape saddle points in 2D.
problem Worst-case inefficiency of gradient descent in non-convex optimization.
method Analysis of gradient descent's performance on 2D functions.
result Gradient descent can take exponentially long to escape saddle points.
New algorithm helps escape saddle points in optimization problems.
problem Optimizing smooth non-convex functions to avoid saddle points.
method Perturbed Saddle-escape Descent (PSD) algorithm with explicit constants.
result PSD finds approximate second-order stationary points efficiently.
New methods help escape strict saddle points in nonsmooth optimization.
problem Escaping strict saddle points in nonsmooth optimization.
method An inexact stochastically perturbed gradient method applied to the Moreau envelope.
result A variety of algorithms for nonsmooth optimization can efficiently escape strict saddle points of the Moreau envelope.
Deep ReLU networks escape from the origin via saddle points with a low-rank bias.
problem Understanding the dynamics of gradient descent in deep ReLU networks.
method Analysis of escape directions and singular values of weight matrices.
result The first singular value of the ℓ-th layer weight matrix is at least ℓ41 larger than any other singular value. Many nonparametric regressors were recently shown to converge at rates that depend only on the intrinsic dimension of data. These regressors thus escape the curse of dimension when high-dimensional data has low intrinsic dimension (e.g. a manifold). We show that k-NN regression is also adaptive to intrinsic dimension. …
Geodesic loops escape from balls at a sublinear rate imply virtually abelian fundamental group.
problem Understanding fundamental groups of open manifolds with nonnegative Ricci curvature.
method Generalizing the Cheeger-Gromoll splitting theorem to sublinear escape rates.
result Fundamental groups of open manifolds with nonnegative Ricci curvature are virtually abelian if geodesic loops escape sublinearly.
Houdini finds high-dimensional saddle points under few constraints.
problem Escaping from saddle points in high-dimensional spaces with constraints.
method Gradient descent methods under logarithmic inequality constraints.
result Polynomial time algorithms for escaping saddle points under constraints.
The Dirichlet random walk on manifolds has a positive escape rate if the cover is non-amenable.
problem Analyzing the stochastic behavior of Dirichlet random walks on manifolds.
method Defining a recursive process on Galoisian covers and proving a theorem about the escape rate.
result The escape rate is positive if and only if the cover is non-amenable.
HA-SME models SGD dynamics with Hessian info for better escaping behaviors.
problem Capturing the escaping behaviors of SGD from stationary points.
method HA-SME, a novel SDE with Hessian info in drift and diffusion.
result HA-SME achieves best approximation error and recovers SGD dynamics for quadratics.
Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this …
New result on group actions in CAT(0) spaces with vanishing escape rate.
problem Understanding group actions with vanishing escape rate on CAT(0) spaces.
method Equivariant μ-harmonic map proof. result Existence of a flat subspace invariant under the action of Γ. We solve the escape problem for the Heston random diffusion model. We obtain exact expressions for the survival probability (which ammounts to solving the complete escape problem) as well as for the mean exit time. We also average the volatility in order to work out the problem for the return alone regardless volatilit…
A new depth measure and median defined on Hadamard manifolds.
problem Statistical depth and median on Hadamard manifolds.
method Horospherical depth and Busemann median defined using renormalized distance functions.
result The Busemann median exists for every Borel probability measure on Hadamard manifolds.
Paper studies identifiability and stability of drifting fields in generative modeling.
problem Identify and stabilize drifting fields in generative modeling.
method Introduces companion-elliptic kernel families to address limitations of Laplace kernel.
result Establishes field identifiability and demonstrates scalar observables for weak convergence.
We study the mean escape time in a market model with stochastic volatility. The process followed by the volatility is the Cox Ingersoll and Ross process which is widely used to model stock price fluctuations. The market model can be considered as a generalization of the Heston model, where the geometric Brownian motion…
This paper proposes a new global optimization algorithm using deep learning.
problem Developing efficient algorithms for global optimization of non-convex functions.
method Two-phase approach: minimization phase with model-driven deep learning, escaping phase with reinforcement learning.
result The proposed algorithm significantly outperforms classical optimization methods and handles ill-posed functions.
The paper explores identifiability and stability in drifting fields using companion-elliptic kernels.
problem Identifying and stabilizing drifting fields in generative modeling.
method Introduces companion-elliptic kernel families and analyzes their properties to address identifiability and stability issues.
result Established field identifiability for arbitrary Borel probability measures and demonstrated that field convergence alone does not guarantee weak convergence.
Hill-ADAM optimizes loss landscapes by exploring state space deterministically.
problem Escaping local minima in loss landscapes.
method Hill-ADAM alternates between minimizing and maximizing error to explore the loss space.
result Hill-ADAM finds the global minimum state in loss landscapes.
Nonconvex optimization algorithms with random initialization have attracted increasing attention recently. It has been showed that many first-order methods always avoid saddle points with random starting points. In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoi…
Study of SGD with state-dependent noise, improving escape from local minima.
problem Understanding and improving the dynamics of SGD in non-convex optimization.
method Formal study on SGD with state-dependent noise, proposing power-law dynamic with state-dependent diffusion.
result Power-law dynamic can escape from sharp minima exponentially faster than flat minima.
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape. On the other hand, g…
A new method helps escape saddle points in non-convex optimization.
problem Escaping saddle points in non-convex optimization problems.
method CNC-SCSG method using a separate SGD step to help escape from strict saddle points.
result The method converges to a second-order stationary point with a rate of O(ε−2log(1/ε)). The paper analyzes how noise geometry influences the performance of SGD in machine learning.
problem Understanding how noise geometry affects the performance of stochastic gradient descent.
method Developed two metrics to quantify noise alignment strength and analyzed their effects on loss and subspace projection dynamics.
result Noise geometry can be used to guarantee alignment under certain conditions, aiding SGD's ability to escape from sharp minima.
The roundworm C. elegans exhibits robust escape behavior in response to rapidly rising temperature. The behavior lasts for a few seconds, shows history dependence, involves both sensory and motor systems, and is too complicated to model mechanistically using currently available knowledge. Instead we model the process p…
New metrics help predict Brownian motion on surfaces and higher dimensions.
problem Predicting Brownian motion on complex surfaces and higher dimensions.
method Developed new metrics (Uniform Drainage Metric) for surfaces and higher dimensions.
result Uniform Drainage Metric predicts Brownian motion's narrow escape time consistently.
Adaptive methods such as Adam and RMSProp are widely used in deep learning but are not well understood. In this paper, we seek a crisp, clean and precise characterization of their behavior in nonconvex settings. To this end, we first provide a novel view of adaptive methods as preconditioned SGD, where the precondition…
A statistical analysis of financial, economic, and demographic indicators performed by the authors demonstrates (1) that the main countries of East Africa (Uganda, Kenya, and Tanzania) have not escaped the Malthusian Trap yet; (2) that this countries are not likely to follow the "North African path" and to achieve this…
Researchers extend microlocal analysis across event horizons of rotating black holes.
problem Incomplete microlocal theory of fields across black hole event horizons.
method Extended microlocal theory for extremal rotating black holes, showing null covectors form an involutive double characteristic manifold.
result Mathematical basis for asymptotic oscillatory solutions near event horizons.
Two-layer networks learn hard GLMs with SGD in high dimensions.
problem Learning hard generalized linear models with SGD in high-dimensional settings.
method Reduction of SGD dynamics to a stochastic process in lower dimensions, focusing on the role of stochasticity.
result Overparameterization enhances convergence by a constant factor, suggesting minimal role of stochasticity.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
New k-step policy gradient method avoids local optima in restricted policy classes.
problem Suboptimal local optima in policy gradient methods for restricted policy classes.
method Proposes a k-step policy gradient method to escape myopic local optima. result The method converges to near optimal solutions exponentially close to the optimal deterministic policy.
The expectation-maximization (EM) algorithm has been widely used in minimizing the negative log likelihood (also known as cross entropy) of mixture models. However, little is understood about the goodness of the fixed points it converges to. In this paper, we study the regions where one component is missing in two-comp…
SGD transitions between maxima and minima with varying time scales.
problem Understanding SGD's behavior near critical points in noisy landscapes.
method Analyzing SGD convergence and escape dynamics in 1D landscapes with infinite- and finite-variance noise.
result SGD reliably moves to the basin's minimum unless close to a local maximum, where it can linger.
PWGF escapes saddle points in nonconvex optimization.
problem Escaping saddle points in nonconvex optimization.
method PWGF uses noisy perturbations via Gaussian process to escape saddle points.
result PWGF achieves second-order optimality for nonconvex objectives.
Diagonal Frog: High-order positivity-preserving FD schemes for anisotropic Fokker-Planck equations
problem Positivity-preserving discretizations for anisotropic Fokker-Planck equations
method Diagonal Frog discretization
result Second-order accuracy and mass conservation
New PINNs method improves accuracy in computing Mean Escape Time from bounded domains.
problem Computing Mean Escape Time from bounded domains with high accuracy.
method Boundary-adapted Physics-Informed Neural Networks (PINNs) with exact Dirichlet boundary enforcement.
result Derivation of H2(Ω) a priori error bounds for PINNs with normalized distance approximations. Theory explains deep nonlinear networks' plateaus and transitions.
problem Understanding long plateaus and feature acquisition transitions in deep nonlinear networks.
method Derived an exact identity for Frobenius norms, classified activation functions, and reduced matrix flow to a scalar ODE.
result Escape time law τ⋆=Θ(ε−(r−2)) for deep nonlinear networks, where r is the number of bottleneck layers. Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima…
We study spectral theory for the Schrodinger operator on manifolds possessing an escape function. A particular class of examples are manifolds with Euclidean and/or hyperbolic ends.