SGD works well with large learning rates at the edge of stability.
problem Stochasticity at the edge of stability in deep learning.
method Sharp convergence guarantees for SGD with multiclass cross-entropy loss.
result SGD self-stabilizes, ensuring convergence with large learning rates.
The paper analyzes stability and convergence rates of entropic and Sinkhorn potentials.
problem Stability and convergence rates of entropic and Sinkhorn potentials.
method Semiconcavity properties of entropic potentials and Schrödinger bridges.
result Exponential convergence rates for gradient and Hessian of Sinkhorn iterates.
Study identifies and validates a method for system identification of Markov jump linear systems.
problem System identification for autonomous Markov jump linear systems with complete state observations.
method Proposes switched least squares method for identification and derives rates of convergence.
result Data-independent rate of convergence is O ( log ( T ) / T ) \mathcal{O}\big(\sqrt{\log(T)/T} \big) O ( log ( T ) / T ) , showing strong consistency. Decentralized learning ensures stability in online queuing systems with packet rates above 1.
problem Ensuring stability in online queuing systems with decentralized learning.
method Proposed cooperative queues and a learning algorithm for packet rates above 1.
result Decentralized learning strategies guarantee stability in queuing systems with packet rates above 1.
New algorithms robustly estimate mean with near-optimal error rates.
problem Outlier robust mean estimation in high-dimensional data.
method Stability condition and iterative filtering algorithms.
result Optimal error rates with subgaussian rates for robust mean estimation.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
The paper explores how dynamic preconditioning affects the CLT in online averaging.
problem When does dynamic preconditioning preserve the Polyak-Ruppert CLT?
method The authors decompose the averaged error and identify a stabilization-rate threshold for the CLT to hold.
result The CLT holds if the dynamic remainder vanishes in L 2 L^2 L 2 and the stabilization rate exceeds a threshold. OMD and DA perform similarly in static settings but OMD is inferior under dynamic learning rates.
problem Proving and understanding the performance difference between OMD and DA under dynamic learning rates.
method Introducing stabilization to OMD and modifying its convergence analysis.
result OMD with stabilization and DA have the same performance guarantees under dynamic learning rates.
Studied how SGD's stability regularization affects generalization in neural networks.
problem Understanding why SGD often generalizes better than GD in neural networks.
method Analyzed stability of SGD and GD through Frobenius norm and trace of Hessian, and compared their generalization properties.
result Stable minima of SGD generalize well, while GD's stability-induced regularization is too weak.
Study uses VIX for zero-coupon Treasury rates, proving long-term stability and returns.
problem Modeling zero-coupon Treasury rates with VIX for volatility.
method Multivariate autoregressive stochastic volatility model, proving stability and Law of Large Numbers.
result VIX accurately models zero-coupon Treasury rates and returns.
Develops a minimax optimal estimator for system stability under distribution shift.
problem Ensuring system reliability under changes in the underlying environment.
method Minimax optimal estimation of stability defined in terms of acceptable performance degradation.
result Characterizes the minimax convergence rate and demonstrates practical utility.
Study examines USD exchange rate dynamics using Kramers-Moyal expansion.
problem Understanding and predicting exchange rate instability.
method Kramers-Moyal expansion and Fokker-Planck formalism applied to log-return data.
result Identifies a stabilizing linear drift and nonlinear diffusion term in exchange rate fluctuations.
AKO improves stability and power of Knockoff inference.
problem Instability of Knockoff-based inference.
method Aggregation of Multiple Knockoffs (AKO) method.
result AKO maintains FDR control while improving stability and power.
Study proves quantitative results for isoperimetric problem outside convex bodies in the plane.
problem Quantitative estimates for the relative isoperimetric problem outside convex bodies in the plane.
method Flow approach and Łojasiewicz estimates to prove quantitative stability for minimizers.
result Explicit constants and optimal exponents/rates for Łojasiewicz estimates and rates of convergence for gradient flow.
New bounds show faster convergence for learning algorithms.
problem Improving risk bounds for learning algorithms.
method Using algorithmic stability and common assumptions like Polyak-Lojasiewicz condition, smoothness, and Lipschitz continuity.
result Achieves convergence rate of O ( log 2 ( n ) / n 2 ) O(\log^2(n)/n^2) O ( log 2 ( n ) / n 2 ) with high probability. Paper analyzes neural network distances and stability, leading to a new learning rule.
problem Stability and efficiency in training deep neural networks.
method Relational trust distance and descent lemma for neural networks.
result New learning rule requires minimal learning rate tuning.
We argue that a negative interest rate policy (NIRP) can be an effect tool for macroeconomic stabilization. We first discuss how implementing negative rates on reserves held at a central bank does not pose any theoretical difficulty, with a reduction in rates operating in exactly the same way when rates are positive or…
The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.
problem The tension between uniform stability and L 2 L_2 L 2 -stability in generalization bounds. method Establishes in-expectation first moment generalization error bounds for L 2 L_2 L 2 -stable randomized learning algorithms and uses subbagging to achieve near-tight exponential bounds. result Improves generalization bounds for convex and non-convex optimization problems with SGD.
We prove stability of rotationally symmetric translating solutions to mean curvature flow. For initial data that converge spatially at infinity to such a soliton, we obtain convergence for large times to that soliton without imposing any decay rates.
The stability of statistical analysis is an important indicator for reproducibility, which is one main principle of scientific method. It entails that similar statistical conclusions can be reached based on independent samples from the same underlying population. In this paper, we introduce a general measure of classif…
New algorithm achieves small-loss bounds in online learning with improved rates.
problem Achieving strong stability in online learning algorithms.
method Introduces ρ ρ ρ -separation to enforce strong stability, unifying previous approaches. result Oracle-efficient algorithm achieves small-loss bounds with improved rates.
Stability inequalities for specific solutions in high dimensions.
problem Stability of solutions to the one-phase Bernoulli problem.
method Proving strict stability inequalities for cohomogeneity one solutions with bi-orthogonal symmetry.
result Strict stability for cohomogeneity one solutions in dimensions 7 and above.
Background: Recent developments have made it possible to accelerate neural networks training significantly using large batch sizes and data parallelism. Training in an asynchronous fashion, where delay occurs, can make training even more scalable. However, asynchronous training has its pitfalls, mainly a degradation in…
Emerging economies use countercyclical policies to manage crises and dominant currencies.
problem Managing economic crises and fluctuations in dominant currencies like USD and EUR.
method Theoretical analysis, case studies, econometric modeling.
result Emerging economies can stabilize growth with countercyclical monetary policies.
Large learning rates lead to various implicit biases in nonconvex optimization.
problem Understanding the conditions under which large learning rates yield edge of stability, balancing, and catapult phenomena.
method Developed a global convergence theory for nonconvex functions without globally Lipschitz continuous gradient, focusing on functions with good regularity.
result These implicit biases are more likely to occur in functions with good regularity, and large learning rates favor flatter regions.
This study analyzes AdaGrad's stability and convergence in non-convex optimization.
problem Lack of theoretical analysis for AdaGrad in non-convex optimization.
method Novel stopping time-based techniques from probability theory.
result Established stability and derived convergence rates for AdaGrad.
New stability theorem for hyperbolic metrics without volume bounds.
problem Stability of finite volume hyperbolic metrics without upper volume bounds.
method Abstract axiomatic framework and bootstrap argument to extend stability result.
result Weaker exponential control of the metric allows for a broader application of the stability theorem.
Paper proves stability of quermassintegral inequalities using inverse curvature flow.
problem Stability of quermassintegral inequalities for nearly spherical sets.
method Inverse curvature flow with special rescaling to study quermassintegral inequalities.
result Decreasing rate of k-th quermassintegral is faster than Fraenkel asymmetry for nearly spherical sets.
Study on reducing dimensionality in high-dimensional regression with kernel methods and stability analysis.
problem Analyzing errors in high-dimensional regression with dimensionality reduction and kernel regression.
method Derive a stability result for kernel regression with Wasserstein distance and apply it to PCA to deduce convergence rates.
result Two-step procedure yields useful convergence rates in semi-supervised settings.
Algorithm stabilizes queues in asymmetric systems with unknown service rates.
problem Stabilizing queues in multi-class multi-server systems with unknown service rates.
method Proposes UCB and Thompson Sampling algorithms to stabilize queues while learning service rates.
result Achieves system stability with an average queue length bound of \(O(\min\{N,K\}/ε)\) for large time horizon \(T\).
New ensemble method improves model stability exponentially.
problem Improving model stability for discontinuous base learners.
method Selecting the most frequently generated model from subsamples.
result Exponentially decaying tails for excess risk.
We develop and study stability properties of a hybrid approximation of functionals of the Bates jump model with stochastic interest rate that uses a tree method in the direction of the volatility and the interest rate and a finite-difference approach in order to handle the underlying asset price process. We also propos…
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable r r r . result VAV method achieves faster convergence and superior stability with larger learning rates.
Gradient descent forces neural network eigenvalues to a specific threshold.
problem Understanding why gradient descent drives eigenvalues to a specific threshold.
method Introduced edge coupling, a functional on consecutive iterate pairs, to explain the trajectory towards the eigenvalue threshold.
result Gradient descent forces the Hessian eigenvalue to the threshold 2 / η 2/η 2/ η from arbitrary initialization. Study finds relevance of exchange and inflation rates to economic factors.
problem Determining the relevance of exchange and inflation rates to economic factors.
method Introduced concept of adequacy, established positive relation between exchange and inflation rates and other economic factors.
result Close positive relation found between exchange and inflation rates and other economic factors.
New adaptive learning rate improves FTRL's adaptivity to sparsity, game-dependency, and best-of-both-worlds.
problem Improving adaptivity in sequential decision-making problems.
method Developed a stability-penalty-adaptive (SPA) learning rate for FTRL.
result First BOBW algorithm with sparsity-dependent bound.
New insights into network generalization show learning rate affects both norm and sharpness.
problem Understanding the generalization of overparameterized networks.
method Empirical analysis and theoretical proof of the trade-off between norm and sharpness.
result Learning rate influences both norm and sharpness, neither alone minimizes generalization error.
Study examines how training regime affects neural networks' forgetting.
problem Catastrophic forgetting in neural networks when learning multiple tasks sequentially.
method Analyzes the impact of different training regimes (learning rate, batch size, regularization) on forgetting.
result Training regimes that widen tasks' local minima help prevent catastrophic forgetting.
This paper extends stability and generalization analysis of GD for multi-layer NNs.
problem Understanding the generalization of multi-layer neural networks trained by GD.
method Comprehensive stability and generalization analysis of GD for multi-layer NNs, focusing on two-layer and three-layer networks.
result Derives excess risk rates of O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) for GD in two-layer and three-layer NNs under specific conditions. New model stabilizes asynchronous LTI systems, independent of synchronous stability.
problem Stability of asynchronous LTI systems under randomization and asynchrony.
method Introduced a new model for random asynchronous LTI systems and developed a method for system identification.
result Stability of random asynchronous LTI systems is independent of synchronous stability.
The paper analyzes rates of convergence for optimal transport map estimators using barycentric projections.
problem Estimating optimal transport maps from data sampled according to two distributions.
method Comprehensive analysis of rates of convergence for plug-in estimators defined via barycentric projections.
result New stability estimate for barycentric projections under minimal smoothness assumptions.
Stability proved for open Milne spacetime, showing gravity's long-term behavior.
problem Global stability of open Milne spacetime for Einstein-scalar field equations.
method Gaussian normal coordinates, exploiting expanding geometry of Milne spacetime.
result Spatial metric tends to hyperbolic metric as time goes to infinity.
AdamNX improves Adam's stability by adjusting its learning rate.
problem Adam's tendency to converge to non-flat minima in large-scale models.
method Proposes a novel exponential decay mechanism for Adam's second-order moment estimate.
result AdamNX outperforms Adam and its variants in stability and performance.
New method shows how order of gradient updates impacts stability and convergence in deep learning.
problem Training deep learning models can be unstable and computationally expensive.
method Theoretical analysis and experiments with backward-SGD.
result The order of gradient updates affects stability and convergence, leading to improved performance.
SGD handles label noise with bounds improving over SGLD.
problem Label noise in non-convex optimization.
method Stochastic gradient descent with uniform dissipativity and smoothness conditions, using Wasserstein distance and algorithmic stability.
result Generalization error bounds with a rate of n − 2 / 3 n^{-2/3} n − 2/3 , better than SGLD's n − 1 / 2 n^{-1/2} n − 1/2 . Collaborative filtering (CF) is a popular technique in today's recommender systems, and matrix approximation-based CF methods have achieved great success in both rating prediction and top-N recommendation tasks. However, real-world user-item rating matrices are typically sparse, incomplete and noisy, which introduce ch…
Future stability of FLRW solutions in expanding 3D space is shown for compact perturbations.
problem Future stability of expanding FLRW solutions with spatial topology R^3.
method Nonlinear stability analysis of spherically symmetric perturbations.
result Decay rates of energy momentum tensor components compared to Minkowski space.
We consider a system of diffusion processes that interact through their empirical mean and have a stabilizing force acting on each of them, corresponding to a bistable potential. There are three parameters that characterize the system: the strength of the intrinsic stabilization, the strength of the external random per…