Bagging stabilizes models without distributional assumptions.
problem Stability of machine learning models without distributional assumptions.
method Derives a finite-sample guarantee on bagging stability for any model.
result Guarantee applies to many bagging variants and is optimal.
Paper explores using bootstrap methods to improve SGD's stability and robustness.
problem Improving the stability and robustness of SGD.
method Investigates empirical bootstrap approaches for SGD from algorithmic stability and statistical robustness perspectives.
result Demonstrates construction of purely distribution-free confidence intervals using bootstrap SGD.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
Study shows minimizing the norm of the ERM solution stabilizes kernel ridge-less regression.
problem Stability of kernel ridge-less regression.
method Minimizing the norm of the ERM solution to minimize CV stability.
result Interpolating solution with minimum norm minimizes CV stability.
Enhanced stability improves privacy in machine learning.
problem Improving privacy in machine learning training while maintaining accuracy.
method Study of stability in private empirical risk minimization, focusing on strongly-convex loss functions and uniform stability.
result An algorithm with uniform stability of β implies a bound of O(√β) on the scale of noise required for differential privacy.
New stability framework relaxes boundedness assumptions for generalization bounds.
problem Overly restrictive assumptions for modern learning settings with heavy-tailed or unbounded losses.
method Develops a stability-based framework requiring only finite Lp moment conditions. result Sharp generalization bounds derived for various learning paradigms.
Improves Lasso's stability in correlated predictor settings.
problem Lasso's selection stability deteriorates with correlated predictors.
method Integrates a weighting scheme into the Lasso penalty function, using a correlation-adjusted ranking.
result Demonstrates improved selection stability on simulated and real-world datasets.
New framework improves worst-case generalization bounds for stochastic optimization.
problem Challenges in providing generalization guarantees for stochastic optimization algorithms.
method Introduces random set stability and empirically relevant complexity measures to avoid intractable mutual information terms.
result Bounded worst-case generalization error in terms of random set stability and empirically relevant complexity measures.
New analysis improves understanding of bilevel optimization stability and generalization.
problem Understanding how well bilevel optimization algorithms generalize.
method Algorithmic stability arguments and generalization bounds for three bilevel minimax solvers.
result Precise trade-off between algorithmic stability, generalization gaps, and practical settings.
Proposes a stability evaluation criterion for learning models using distributional perturbations.
problem Ensuring reliable deployment of learning models in out-of-sample environments.
method Uses optimal transport discrepancy with moment constraints to quantify minimal perturbation required for model deterioration.
result Validates the practical utility of the stability evaluation criterion across various real-world applications.
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
We introduce a notion of algorithmic stability of learning algorithms---that we term \emph{argument stability}---that captures stability of the hypothesis output by the learning algorithm in the normed space of functions from which hypotheses are selected. The main result of the paper bounds the generalization error of…
The paper derives uniform stability-based coverage bounds for conformal prediction methods.
problem Establishing theoretical guarantees for conformal prediction methods.
method Uniform stability perspective applied to full-conformal, jackknife+, and CV+ prediction regions.
result Coverage bounds for finite-dimensional models derived using a concentration argument.
A fast method estimates stability of ensemble feature selectors.
problem Improving stability of ensemble feature selectors for better prediction.
method Simulator of a feature selector to estimate stability.
result Reduces computation time for estimating stability.
This paper analyzes quantiles of heavy-tailed distributions, separating projection direction and quantile threshold effects.
problem Analyzing quantiles of heavy-tailed distributions with estimated parameters.
method Introduces a Q-Q orthogonality formulation to separate projection-direction and quantile-threshold effects.
result Decomposes the difference between empirical and population quantiles into three terms.
Detect changes in noisy dynamical systems using empirical approximations and finite-sample bounds.
problem Change detection in noisy dynamical systems
method Partition-based empirical approximations and finite-state stationary distribution stability
result Finite-sample bound for empirical stationary density
To understand the empirical success of approximate MAP inference, recent work (Lang et al., 2018) has shown that some popular approximation algorithms perform very well when the input instance is stable. The simplest stability condition assumes that the MAP solution does not change at all when some of the pairwise pote…
The paper improves the empirical bootstrap method for non-normal estimators.
problem Theoretical properties of empirical bootstrap for non-asymptotically normal estimators.
method Establishing limiting distribution, deriving consistency conditions, proposing alternative methods.
result The empirical bootstrap method can be asymptotically consistent under stability conditions.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
This paper investigates a type of instability that is linked to the greedy policy improvement in approximated reinforcement learning. We show empirically that non-deterministic policy improvement can stabilize methods like LSPI by controlling the improvements' stochasticity. Additionally we show that a suitable represe…
The paper examines stability of ReLU networks in tangent space and activation regions.
problem Stability and sensitivity of ReLU networks to small changes.
method Tangent sensitivity measure for ReLU networks, focusing on stability induced by individual examples.
result Tangent sensitivity correlates with the distribution of activation regions and generalization gap.
Many of our core assumptions about how neural networks operate remain empirically untested. One common assumption is that convolutional neural networks need to be stable to small translations and deformations to solve image recognition tasks. For many years, this stability was baked into CNN architectures by incorporat…
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
The paper presents a model-free method for stabilizing unknown control systems.
problem Stabilizing unknown control systems in engineering.
method Solving discounted LQR problems with increasing discount factors.
result The method efficiently recovers a stabilizing controller for linear and smooth nonlinear systems.
New model predicts stock performance in large equity markets.
problem Predicting stock performance in large equity markets over long time horizons.
method Rank-based volatility stabilized models calibrated to empirical data.
result The model exhibits relative arbitrage and statistically fits empirical features.
VL finds flatter solutions at edge of stability, matching theory with practice.
problem Understanding implicit regularization in deep learning.
method Edge of Stability framework, controlling variational posterior shape and sample number.
result VL finds even flatter solutions than gradient descent.
Paper improves zero-shot protein stability prediction by clarifying free-energy foundations.
problem Improving zero-shot protein stability prediction using inverse folding models.
method Clarifying the free-energy foundations of inverse folding models and proposing better estimates of relative stability.
result Significant gains in zero-shot performance can be achieved with simple methods.
Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…
We explore in some detail the notion of algorithmic stability as a viable framework for analyzing the generalization error of learning algorithms. We introduce the new notion of training stability of a learning algorithm and show that, in a general setting, it is sufficient for good bounds on generalization error. In t…
In this paper, we introduce a new concept of stability for cross-validation, called the (β,ϖ)-stability, and use it as a new perspective to build the general theory for cross-validation. The (β,ϖ)-stability mathematically connects the generalization ability and the stability of…
Gradient descent with large momentum finds flatter minima.
problem Understanding the effects of momentum in gradient descent.
method Empirical and theoretical analysis of gradient descent with large momentum.
result Large momentum leads to flatter minima than gradient descent.
Algorithmic stablecoins optimize monetary policy to balance price stability.
problem Persistent inflation from centralized monetary policy.
method Propose and study a rule-based monetary policy model for algorithmic stablecoins.
result Optimal trade-off between price stability and supply stability.
We consider a system of diffusion processes that interact through their empirical mean and have a stabilizing force acting on each of them, corresponding to a bistable potential. There are three parameters that characterize the system: the strength of the intrinsic stabilization, the strength of the external random per…
New bounds link SGD's generalization to heavy tails without topological assumptions.
problem Linking SGD's generalization error to heavy tails without additional assumptions.
method Developed Wasserstein stability bounds for heavy-tailed SDEs and their discretizations, converting to generalization bounds.
result Generalization bounds for a broader class of objective functions, including non-convex functions, without topological assumptions.
Generative adversarial networks (GANs) are effective in generating realistic images but the training is often unstable. There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods. To this end, we present …
Boosting framework for vector-valued prediction with geometric stability.
problem Lack of a general theoretical understanding of aggregation for structured prediction.
method Identifies (α,β)-stability property and proposes a boosting framework based on exponential reweighting and geometric-median aggregation. result Obtains exponential decay of empirical divergence error under weak learner condition and (α,β)-stability. SAM improves generalization by operating near the edge of stability.
problem Improving generalization in neural networks.
method Sharpness-Aware Minimization (SAM) approach to training neural networks.
result SAM operates near the 'edge of stability' identified by the analysis.
Develops a minimax optimal estimator for system stability under distribution shift.
problem Ensuring system reliability under changes in the underlying environment.
method Minimax optimal estimation of stability defined in terms of acceptable performance degradation.
result Characterizes the minimax convergence rate and demonstrates practical utility.
Paper shows ERM's suboptimality due to bias, not variance.
problem Understanding why ERM fails to achieve optimal rates.
method Probabilistic and admissibility proofs for ERM in various settings.
result ERM's suboptimality is due to bias, not variance.
New stability conditions for ZO methods reveal unique regularization effects.
problem Understanding optimization dynamics of ZO methods in deep learning.
method Explicit step size conditions and stability bounds derived for ZO methods.
result ZO methods operate near the edge of stability, with regularization effects specific to Hessian trace vs. eigenvalue.
Stabilized neural differential equations enforce constraints on dynamical systems.
problem Ensuring dynamical systems preserve known constraints like conservation laws.
method SNDEs with a stabilization term to enforce manifold constraints.
result SNDEs outperform existing methods and broaden constraint types.
We analyze why some models resist unlearning using linear stability theory.
problem Understanding and predicting when machine learning models resist unlearning.
method Linear stability theory applied to machine learning models, focusing on data coherence and optimization dynamics.
result Data coherence and signal-to-noise ratio (SNR) influence unlearning resistance; lower SNR makes unlearning easier.
Gradient descent on neural nets often operates at the Edge of Stability, where loss behavior is complex but loss decreases over time.
problem Understanding the optimization dynamics of neural networks at the Edge of Stability.
method Empirical demonstration of gradient descent behavior in neural network training.
result Gradient descent on neural networks typically occurs at the Edge of Stability, where loss behavior is non-monotonic but loss decreases over time.
Copula-based normalizing flows improve flexibility and stability for heavy-tailed data.
problem Limited expressive power of vanilla normalizing flows.
method Generalize base distribution to copula for more accurate representation of target distribution.
result Copula-based normalizing flows improve flexibility, stability, and effectiveness for heavy-tailed data.
In financial markets, greater volatility is usually considered synonym of greater risk and instability. However, large market downturns and upturns are often preceded by long periods where price returns exhibit only small fluctuations. To investigate this surprising feature, here we propose using the mean first hitting…
JFR-rg model explains Japan's stable debt despite high interest rates and low growth.
problem Understanding Japan's stable government debt despite high interest rates and low growth.
method Formalizes financial repression channels through JFR-rg model, incorporating financial repression bias and exchange-rate channel.
result Identifies Normalization Trap and Captive Financial System Parameter, showing debt dynamics under financial repression.
Fair classification has been a topic of intense study in machine learning, and several algorithms have been proposed towards this important task. However, in a recent study, Friedler et al. observed that fair classification algorithms may not be stable with respect to variations in the training dataset -- a crucial con…
New models learn stable latent clusters without side info.
problem Stability of non-linear ICA representations without side information.
method Deep generative models with latent clusterings, compared to standard VAEs and auxiliary labeled models.
result Deep generative models with latent clusterings are as stable as models with side information.