Study on generalization for data-dependent hypothesis sets.
problem Understanding generalization in hypothesis sets dependent on data.
method Learning guarantee based on transductive Rademacher complexity and hypothesis set stability.
result Generalization bound for data-dependent hypothesis sets.
The paper analyzes how stacking improves model stability.
problem Lack of theoretical insight into how stacking works.
method Stability analysis of learning algorithms, focusing on hypothesis stability.
result The hypothesis stability of stacking is a product of base models and combiner.
New bounds improve generalization in learning scenarios.
problem Limitations of existing information-theoretic bounds in SCO problems.
method Sample-conditioned hypothesis stability and neighboring-hypothesis matrix.
result Sharper generalization guarantees in various learning scenarios.
We introduce a notion of algorithmic stability of learning algorithms---that we term \emph{argument stability}---that captures stability of the hypothesis output by the learning algorithm in the normed space of functions from which hypotheses are selected. The main result of the paper bounds the generalization error of…
This paper analyzes HTL using stability theory for binary classification.
problem Analyzing HTL's theoretical behavior in binary classification tasks.
method Stability analysis of regularized empirical risk minimizers.
result Derives generalization bounds for training error, excess risk, and cross-validation.
Paper derives exponential bounds for learning risk using stable hypothesis.
problem Tackles the gap between optimal and suboptimal generalization bounds.
method Uses recent advances in concentration inequalities and a weaker stability notion.
result Derives an exponential tail bound for the concentration of the estimated risk.
This paper has been withdrawn due to a missing hypothesis in the main statement.
Paper proves Chow stability implies balanced embedding.
problem Chow stability and balanced embeddings in projective varieties.
method Continuity method conditional on technical hypothesis.
result Chow stability implies balanced embedding.
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …
The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.
problem The tension between uniform stability and L2-stability in generalization bounds. method Establishes in-expectation first moment generalization error bounds for L2-stable randomized learning algorithms and uses subbagging to achieve near-tight exponential bounds. result Improves generalization bounds for convex and non-convex optimization problems with SGD.
S-LIME stabilizes LIME for more reliable model explanations.
problem Instability of post hoc explanation methods like LIME.
method Uses hypothesis testing based on central limit theorem to stabilize explanations.
result Demonstrates effectiveness of S-LIME on simulated and real-world data.
This paper analyzes stability of decision trees and logistic regression.
problem Stability of decision trees and logistic regression is analyzed to understand their performance and sensitivity.
method Two stability notions (hypothesis and pointwise hypothesis stability) are derived for decision trees and logistic regression. The stability of decision trees depends on the number of leaves, while for logistic regression, it depends on the smallest eigenvalue of the Hessian matrix. Upper bounds on generalization error are constructed.
result Logistic regression is not a stable learning algorithm.
Study on CLT for Riemannian manifolds, focusing on cut locus stability.
problem Analyzing the Central Limit Theorem for Riemannian manifolds.
method Assessing stability of cut locus and applying it to clarify hypotheses in CLT for Fréchet means.
result Obtained a Central Limit Theorem for closed Riemannian manifolds, clarifying hypotheses.
Optimal private tests for simple hypotheses are characterized.
problem Private testing of simple hypotheses under differential privacy constraints.
method Characterization of sample complexity and optimal tests using log-likelihood ratio tests.
result Optimal sample complexity achieved by a specific randomized and clamped variant of the log-likelihood ratio test.
In this short note we extend an estimate due to J. Simons on the first stability eigenvalue of minimal hypersurfaces in spheres to the singular setting. Specifically, we show that any singular minimal hypersurface in Sn+1, which is not totally geodesic and satisfies the α-structural hypothesis, has first stability…
The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of Lq stability, and the strong use of moment inequalities. Lq stability e…
Randomized hierarchical clustering tests for stability and detects clusters.
problem Greedy hierarchical clustering's sensitivity to data perturbations.
method Randomization scheme and p-values at each node.
result Valid hypothesis testing procedures for clustering results.
Gradient descent with large momentum finds flatter minima.
problem Understanding the effects of momentum in gradient descent.
method Empirical and theoretical analysis of gradient descent with large momentum.
result Large momentum leads to flatter minima than gradient descent.
The paper proves a stability conjecture for manifolds with zero Euler characteristic.
problem Stability of manifolds with zero Euler characteristic under certain curvature conditions.
method Analyzes manifolds with dimensions 5 or more, proving stability under specific curvature and completeness conditions.
result 2006 Rosenberg's S1-stability holds for manifolds with zero Euler characteristic. New methods ensure feature importance rankings are correct with high probability.
problem Stability issues in feature importance scores due to random sampling.
method Hypothesis testing-based techniques to assess and verify the stability of top-ranked features.
result Ensures the most important features are correct with high-probability guarantees.
New method finds smaller, stable subnetworks that can train to full network accuracy.
problem Pruning neural networks at initialization to find subnetworks that can train to similar accuracy.
method Modified IMP to search for subnetworks that could have been obtained by pruning early in training, focusing on 0.1% to 7% through.
result Pruned subnetworks of deeper networks can complete training to match the accuracy of the original network on challenging tasks.
We contrast Arbitrage Pricing Theory (APT), the theoretical basis for the development of financial instruments, with a dynamical picture of an interacting market, in a simple setting. The proliferation of financial instruments apparently provides more means for risk diversification, making the market more efficient and…
Statistical analysis of financial data most focused on testing the validity of Brownian motion (Bm). Analysis performed on several time series have shown deviation from the Bm hypothesis, that is at the base of the evaluation of many financial derivatives. We inquiry in the behavior of measures of performance based on …
A new method uses maximum entropy for time series analysis.
problem Challenges in testing statistical properties of multivariate time series.
method Statistical mechanical approach for ensembles of time series.
result Shows possible applications in financial portfolio selection.
New examples challenge Geroch conjecture stability.
problem Stability of the Geroch conjecture in warped products.
method Constructing warped-product manifolds with specific curvature properties.
result First counterexample to Sormani's conjecture on stability.
Survey on new data-dependent bounds for neural networks.
problem Generalization of overparameterized neural networks.
method Extending PAC-Bayesian theory, refining complexity terms, and replacing information-theoretic terms with stability assumptions.
result Unified template inequality and comparison of bounds.
We reveal a model rank that predicts successful recovery of target functions at overparameterization.
problem Understanding the mysterious good generalization performance of overparameterized nonlinear models.
method Rank stratification and linear stability theory for general nonlinear models.
result Linearly stable functions are preferred by nonlinear training, and model rank predicts minimal training data size.
Given a hypothesis space, the large volume principle by Vladimir Vapnik prioritizes equivalence classes according to their volume in the hypothesis space. The volume approximation has hitherto been successfully applied to binary learning problems. In this paper, we extend it naturally to a more general definition which…
We investigate the continuity of expected exponential utility maximization with respect to perturbation of the Sharpe ratio of markets. By focusing only on continuity, we impose weaker regularity conditions than those found in the literature. Specifically, we require, in addition to the V-compactness hypothesis of La…
Study how neural networks optimize to stable linearly connected regions.
problem Understanding how neural networks converge to stable solutions under different training conditions.
method Investigate the stability of neural networks to SGD noise and apply it to iterative magnitude pruning.
result Subnetworks that reach full accuracy must be stable to SGD noise, either at initialization or early in training.
In this note we identify the leading terms of the (reduced) K-energy map with a universal linear combination of the principal and subdominant coefficients of the weight of the mth Hilbert point. This shows that the weight F1(λ;X) introduced by Donaldson in [SKD02] is just the weight of the CM-polarisation.The eq…
Mitigates instability in reinforcement learning for safer robotics.
problem Unstable training dynamics in reinforcement learning, especially for safety-sensitive tasks.
method Maintains a history of the agent and reverts to previous parameters when performance decreases.
result Improves performance and stability compared to state-of-the-art algorithms.
We consider a priori generalization bounds developed in terms of cross-validation estimates and the stability of learners. In particular, we first derive an exponential Efron-Stein type tail inequality for the concentration of a general function of n independent random variables. Next, under some reasonable notion of s…
Quantum model improves safety in machine learning.
problem Improving safety and robustness in machine learning models.
method Variational quantum classifier with amplitude encoding and SAFE-AI metrics.
result Quantum model provides competitive performance and improved robustness.
On a 4-dimensional compact symplectic manifold, we consider a smooth family of compatible almost-complex structures such that at time zero the induced metric is Hermite-Einstein almost-Kähler metric with zero or negative Hermitian scalar curvature. We prove, under certain hypothesis, the existence of a smooth family of…
New bounds explain modern machine learning algorithms' generalization.
problem Explaining generalization behavior of modern machine learning algorithms.
method Proposes a new complexity measure based on empirical Rademacher complexity of an algorithm- and data-dependent hypothesis class.
result Obtains novel bounds with finite fractal dimension, simplifies proofs, and recovers known results.
Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…
Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
Peters (2011a) defined an optimal leverage which maximizes the time-average growth rate of an investment held at constant leverage. It was hypothesized that this optimal leverage is attracted to 1, such that, e.g., leveraging an investment in the market portfolio cannot yield long-term outperformance. This places a str…
A new method to stabilize training by reducing the variance of adaptive learning rates.
problem Large variance of adaptive learning rates in the early stage of training.
method Introducing a warmup phase and a variance rectification term in RAdam.
result RAdam improves convergence and generalization in various tasks.
Transformers learn algorithms for in-context learning with bounds and stability analysis.
problem Understanding and formalizing in-context learning as an algorithm learning problem.
method Formalizing ICL as a multitask learning problem, deriving generalization bounds, and analyzing stability.
result Transformers can implement near-optimal algorithms for classical regression tasks with i.i.d. and dynamic data.
Study stability of selective SSMs with discontinuous gating.
problem Challenges in stability analysis of selective SSMs with discontinuous gating.
method Passivity and Input-to-State Stability (ISS) analysis of continuous-time selective SSMs.
result Derivation of sufficient conditions for global ISS with respect to the port input.
A new method uses vectorized summaries of persistence diagrams for efficient hypothesis testing.
problem Efficient hypothesis testing for large and complex persistence diagrams.
method Vectorized summaries of Betti functions and a new shuffling technique.
result The vectorized Betti function leads to competitive results compared to baseline methods.
Given a path of almost-Kähler metrics compatible with a fixed symplectic form on a compact 4-manifold such that at time zero the almost-Kähler metric is an extremal Kähler one, we prove, for a short time and under a certain hypothesis, the existence of a smooth family of extremal almost-Kähler metrics compatible with t…
Sleep-based regularization stabilizes STDP in recurrent neural networks.
problem Pathological weight dynamics in recurrent SNNs.
method Periodic offline phases with stochastic decay and spontaneous activity.
result Sleep-based renormalization prevents weight saturation and preserves learned structure.
Cryptocurrencies show stable prices as a medium of exchange.
problem Price stability of cryptocurrencies as a medium of exchange.
method Filtered daily returns of major cryptocurrencies compared to major financial assets using Pearson correlations, dynamic time-warping method, and Black-Scholes model.
result Cryptocurrencies exhibit stable daily returns relative to major financial assets over the years 2016-2020.
The paper examines how to test if two learning algorithms produce similar outcomes.
problem Testing if two learning algorithms produce similar outcomes when trained on different data sets.
method Using Total Variation (TV) distance to measure similarity of posterior distributions.
result TV indistinguishable learning rules are equivalent to existing stability notions and can be statistically amplified.
StatLoRA uses statistical inference to allocate ranks in LoRA fine-tuning, improving performance.
problem Balancing efficiency, expressiveness, and generalization in LoRA rank allocation.
method Formulates LoRA rank allocation as a statistical hypothesis testing problem, using estimated p-values to determine component retention or pruning.
result StatLoRA achieves comparable or better performance than existing methods under matched rank budgets.