arXiv research

New bounds improve generalization in learning scenarios.

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothe…

Stacking is a general approach for combining multiple models toward greater predictive accuracy. It has found various application across different domains, ensuing from its meta-learning nature. Our understanding, nevertheless, on how and why stacking works remains intuitive and lacking in theoretical insight. In this …

We introduce a notion of algorithmic stability of learning algorithms---that we term \emph{argument stability}---that captures stability of the hypothesis output by the learning algorithm in the normed space of functions from which hypotheses are selected. The main result of the paper bounds the generalization error of…

This paper analyzes HTL using stability theory for binary classification.

This paper has been withdrawn due to a missing hypothesis in the main statement.

Paper proves Chow stability implies balanced embedding.

PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …

The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.

S-LIME stabilizes LIME for more reliable model explanations.

Decision trees and logistic regression are one of the most popular and well-known machine learning algorithms, frequently used to solve a variety of real-world problems. Stability of learning algorithms is a powerful tool to analyze their performance and sensitivity and subsequently allow researchers to draw reliable c…

There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …

In this short note we extend an estimate due to J. Simons on the first stability eigenvalue of minimal hypersurfaces in spheres to the singular setting. Specifically, we show that any singular minimal hypersurface in $S^{n+1}$, which is not totally geodesic and satisfies the α-structural hypothesis, has first stability…

The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of $L^q$ stability, and the strong use of moment inequalities. $L^q$ stability e…

Randomized hierarchical clustering tests for stability and detects clusters.

Gradient descent with large momentum finds flatter minima.

The paper proves a stability conjecture for manifolds with zero Euler characteristic.

We obtain a Central Limit Theorem for closed Riemannian manifolds, clarifying along the way the geometric meaning of some of the hypotheses in Bhattacharya and Lin's Omnibus Central Limit Theorem for Fréchet means. We obtain our CLT assuming certain stability hypothesis for the cut locus, which always holds when the ma…

New methods ensure feature importance rankings are correct with high probability.

We contrast Arbitrage Pricing Theory (APT), the theoretical basis for the development of financial instruments, with a dynamical picture of an interacting market, in a simple setting. The proliferation of financial instruments apparently provides more means for risk diversification, making the market more efficient and…

Statistical analysis of financial data most focused on testing the validity of Brownian motion (Bm). Analysis performed on several time series have shown deviation from the Bm hypothesis, that is at the base of the evaluation of many financial derivatives. We inquiry in the behavior of measures of performance based on …

New examples challenge Geroch conjecture stability.

Natural and social multivariate systems are commonly studied through sets of simultaneous and time-spaced measurements of the observables that drive their dynamics, i.e., through sets of time series. Typically, this is done via hypothesis testing: the statistical properties of the empirical time series are tested again…

Survey on new data-dependent bounds for neural networks.

We reveal a model rank that predicts successful recovery of target functions at overparameterization.

Given a hypothesis space, the large volume principle by Vladimir Vapnik prioritizes equivalence classes according to their volume in the hypothesis space. The volume approximation has hitherto been successfully applied to binary learning problems. In this paper, we extend it naturally to a more general definition which…

We investigate the continuity of expected exponential utility maximization with respect to perturbation of the Sharpe ratio of markets. By focusing only on continuity, we impose weaker regularity conditions than those found in the literature. Specifically, we require, in addition to the $V$-compactness hypothesis of La…

In this note we identify the leading terms of the (reduced) K-energy map with a universal linear combination of the principal and subdominant coefficients of the weight of the $mth$ Hilbert point. This shows that the weight $F_{1}(λ;X)$ introduced by Donaldson in [SKD02] is just the weight of the CM-polarisation.The eq…

We consider a priori generalization bounds developed in terms of cross-validation estimates and the stability of learners. In particular, we first derive an exponential Efron-Stein type tail inequality for the concentration of a general function of n independent random variables. Next, under some reasonable notion of s…

Quantum model improves safety in machine learning.

On a 4-dimensional compact symplectic manifold, we consider a smooth family of compatible almost-complex structures such that at time zero the induced metric is Hermite-Einstein almost-Kähler metric with zero or negative Hermitian scalar curvature. We prove, under certain hypothesis, the existence of a smooth family of…

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

New bounds explain modern machine learning algorithms' generalization.

Proof of Gaussian ML estimator consistency in linear auto-regressive models.

Pruning is a well-established technique for removing unnecessary structure from neural networks after training to improve the performance of inference. Several recent results have explored the possibility of pruning at initialization time to provide similar benefits during training. In particular, the "lottery ticket h…

Hypothesis testing plays a central role in statistical inference, and is used in many settings where privacy concerns are paramount. This work answers a basic question about privately testing simple hypotheses: given two distributions $P$ and $Q$, and a privacy level $\varepsilon$, how many i.i.d. samples are needed to…

Peters (2011a) defined an optimal leverage which maximizes the time-average growth rate of an investment held at constant leverage. It was hypothesized that this optimal leverage is attracted to 1, such that, e.g., leveraging an investment in the market portfolio cannot yield long-term outperformance. This places a str…

Transformers learn algorithms for in-context learning with bounds and stability analysis.

Study stability of selective SSMs with discontinuous gating.

Given a path of almost-Kähler metrics compatible with a fixed symplectic form on a compact 4-manifold such that at time zero the almost-Kähler metric is an extremal Kähler one, we prove, for a short time and under a certain hypothesis, the existence of a smooth family of extremal almost-Kähler metrics compatible with t…

A new method uses vectorized summaries of persistence diagrams for efficient hypothesis testing.

Sleep-based regularization stabilizes STDP in recurrent neural networks.

Cryptocurrencies show stable prices as a medium of exchange.

The paper examines how to test if two learning algorithms produce similar outcomes.

StatLoRA uses statistical inference to allocate ranks in LoRA fine-tuning, improving performance.

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, comprised of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and tr…

Quantum channels' contraction under privacy constraints studied.

A framework for hypothesis testing on attributed graphs using sampling.