arXiv research

The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.

Trading strategy uses Hoeffding's Inequality to predict financial regime change.

Motivated by applications in model-free finance and quantitative risk management, we consider Fréchet classes of multivariate distribution functions where additional information on the joint distribution is assumed, while uncertainty in the marginals is also possible. We derive optimal transport duality results for the…

We introduce a novel incremental decision tree learning algorithm, Hoeffding Anytime Tree, that is statistically more efficient than the current state-of-the-art, Hoeffding Tree. We demonstrate that an implementation of Hoeffding Anytime Tree---"Extremely Fast Decision Tree", a minor modification to the MOA implementat…

HATT improves online decision tree ensembles by using a more eager splitting strategy.

Paper refines PAC-Bayes bounds for bandit problems.

Concentration inequalities are indispensable tools for studying the generalization capacity of learning models. Hoeffding's and McDiarmid's inequalities are commonly used, giving bounds independent of the data distribution. Although this makes them widely applicable, a drawback is that the bounds can be too loose in so…

A new method combines multiple bounds and betting strategies for selective prediction, improving risk coverage in data-scarce settings.

Paper develops probabilistic bounds for a stochastic gradient algorithm in non-convex problems.

The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …

This paper develops a Hoeffding inequality for the partial sums $\sum_{k=1}^n f (X_k)$, where $\{X_k\}_{k \in \mathbb{Z}_{> 0}}$ is an irreducible Markov chain on a finite state space $S$, and $f : S \to [a, b]$ is a real-valued function. Our bound is simple, general, since it only assumes irreducibility and finiteness…

TreeHFD algorithm explains tree ensemble models through hierarchical orthogonality.

The paper develops concentration inequalities for structured random data, extending beyond independent terms.

Paper proposes a new confidence dimension to measure DNN generalization.

We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time versi…

Paper develops a new inequality for non-causal machine learning.

Generalizes Hoeffding's decomposition for dependent inputs under mild conditions.

New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.

When I first encountered PAC-Bayesian concentration inequalities they seemed to me to be rather disconnected from good old-fashioned results like Hoeffding's and Bernstein's inequalities. But, at least for one flavour of the PAC-Bayesian bounds, there is actually a very close relation, and the main innovation is a cont…

Concentration of infinitely exchangeable sequences with bounded-difference constants

In this paper, we propose a novel framework to analyze the theoretical properties of the learning process for a representative type of domain adaptation, which combines data from multiple sources and one target (or briefly called representative domain adaptation). In particular, we use the integral probability metric t…

New methods for private statistical inference under local differential privacy.

This paper improves Q-learning bounds using reference-advantage decomposition.

We derive bounds on the distribution function, therefore also on the Value-at-Risk, of $\varphi(\mathbf X)$ where $\varphi$ is an aggregation function and $\mathbf X = (X_1,\dots,X_d)$ is a random vector with known marginal distributions and partially known dependence structure. More specifically, we analyze three type…

Fine-grained gap-dependent regret bounds for reinforcement learning.

The betting CI outperforms classical methods in constructing confidence intervals for bounded means.

Estimates barycenter in geodesic spaces with finite sample bounds.

Develops asymptotic theory for deep Cox models to enable valid inference.

New unbiased variance estimator for random forests using Hoeffding decomposition.

Valid p-value for bounded random variables without distributional assumptions.

Machine learning software accounts for a significant amount of energy consumed in data centers. These algorithms are usually optimized towards predictive performance, i.e. accuracy, and scalability. This is the case of data stream mining algorithms. Although these algorithms are adaptive to the incoming data, they have…

Bayesian sequence prediction is a simple technique for predicting future symbols sampled from an unknown measure on infinite sequences over a countable alphabet. While strong bounds on the expected cumulative error are known, there are only limited results on the distribution of this error. We prove tight high-probabil…

In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for subagged estimators, both for classification and regressor. General loss functions and class of predictors with both finite and infinite VC-dimension are considered. We slightly generalize the formali…

Improves decision tree performance by correcting split selection errors.

Near-optimal confidence intervals for bounded data.

HCBM improves deep learning explainability by non-linear concept aggregation.

In this paper, we develop a general theory of truncated inverse binomial sampling. In this theory, the fixed-size sampling and inverse binomial sampling are accommodated as special cases. In particular, the classical Chernoff-Hoeffding bound is an immediate consequence of the theory. Moreover, we propose a rigorous and…

This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued tha…

In this paper, we propose an R package, called RKHSMetaMod, that implements a procedure for estimating a meta-model of a complex model. The meta-model approximates the Hoeffding decomposition of the complex model and allows us to perform sensitivity analysis on it. It belongs to a reproducing kernel Hilbert space that …

Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show t…

We consider the Lipschitz bandit optimization problem with an emphasis on practical efficiency. Although there is rich literature on regret analysis of this type of problem, e.g., [Kleinberg et al. 2008, Bubeck et al. 2011, Slivkins 2014], their proposed algorithms suffer from serious practical problems including extre…

Dynamic Model Tree improves online learning for evolving data streams.

Paper improves CI and CS for bounded means using betting and mixtures.

Unified method for MMD variance estimation improves accuracy and computational efficiency.

We present a set of high-probability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. Our results extend the PAC-Bayesian analysis in learning theory from the i.i.d. setting to martingales opening the way for…

Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setti…

This paper improves neural tangent kernels for better generalization and local elasticity.