We introduce a novel incremental decision tree learning algorithm, Hoeffding Anytime Tree, that is statistically more efficient than the current state-of-the-art, Hoeffding Tree. We demonstrate that an implementation of Hoeffding Anytime Tree---"Extremely Fast Decision Tree", a minor modification to the MOA implementat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Hoeffding Trees”
HATT improves online decision tree ensembles by using a more eager splitting strategy.
problem Improving the efficiency of online decision tree ensembles.
method Replacing Hoeffding Tree's split strategy with HATT, which uses the Hoeffding Test for candidate splits.
result HATT outperforms Hoeffding Tree in online bagging and boosting ensembles, as shown by significant performance improvements in various testbenches.
Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithmstat.ML
TreeHFD algorithm explains tree ensemble models through hierarchical orthogonality.
problem Difficulty in explaining black-box tree ensemble models.
method TreeHFD algorithm using hierarchical orthogonality constraints.
result TreeHFD estimates Hoeffding decomposition from data samples.
Dynamic Model Tree improves online learning for evolving data streams.
problem Effective and transparent machine learning on data streams is challenging.
method Revisit Model Trees for data stream applications, introducing Dynamic Model Tree.
result Dynamic Model Tree reduces the number of splits and outperforms state-of-the-art models.
Machine learning software accounts for a significant amount of energy consumed in data centers. These algorithms are usually optimized towards predictive performance, i.e. accuracy, and scalability. This is the case of data stream mining algorithms. Although these algorithms are adaptive to the incoming data, they have…
FAHT improves fairness in online decision-making systems.
problem Discrimination in decision support systems due to biased data.
method FAHT is an adaptive fairness-aware decision tree classifier based on Hoeffding Tree algorithm.
result FAHT maintains fairness while maintaining moderate predictive performance in streaming environments.
Improves decision tree performance by correcting split selection errors.
problem Invalid statistical guarantees in split selection for decision trees.
method Introduces anytime-valid inference to provide valid statistical guarantees.
result Provides anytime-valid control of false splits under arbitrary data streams.
Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setti…
New algorithm detects and handles concept drift in data streams.
problem Handling concept drift in data streams for timely predictions.
method Hybrid Forest algorithm combining Hoeffding Trees and fast startup.
result The algorithm outperforms other methods in classification and regression tasks.
HCBM improves deep learning explainability by non-linear concept aggregation.
problem Lack of explainable and accurate predictions in deep learning for high-stake decisions.
method Introduce Hoeffding Concept Bottleneck Models (HCBM) using Hoeffding functional decomposition of gradient-boosted trees for non-linear and sparse concept aggregation.
result HCBM outperforms standard linear CBM and is robust to interconcept leakage.
The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.
problem Applying Hoeffding's inequality to non-ergodic Markov chains.
method Integrates generalized concentrability condition via IPM to extend traditional hypotheses.
result Demonstrates utility in machine learning applications such as empirical risk minimization and bandits.
Improved algorithm for Lipschitz bandit optimization with reduced complexity.
problem Efficiently solving the Lipschitz bandit optimization problem.
method Tree UCB-Hoeffding algorithm with adaptive partitions and tree-based search strategy.
result Achieves the regret lower bound up to a logarithmic factor with computational cost.
Trading strategy uses Hoeffding's Inequality to predict financial regime change.
problem Predicting financial regime change for trading strategies.
method Applies Hoeffding's Inequality to trading performance data.
result Early warning of financial regime change can be detected.
A Hoeffding Inequality for Finite State Markov Chains and its Applications to Markovian Banditsmath.ST
The paper develops a Hoeffding inequality for Markov chains and applies it to bandit problems.
problem Developing a Hoeffding inequality for Markov chains and applying it to bandit problems.
method Developed a Hoeffding inequality for the partial sums of an irreducible Markov chain on a finite state space.
result Demonstrated the inequality's effectiveness in identifying approximately best Markovian arms and minimizing regret in Markovian bandits.
Generalizes Hoeffding's decomposition for dependent inputs under mild conditions.
problem Performing global sensitivity analysis on black-box models with dependent inputs.
method Proposes a novel framework based on probability theory, functional analysis, and combinatorics to handle dependencies.
result Any square-integrable, real-valued function of random elements with mild dependence assumptions can be uniquely additively decomposed.
RKHSMetaMod estimates complex models' Hoeffding decomposition for sensitivity analysis.
problem Estimating the Hoeffding decomposition of complex models for sensitivity analysis.
method Penalized empirical least-squares minimization with RKHS ridge group sparse optimization.
result Estimates non-zero Sobol indices for sensitivity analysis.
Proposes a new online learning strategy for multi-target regression in data streams.
problem Challenges in learning from high-throughput data streams, especially in multi-target regression.
method Extends existing online decision tree learning algorithm to consider inter-target dependencies.
result SST-HT presents superior predictive accuracy compared to state-of-the-art algorithms.
New unbiased variance estimator for random forests using Hoeffding decomposition.
problem Uncertainty quantification in random forests with large kernel sizes and small sample sizes.
method Proposes a new Hoeffding decomposition view for variance estimation, establishing unbiased estimators and ratio consistency.
result Establishes the ratio consistency of the proposed variance estimator, justifying confidence interval coverage rates.
Paper proposes a new confidence dimension to measure DNN generalization.
problem Measuring the generalization ability of deep neural networks is challenging.
method Introduces confidence dimension (CD) based on Hoeffding's inequality and VC-dimension.
result CD provides a feasible framework to calculate the upper bound of generalization.
Motivated by applications in model-free finance and quantitative risk management, we consider Fréchet classes of multivariate distribution functions where additional information on the joint distribution is assumed, while uncertainty in the marginals is also possible. We derive optimal transport duality results for the…
ESRF reduces ARF ensemble size without sacrificing accuracy.
problem Over-provisioning of ARF ensemble leads to high CPU and memory consumption.
method ESRF uses a swap and elastic component to dynamically adjust the number of classifiers.
result ESRF reduces the number of classifiers by up to one third without sacrificing accuracy.
Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show t…
A new method combines multiple bounds and betting strategies for selective prediction, improving risk coverage in data-scarce settings.
problem Selective prediction with risk control in data-scarce domains.
method Combines concentration inequalities, multiple-testing corrections, and betting-based confidence sequences.
result Transfer-Informed Betting achieves tighter bounds and better coverage in data-scarce settings.
Paper develops a new inequality for non-causal machine learning.
problem Current concentration inequalities cannot be applied to non-causal machine learning.
method Develops a framework for non-causal random fields and proves a Hoeffding-type inequality.
result Obtains a Hoeffding-type concentration inequality for non-causal random fields.
Paper develops probabilistic bounds for a stochastic gradient algorithm in non-convex problems.
problem Stochastic optimization in non-convex finite sum problems.
method Develops a new dimension-free Azuma-Hoeffding type bound for a martingale difference sequence.
result Empirical results show superior probabilistic performance of Prob-SARAH compared to other algorithms.
Big data trend has enforced the data-centric systems to have continuous fast data streams. In recent years, real-time analytics on stream data has formed into a new research field, which aims to answer queries about what-is-happening-now with a negligible delay. The real challenge with real-time stream data processing …
Paper refines PAC-Bayes bounds for bandit problems.
problem Improving probabilistic bounds for off-policy learning.
method Optimizes PAC-Bayesian bounds using a new parameter optimization approach.
result Provides two parameter-free PAC-Bayes bounds that nearly match optimal rates.
Concentration inequalities are indispensable tools for studying the generalization capacity of learning models. Hoeffding's and McDiarmid's inequalities are commonly used, giving bounds independent of the data distribution. Although this makes them widely applicable, a drawback is that the bounds can be too loose in so…
Unified method for MMD variance estimation improves accuracy and computational efficiency.
problem Variance estimation for MMD in nonparametric testing.
method Unified finite-sample characterization of MMD variance through U-statistic and Hoeffding decomposition; exact acceleration method for univariate case.
result Unified estimators improve accuracy and computational efficiency for MMD variance.
Beyond Consistency: Inference for the Relative risk functional in Deep Nonparametric Cox Modelsstat.ML
Develops asymptotic theory for deep Cox models to enable valid inference.
problem Theoretical gaps in deep neural network estimators for Cox models.
method Asymptotic distribution theory linking in-sample optimization error to population risk.
result Pointwise and multivariate asymptotic normality for subsampled ensemble estimators.
The paper provides mean-square error bounds for stochastic approximation algorithms.
problem Error bounds for recursive equations with Markovian disturbances.
method Analysis of mean-square error for stochastic approximation algorithms.
result Mean-square error achieves the optimal rate of under certain conditions.
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …
This paper improves neural tangent kernels for better generalization and local elasticity.
problem Performance gap between neural tangent kernels and real-world neural networks.
method Introduces label-aware kernels using Hoeffding decomposition.
result Models trained with proposed kernels simulate NNs better in terms of generalization and local elasticity.
The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.
problem Constructing reliable confidence regions for band-limited functions from noisy data.
method Improved norm bounds using Hoeffding's inequality and empirical Bernstein bound, majority voting to aggregate intervals.
result Confidence intervals retain their simultaneous coverage guarantee even when aggregated from random subsamples.
The International Classification of Functioning, Disability, and Health for Children and Youth (ICF-CY) is a scaffold for designating and systematizing data on functioning and disability. It offers a standard semantic and a theoretical foundation for the demarcation and extent of wellbeing and infirmity. The multidimen…
New methods for private statistical inference under local differential privacy.
problem Private statistical inference for population means with bounded observations.
method Nonparametric, nonasymptotic statistical inference using a generalized randomized response mechanism.
result Private confidence intervals and sequences for population means under LDP constraints.
The paper develops concentration inequalities for structured random data, extending beyond independent terms.
problem Developing concentration inequalities for structured weighted sums of random data, including tensors and matrix-valued data.
method The paper develops Hoeffding and Bernstein bounds for structured weighted sums under exchangeability, extending beyond the classical framework of independent terms.
result The paper develops a sharper concentration bound for combinatorial sums of matrix arrays.
Learning from data streams is an increasingly important topic in data mining, machine learning, and artificial intelligence in general. A major focus in the data stream literature is on designing methods that can deal with concept drift, a challenge where the generating distribution changes over time. A general assumpt…
New method detects changes in high-dimensional Markov processes without explicit likelihood evaluation.
problem Quickest change detection in Markov processes with unknown transition kernels.
method Learn conditional score from sample pairs, develop score-based CUSUM procedure.
result Exponential lower bounds on mean time to false alarm and asymptotic upper bounds on detection delay.
We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time versi…
Proves new concentration inequalities for sub-gaussian and sub-exponential variables.
problem Understanding functions of independent random variables better.
method Sub-gaussian and sub-exponential conditions, Rademacher complexities, Lipschitz function classes.
result Extension of Rademacher complexities to unbounded sub-exponential distributions.
In this paper, we propose a novel framework to analyze the theoretical properties of the learning process for a representative type of domain adaptation, which combines data from multiple sources and one target (or briefly called representative domain adaptation). In particular, we use the integral probability metric t…
Concentration of infinitely exchangeable sequences with bounded-difference constants
problem Quantifying uncertainty in AI benchmarks
method Using a mixture-free Hoeffding-type bound
result Tight, mixture-free Hoeffding-type bound for zero-sum linear contrasts
When I first encountered PAC-Bayesian concentration inequalities they seemed to me to be rather disconnected from good old-fashioned results like Hoeffding's and Bernstein's inequalities. But, at least for one flavour of the PAC-Bayesian bounds, there is actually a very close relation, and the main innovation is a cont…
Fine-grained gap-dependent regret bounds for reinforcement learning.
problem Achieving optimal regret bounds for reinforcement learning with suboptimality gaps.
method Developed novel analytical frameworks and refined algorithms for UCB-based and non-UCB-based reinforcement learning.
result Established the first fine-grained gap-dependent regret bounds for both UCB-based and non-UCB-based algorithms.
The betting CI outperforms classical methods in constructing confidence intervals for bounded means.
problem Constructing nonasymptotic confidence intervals for bounded means.
method A betting-based approach to define and time-uniform variants of confidence intervals (CSs).
result The betting CI matches the fundamental limits, outperforming existing empirical Bernstein CIs.
This paper improves Q-learning bounds using reference-advantage decomposition.
problem Improving Q-learning bounds in MDPs with positive suboptimality gaps.
method Develops a novel error decomposition framework to prove gap-dependent regret bounds.
result Establishes logarithmic gap-dependent regret bounds for Q-learning.
Model-free bounds on Value-at-Risk using extreme value information and statistical distancesq-fin.RM
We derive bounds on the distribution function, therefore also on the Value-at-Risk, of where is an aggregation function and is a random vector with known marginal distributions and partially known dependence structure. More specifically, we analyze three type…