Boosting meta-trees improve decision tree performance.
problem Overfitting in decision trees.
method Boosting approach to construct multiple meta-trees.
result Ensembles of meta-trees prevent overfitting.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “boosted tree ensembles”
This study extends verifiable learning to boosted tree ensembles, enabling efficient security verification.
problem Efficiently verifying the robustness of boosted tree ensembles against norm-based attackers.
method Formal verification of robustness for large-spread boosted tree ensembles, considering -norm and pseudo-polynomial time for -norm verification.
result Polynomial time verification for -norm attackers, NP-hard for other norms, and pseudo-polynomial time for -norm verification.
Paper studies ensemble probabilistic regression trees for smooth approximations.
problem Smooth approximations of regression functions.
method Ensemble versions of probabilistic regression trees.
result Ensemble probabilistic regression trees are consistent and perform well.
Boost-R uses gradient boosted trees for analyzing recurrence data.
problem Analyzing recurrence data with static and dynamic features.
method Gradient boosted additive trees with time-dependent functions.
result Estimates the cumulative intensity function of recurrent event processes.
Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based le…
Classifier evasion consists in finding for a given instance the nearest instance such that the classifier predictions of and are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixe…
Tree ensembles are flexible predictive models that can capture relevant variables and to some extent their interactions in a compact and interpretable manner. Most algorithms for obtaining tree ensembles are based on versions of boosting or Random Forest. Previous work showed that boosting algorithms exhibit a cyclic b…
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative poin…
TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…
BoostForest combines multiple BoostTree models for improved accuracy.
problem Improving ensemble learning performance.
method BoostTree uses gradient boosting and random cut-points. BoostForest bootstraps training data and randomly samples parameters.
result BoostForest outperforms classical ensemble methods on 35 datasets.
A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
problem Lack of effective short-term load forecasting methods.
method Hybrid strategy integrating four different inference models: tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB).
result Demonstrates effectiveness of hybrid strategy on real datasets.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
agtboost speeds up gradient tree boosting with automatic complexity adjustment.
problem Speeding up and simplifying gradient tree boosting computations.
method Adaptive gradient tree boosting with automatic complexity adjustment and feature importance.
result Significant decrease in computation time and simplification of model complexity.
Tree ensembles, such as random forest and boosted trees, are renowned for their high prediction performance, whereas their interpretability is critically limited. In this paper, we propose a post processing method that improves the model interpretability of tree ensembles. After learning a complex tree ensembles in a s…
In this paper we propose a method to build a neural network that is similar to an ensemble of decision trees. We first illustrate how to convert a learned ensemble of decision trees to a single neural network with one hidden layer and an input transformation. We then relax some properties of this network such as thresh…
This paper improves prediction rule ensembles using model-based data generation.
problem Improving the sparsity and predictive accuracy of prediction rule ensembles.
method The authors use surrogate models to train Lasso regression with data generated by a boosted decision tree ensemble, improving PRE performance.
result The use of surrogacy models can substantially improve the sparsity of PRE while retaining predictive accuracy.
We introduce a novel boosting algorithm called `KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous …
This research tackles uncertainty in gradient boosting models using ensemble methods.
problem Quantifying uncertainty in gradient boosting models for high-risk applications.
method Probabilistic ensemble-based framework for gradient boosting classification and regression models.
result Ensembles of gradient boosting models detect anomalous inputs but have limited ability to improve total uncertainty.
Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we …
Gradient boosting with randomized trees reduces discontinuities and complexity.
problem Discontinuities in regression functions due to sparse training data.
method Gradient boosting machine with partially randomized decision trees.
result Improves robustness and computational efficiency of gradient boosting.
New test improves tree ensemble pruning for better model performance.
problem Lack of robust theoretical justification for penalty terms in tree ensembles.
method Developed a novel hypothesis test for tree ensemble split quality.
result Significant reduction in out-of-sample loss using the new test.
VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Treescs.LG
VisRuler simplifies decision extraction from bagged and boosted trees.
problem Complexity and lack of interpretability in ensemble models.
method Visual analytics tool for selecting robust models, important features, and essential decisions.
result Users successfully extracted and explained decisions from ensemble models.
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
New methods improve tree ensemble models by compressing them while maintaining accuracy.
problem Theoretical understanding and practical compression of tree ensembles like random forests and gradient boosting machines.
method Spectral perspective on tree ensembles, deriving minimax rates and developing compression schemes.
result Leading eigenfunctions/singular vectors capture dominant predictive directions, leading to smaller, competitive models.
Slow-Growing Treesstat.ML
A single slow-growing tree matches Random Forest's performance.
problem Matching Random Forest's performance with a single tree.
method SGT uses a learning rate to tame CART's greedy algorithm, improving on greedy ML algorithms.
result SGT and tree ensembles like Booging, BT, and RF improve performance.
We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems. Boosting combines several regression trees to improve their accuracy without significantly reducing their inherent interpretability. Prior work has focused independently on reinforcement lear…
Gradient boosted decision trees are a popular machine learning technique, in part because of their ability to give good accuracy with small models. We describe two extensions to the standard tree boosting algorithm designed to increase this advantage. The first improvement extends the boosting formalism from scalar-val…
In machine learning ensemble methods have demonstrated high accuracy for the variety of problems in different areas. Two notable ensemble methods widely used in practice are gradient boosting and random forests. In this paper we present InfiniteBoost - a novel algorithm, which combines important properties of these two…
Credit scoring plays a vital role in the field of consumer finance. Survival analysis provides an advanced solution to the credit-scoring problem by quantifying the probability of survival time. In order to deal with highly heterogeneous industrial data collected in Chinese market of consumer finance, we propose a nonp…
Shallow trees in ensemble models make models more interpretable and sometimes better.
problem Lack of transparency in high-performing tree ensemble models.
method Developed an interpretation algorithm to convert tree ensembles into functional ANOVA representations. Proposed strategies to enhance interpretability.
result Shallow trees in ensemble models can lead to better generalization performance and improved interpretability.
Boosted ensemble of decision tree (DT) classifiers are extremely popular in international competitions, yet to our knowledge nothing is formally known on how to make them \textit{also} differential private (DP), up to the point that random forests currently reign supreme in the DP stage. Our paper starts with the proof…
DOFEN improves DNN performance on tabular data benchmarks.
problem DOFEN tackles the performance gap between DNNs and tree-based models on tabular data.
method DOFEN uses a two-level rODT forest ensembling process inspired by oblivious decision trees.
result DOFEN achieves state-of-the-art results on the Tabular Benchmark.
TREX explains tree ensembles by identifying key training examples.
problem Identifying which training examples most influence tree ensemble predictions.
method TREX builds a surrogate model using a kernel that captures tree ensemble structure, approximating the original model.
result TREX provides accurate and effective explanations for tree ensembles.
Proposes GBBHE for efficient large-scale regression.
problem Large-scale regression problems.
method Gradient Boosting with binary histogram partition and ensemble learning.
result Improves computational efficiency and performance on large datasets.
HATT improves online decision tree ensembles by using a more eager splitting strategy.
problem Improving the efficiency of online decision tree ensembles.
method Replacing Hoeffding Tree's split strategy with HATT, which uses the Hoeffding Test for candidate splits.
result HATT outperforms Hoeffding Tree in online bagging and boosting ensembles, as shown by significant performance improvements in various testbenches.
Gradient boosting adapted for vector inputs.
problem Applying gradient boosting to multi-class classification problems.
method Extended gradient boosting framework to vector inputs using histogram-based decision trees.
result Efficient algorithm for vector-valued objectives.
Technology and collaboration enable dramatic increases in the size of psychological and psychiatric data collections, but finding structure in these large data sets with many collected variables is challenging. Decision tree ensembles like random forests (Strobl, Malley, and Tutz, 2009) are a useful tool for finding st…
SCORE improves tree-based predictions with boosted residual extraTrees.
problem Improving tree-based prediction models with reduced errors.
method Inspired by representation learning, SCORE uses boosting, regularized regression, and variable selection.
result SCORE provides comparable or superior performance compared to other models.
Algorithm infers sampling distribution from i.i.d. samples without supervision.
problem Learning probability distributions from unlabeled data.
method Unsupervised tree boosting using additive tree ensembles and new distributional operations.
result Algorithm outperforms deep learning in multivariate density estimation.
New methods improve prediction performance and reduce computation time in boosting and random forest models.
problem Improving prediction performance and reducing computation time in boosting and random forest models.
method Random tree depth injection approach for Boosting and Random Forests.
result The new methods can improve prediction performance and reduce computation time by up to 40%.
Tree ensembles like RF and GBT can be seen as kernels, improving regression and classification performance.
problem Improving kernel methods for tree ensemble based models.
method Investigation of RF and GBT kernels in simulation and real data.
result RF and GBT kernels are competitive to their respective ensembles in higher dimensions, particularly with noisy features.
A Framework for an Assessment of the Kernel-target Alignment in Tree Ensemble Kernel Learningstat.ML
The study investigates kernel-target alignment in tree ensemble kernels.
problem The degree of kernel-target alignment affects the performance of tree ensemble kernels in kernel learning.
method Eigenanalysis of the kernel matrix and sensitivity analysis via landmark learning.
result Good performance of tree ensemble kernels is associated with strong kernel-target alignment.
FAST-DAD distills complex ensemble models into faster, more accurate individual models.
problem Deploying complex AutoML ensemble predictors on tabular data is slow, large, and opaque.
method Data augmentation strategy based on Gibbs sampling from a self-attention pseudolikelihood estimator.
result FAST-DAD distillation produces significantly better individual models than standard training.
We address the problem of finding influential training samples for a particular case of tree ensemble-based models, e.g., Random Forest (RF) or Gradient Boosted Decision Trees (GBDT). A natural way of formalizing this problem is studying how the model's predictions change upon leave-one-out retraining, leaving out each…
Enhanced Random Forests outperform XGBoost across binary classification datasets.
problem Improving performance of Random Forests in binary classification.
method Adaptive sample and model weighting, iterative algorithm for sample weights, personalized tree weighting schemes.
result Significantly outperforms XGBoost across 15 binary classification datasets.
Understanding how "black-box" models arrive at their predictions has sparked significant interest from both within and outside the AI community. Our work focuses on doing this by generating local explanations about individual predictions for tree-based ensembles, specifically Gradient Boosting Decision Trees (GBDTs). G…
Integrates differentiable decision trees into neural networks for faster training and inference.
problem Combining differentiability and conditional computation in tree ensembles for neural networks.
method Sparse activation function and specialized forward/backward propagation algorithms for efficient training and inference.
result 10x speed-ups and 20x reduction in parameters compared to existing methods, while maintaining performance.
DBDT uses deep boosting decision trees for fraud detection.
problem Fraud detection in imbalanced data.
method Gradient boosting with neural networks (SDT), AUC maximization.
result DBDT significantly improves fraud detection performance.