Alpha-trimming prunes trees in random forests to improve predictive performance.
problem Improving predictive performance of random forests by locally adaptive tree pruning.
method Alpha-trimming is a fast pruning algorithm that prunes trees in a random forest based on signal-to-noise ratio, controlled by a tuning parameter.
result Alpha-trimming often lowers mean squared prediction error compared to fully grown random forests.
A new randomized tree classifier outperforms traditional CARTs.
problem Traditional greedy CARTs are slow and inaccurate.
method Optimal decision trees using continuous optimization.
result Randomized tree classifier outperforms traditional CARTs.
Single tree outperforms random forest in testing accuracy.
problem The challenge of improving single decision tree performance.
method Gradient-based entire tree optimization framework, scaled sigmoid approximation, numerical stability algorithm, subtree polish strategy.
result Optimized single tree outperforms classic random forest by 2.03% on average.
A single slow-growing tree matches Random Forest's performance.
problem Matching Random Forest's performance with a single tree.
method SGT uses a learning rate to tame CART's greedy algorithm, improving on greedy ML algorithms.
result SGT and tree ensembles like Booging, BT, and RF improve performance.
Gradient boosting with randomized trees reduces discontinuities and complexity.
problem Discontinuities in regression functions due to sparse training data.
method Gradient boosting machine with partially randomized decision trees.
result Improves robustness and computational efficiency of gradient boosting.
A new tree-based model improves uncertainty estimation in sequential optimization.
problem Improving uncertainty estimation in sequential model-based optimization.
method Proposed a new ensemble of randomized trees (BwO forest) with bagging and oversampling.
result BwO forest outperforms existing tree-based models in various optimization scenarios.
Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based le…
New methods improve prediction performance and reduce computation time in boosting and random forest models.
problem Improving prediction performance and reducing computation time in boosting and random forest models.
method Random tree depth injection approach for Boosting and Random Forests.
result The new methods can improve prediction performance and reduce computation time by up to 40%.
A new method for decision tree selection in recommendation systems.
problem Feature-based selection of a single tree from an ensemble for dynamic interpretation.
method A multi-armed contextual bandit recommendation framework that trains a system on top of Random Forests to identify the most relevant tree.
result The dynamic method outperforms an independent CART tree and is comparable to Random Forest in predictive performance.
SBT model uses randomized sharding and sub-models to improve Bayesian Additive Regression Trees.
problem Improving efficiency and accuracy of Bayesian Additive Regression Trees.
method Randomized sharding, sub-models, intersection tree structure, optimal design.
result Theoretical optimal weights and worst-case complexity of SBT model.
A new random forest algorithm improves tree construction for optimal performance.
problem Improving the performance of random forests, especially in complex and smooth scenarios.
method Adaptive split-balancing method using permutation-based splitting criterion.
result Achieves minimax optimality under various Lipschitz and Hölder classes.
Method uses trinomial trees to price nontraditional options.
problem Pricing of random-expiry options with early expiry.
method Developed a trinomial tree approach to interpret early expiry.
result The method is free of arbitrage and can be implemented efficiently.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
This study argues for pruning trees in random forests to improve performance in low signal-to-noise scenarios.
problem Improving random forest performance in scenarios with low signal-to-noise ratio.
method Using regularization theory, the study re-examines the depth of trees in random forests and provides evidence that shallow trees are advantageous.
result Random forests with shallow trees are advantageous when the signal-to-noise ratio is low.
Detecting correlated trees helps align sparse graphs.
problem Detecting correlation between trees for sparse random graphs.
method MPAlign message-passing algorithm for graph alignment.
result MPAlign succeeds in polynomial time for partial alignment.
Improved Random Forests detect pure interactions better.
problem Random Forests struggle with certain pure interactions.
method Alternative partitioning schemes during tree construction.
result Improved Random Forests enhance fitting ability in scenarios with pure interactions.
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
Study a risk model with tree-structured Poisson-Markov random field for rainfall events.
problem Dependence between rainfall frequencies in insurance portfolios.
method Tree-structured Markov random field with Poisson marginals.
result Asymptotic results for portfolio risk and risk allocation.
Paper studies ensemble probabilistic regression trees for smooth approximations.
problem Smooth approximations of regression functions.
method Ensemble versions of probabilistic regression trees.
result Ensemble probabilistic regression trees are consistent and perform well.
The paper uses regression trees/random forests to price Bermudan options more efficiently.
problem Pricing Bermudan options with conditional expectation estimation.
method Estimates conditional expectations using regression trees or random forests instead of traditional regression methods.
result Regression trees/random forests provide better results in high dimensions.
Optimizes random forest inference by defining step order to maximize accuracy.
problem Limited inference time in resource-constrained systems.
method Designs anytime random forest algorithm on step granularity, proposing optimal step order.
result Backward Squirrel Order performs nearly as well as the optimal step order.
The paper proposes a new variant of a decision tree, called an Extreme Learning Tree. It consists of an extremely random tree with non-linear data transformation, and a linear observer that provides predictions based on the leaf index where the data samples fall. The proposed method outperforms linear models on a bench…
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
TRUST improves tree models' accuracy while maintaining interpretability.
problem Piecewise-constant regression trees lack in predictive accuracy compared to black-box models.
method Combines Random Forest accuracy with interpretability of shallow trees and sparsity of linear models, using LLMs for explanations.
result TRUST outperforms other interpretable models in predictive accuracy and matches Random Forest's accuracy.
Estimates tree-based density from random vectors.
problem Estimating the density of a random vector in high dimensions.
method Optimal spanning tree minimizes Kullback-Leibler divergence; tree density estimate constructed from i.i.d. data.
result Tree density estimate converges to true density as sample size increases.
We analyze the consistency of decision trees and random forests in regression.
problem Consistency of decision trees and random forests in regression.
method Elementary proofs following classical arguments for smoothing methods.
result Establish weak and almost sure convergence of honest trees and forest averages to the true regression function.
Paper estimates the order of vertices in random recursive trees.
problem Estimating the order of arrival of vertices in random recursive trees.
method Proposes an order estimator based on the Jordan centrality measure and defines risk measures.
result Establishes a nearly optimal estimator for the problem.
RST improves environmental time series classification accuracy using randomized B-spline trees.
problem Improving accuracy in classifying complex environmental time series.
method Randomized Spline Trees (RST) integrates randomized functional representations into ensemble learning.
result RST variants outperform standard Random Forests and Gradient Boosting on most environmental time series datasets.
We prove that a "random" free group outer automorphism is an ageometric fully irreducible outer automorphism whose ideal Whitehead graph is a union of triangles. In particular, we show that its attracting (and repelling) tree is a nongeometric R-tree all of whose branch points are trivalent
New tree-structured Markov fields with Poisson marginals for counting variables.
problem Counting variables with complex dependencies.
method Tree-structured Markov random fields with Poisson marginals.
result Straightforward sampling and joint probability calculations.
Study finds a linear lower bound on conformal dimension for random hyperbolic groups.
problem Understanding conformal dimension in random hyperbolic groups.
method Building undistorted round trees from lower density groups.
result Achieves a linear lower bound in l at all densities 0<d<1/2. Deep forests enhance expressiveness exponentially with depth, not width or tree size.
problem Understanding the role of depth, width, and tree size in deep forest performance.
method Provided upper and lower bounds on deep forest approximation complexity.
result Depth exponentially enhances deep forest expressiveness.
The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.
problem Random forests' uniform voting fails to correct errors in regions where incorrect tree representations outnumber correct ones.
method The paper introduces using the structural pattern of each tree's decision path as an instance-adaptive reliability signal to identify and weight more reliable trees.
result Using the proposed method yields a statistically significant accuracy improvement over RF on 36 binary classification benchmarks.
This paper presents a brand new nonparametric density estimation strategy named the best-scored random forest density estimation whose effectiveness is supported by both solid theoretical analysis and significant experimental performance. The terminology best-scored stands for selecting one density tree with the best e…
Optimal survival trees ensemble reduces tree count and improves predictive performance.
problem Improving predictive performance in survival analysis.
method Grows a forest of optimal survival trees by ranking and selecting the best trees based on out-of-bag error.
result Reduces the number of trees in the ensemble while improving predictive performance.
Within machine learning, the supervised learning field aims at modeling the input-output relationship of a system, from past observations of its behavior. Decision trees characterize the input-output relationship through a series of nested if−then−else questions, the testing nodes, leading to a set of predictions, th…
HRF enhances tree diversity in random forests to improve performance.
problem Selection bias and lack of diversity in random forests.
method Introducing heterogeneity during tree construction by assigning lower weights to features used in previous trees.
result HRF outperforms other ensemble methods in accuracy across 52 datasets.
The paper challenges the use of decision trees for pointwise inference due to slow convergence rates.
problem The slow convergence rates of decision trees in uniform norm, especially with non-vanishing probability.
method Demonstrates the limitations of adaptive recursive partitioning and shows how random forests can improve performance.
result Decision trees can fail to achieve polynomial rates of convergence in uniform norm, even with pruning.
The infinitesimal jackknife (IJ) has recently been applied to the random forest to estimate its prediction variance. These theorems were verified under a traditional random forest framework which uses classification and regression trees (CART) and bootstrap resampling. However, random forests using conditional inferenc…
Hash codes are efficient data representations for coping with the ever growing amounts of data. In this paper, we introduce a random forest semantic hashing scheme that embeds tiny convolutional neural networks (CNN) into shallow random forests, with near-optimal information-theoretic code aggregation among trees. We s…
Assessing heterogeneous treatment effects has become a growing interest in advancing precision medicine. Individualized treatment effects (ITE) play a critical role in such an endeavor. Concerning experimental data collected from randomized trials, we put forward a method, termed random forests of interaction trees (RF…
Infinitesimal gradient boosting is a new algorithm derived from gradient boosting.
problem Improving the efficiency and smoothness of gradient boosting.
method Introduced a new class of randomized regression trees and used a limit process in vanishing-learning-rate asymptotic.
result Convergence of the stochastic algorithm and characterization of the limiting procedure as a unique solution of a nonlinear ODE.
Decision trees can be biased towards minority class, contrary to belief.
problem Bias in decision trees towards minority class in imbalanced datasets.
method Critical evaluation of past literature, specific conditions analysis, tree-fitting adjustments, and post-hoc calibration methods.
result Decision trees can be biased towards minority class under specific conditions, not always towards majority.
New random forest algorithms for PU learning minimize risk directly.
problem Learning from positive and unlabeled data.
method Recursive greedy risk minimization for decision trees.
result Efficient PU random forest algorithm with robustness and low hyperparameter tuning.
Tree-based models biased when trained on imbalanced data, requiring new calibration methods.
problem Bias in tree-based models trained on imbalanced datasets.
method Analytical calibration of random forest models, demonstrating bias in decision trees.
result Calibrating tree-based models on imbalanced data negatively impacts predictions, especially for the minority class.
We prove that for the harmonic measure associated to a random walk on Out(Fr) satisfying some mild conditions, a typical tree in the boundary of Outer space is trivalent and nongeometric. This answers a question of M. Bestvina.
We characterize and study variable importance (VIMP) and pairwise variable associations in binary regression trees. A key component involves the node mean squared error for a quantity we refer to as a maximal subtree. The theory naturally extends from single trees to ensembles of trees and applies to methods like rando…
The number of BMW groups on tree products is bounded.
problem Counting and understanding BMW groups on tree products.
method Introduced a random model and used group theory.
result Bounds on the number of commensurability classes of BMW groups.