New trees-based models handle correlated data better.
problem Standard trees-based models ignore correlation structure.
method Explicitly accounts for correlation structure in splitting criterion, stopping rules, and fitted values.
result New approach superior to standard models in simulations and real data.
New method improves feature selection in tree-based models.
problem Previous feature selection methods in tree-based models lack sufficient regularization and sub-optimal performance.
method Developed a new gain penalization approach for tree-based models that allows for flexible feature-specific importance weights.
result The new method improves out-of-sample performance, especially with correlated features.
Study reviews tree-based methods and introduces new ensemble strategies.
problem Improving the efficiency and performance of tree-based machine learning models.
method Review of tree-based methods, introduction of ISLE framework, ARM model combination strategy, and modified ISLEs.
result Performance evaluation of modified ISLEs on real data sets.
Tree-based models outperform deep learning on tabular data, especially for medium-sized datasets.
problem Understanding why tree-based models outperform deep learning on tabular data.
method Extensive benchmarks of tree-based and deep learning models on 45 datasets, accounting for hyperparameters.
result Tree-based models remain state-of-the-art on medium-sized tabular data, even without hyperparameter optimization.
Simulation study evaluates tree-based imputation methods for multi-level data.
problem Ignoring dependencies in hierarchical data can compromise imputation accuracy.
method Chained Random Forests and Extreme Gradient Boosting (mixgb) adapted for multi-level data.
result Adapted boosting methods outperform traditional MICE for Level-1 variables at higher missingness rates.
Tree-based model averaging improves CATE estimation from diverse sites.
problem Limited sample size and privacy concerns prevent accurate personalized treatment effect estimation.
method Tree-based model averaging approach to estimate CATEs from multiple heterogeneous sites.
result Improved accuracy in estimating conditional average treatment effects (CATEs) across sites.
A new tree-based model for varying coefficients using CGBM.
problem Modeling varying coefficients with high dimensionality and complex interactions.
method Tree-based varying coefficient model with CGBM for varying coefficients, dimension-wise early stopping, and feature importance scores.
result The model produces comparable out-of-sample loss to neural networks, demonstrating effectiveness.
Improved Shapley Values for tree-based models, more accurate than existing methods.
problem Inaccurate Shapley Values in tree-based models leading to poor explanations.
method Introduced two new estimators exploiting tree structure, derived correct approach for categorical variables.
result More accurate Shapley Values for tree-based models, demonstrated through simulations.
Develops fair feature importance scores for tree-based models to interpret fairness.
problem Ensuring fairness in machine learning models, especially tree-based ones.
method Inspired by decision trees, proposes a novel fair feature importance score based on mean decrease in group bias.
result Valid interpretations of fairness for tree-based ensembles and surrogates of other ML systems.
The main goal of this paper is to study the geometric structures associated with the representation of tensors in subspace based formats. To do this we use a property of the so-called minimal subspaces which allows us to describe the tensor representation by means of a rooted tree. By using the tree structure and the d…
FREEtree improves tree-based methods for correlated longitudinal data.
problem Poor performance of Random Forests in high dimensional longitudinal data with correlated features.
method FREEtree uses a piecewise random effects model and clustering with WGCNA to select features and maintain interpretability.
result FREEtree outperforms other tree-based methods in prediction and feature selection accuracy.
A new approach uses partial likelihood to improve tree-based density estimation and inference.
problem Inference on tree-based models suffers from overfitting and reduced efficiency due to data-independent partitioning.
method Proposes a partial likelihood approach to data-dependent partitioning of tree-based models.
result Significant gains in estimation accuracy and computational efficiency from adopting partial likelihood.
Enhanced tree-based classifiers use derivatives and geometry for better function classification.
problem Improving classification of high-dimensional time series data.
method Integrates Functional Data Analysis with tree-based ensemble techniques, leveraging derivative and geometric features.
result Significant improvements over traditional approaches in function classification.
A new tree-based model improves uncertainty estimation in sequential optimization.
problem Improving uncertainty estimation in sequential model-based optimization.
method Proposed a new ensemble of randomized trees (BwO forest) with bagging and oversampling.
result BwO forest outperforms existing tree-based models in various optimization scenarios.
nTreeClus clusters categorical sequences using tree-based learners and k-mers.
problem Challenges in clustering categorical and sequential data.
method nTreeClus uses Tree-based Learners, k-mers, and autoregressive models for categorical time series.
result nTreeClus outperformed baseline methods in various validation metrics.
TreeDOX predicts chaotic systems without hyperparameter tuning.
problem Forecasting chaotic systems requires hyperparameter tuning, limiting adoption.
method TreeDOX uses time delay overembedding and Extra-Trees Regressors.
result TreeDOX achieves state-of-the-art performance on chaotic systems.
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
Machine learning models classify celestial objects like pulsars and black holes.
problem Classifying high-energy celestial objects using photometric data.
method Applied tree-based models and RNN to classify pulsars and black holes.
result RNN showed potential for real-time object discrimination and classification.
Extends geometrical description of tensor manifolds in tree-based formats.
problem Geometrical description of tensor manifolds in tree-based formats.
method Provided a new geometrical description of manifolds of tensors in tree-based format.
result Geometrical description compatible with Tucker format.
Tree-based models biased when trained on imbalanced data, requiring new calibration methods.
problem Bias in tree-based models trained on imbalanced datasets.
method Analytical calibration of random forest models, demonstrating bias in decision trees.
result Calibrating tree-based models on imbalanced data negatively impacts predictions, especially for the minority class.
New research finds tree-based GAMs are most trustworthy and fair.
problem Variability in GAM algorithms leads to inconsistent models.
method Quantitative and qualitative analysis of various GAM algorithms.
result Tree-based GAMs are the most trustworthy and fair.
This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse arc…
SCORE improves tree-based predictions with boosted residual extraTrees.
problem Improving tree-based prediction models with reduced errors.
method Inspired by representation learning, SCORE uses boosting, regularized regression, and variable selection.
result SCORE provides comparable or superior performance compared to other models.
The paper predicts travel times using tree-based ensembles.
problem Predicting travel times between urban points over short and long horizons.
method Tree-based ensemble methods trained on taxi trip records with additional features from weather and routing data.
result Adding routing data improves model performance and short-term predictions require less data.
The paper introduces a method to control false splits in tree-based data aggregation.
problem Identifying the correct subgroups to treat as a single entity in tree-based data.
method Introduces the 'false split rate' and proposes a multiple hypothesis testing algorithm for tree-based aggregation.
result The proposed algorithm controls the false split rate, demonstrating its effectiveness on stock volatility and taxi fare data.
Local MDI+ improves feature importance for tree-based models, enhancing interpretability and performance.
problem Lack of instance-specific feature importance for tree-based models.
method Local MDI+ extends MDI+ framework to provide instance-specific feature importances.
result Local MDI+ outperforms existing baselines, improving predictive performance by 10%.
Study compares tree-based imputation methods to MICE PMM for missing data.
problem Missing data in empirical studies.
method Various imputation methods including MICE PMM, RF, missRanger, and MIXGBoost.
result Tree-based imputations, especially RF and missRanger with PMM, perform better.
We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density …
We present the first tree-based regressor whose convergence rate depends only on the intrinsic dimension of the data, namely its Assouad dimension. The regressor uses the RPtree partitioning procedure, a simple randomized variant of k-d trees.
Tree-based synthesis improves forecast accuracy in GDP and inflation.
problem Improving forecast accuracy in GDP and inflation.
method Developed a nonparametric synthesis function using regression trees.
result Tree-based synthesis leads to improved forecast accuracy.
Study optimizes tree-based models for better alignment of predicted scores and actual probabilities.
problem Traditional calibration metrics fail to align predicted scores with actual probabilities when score distributions deviate from the underlying data.
method Optimizes tree-based models (Random Forest, XGBoost) using Kullback-Leibler (KL) divergence to minimize the difference between predicted and true probability distributions.
result Optimized tree-based models yield superior alignment between predicted scores and actual probabilities without significant performance loss.
Decision trees improve performance in various fields.
problem Improving performance in machine learning and data mining.
method Introduce tree-based models and training techniques.
result Highlight probabilistic characteristics and practical benefits.
Optimal Survival Trees improve accuracy in medical data analysis.
problem Analyzing censored outcomes in medical data.
method Mixed-integer optimization and local search techniques.
result Improves accuracy in large datasets compared to existing methods.
This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.
problem Understanding the effectiveness of tree-based methods in finite-sample settings, especially symbolic feature selection.
method Local ranking perspective, finite-sample analysis, oracle bounds, posterior contraction results, concordant divergence statistics.
result New insights and statistics for evaluating symbolic feature mappings.
Paper proposes using tree-based surrogate models for efficient Shapley computation.
problem Efficient computation of Shapley values using conditional expectations.
method Surrogate model-based tree for approximating Shapley and SHAP values.
result The proposed algorithm improves accuracy and unifies global interpretation.
HS improves tree-based models' accuracy and interpretability without changing their structure.
problem Overfitting in tree-based models.
method Hierarchical Shrinkage (HS) post-hoc algorithm that shrinks tree predictions towards ancestor means.
result HS significantly improves predictive performance and interpretability of decision trees and RFs.
LassoFlexNet improves deep learning performance on tabular data.
problem Deep learning underperforms tree-based models on tabular data.
method Incorporates five inductive biases and uses Tied Group Lasso for variable selection.
result LassoFlexNet matches or outperforms leading tree-based models on 52 datasets.
Unified view of improving tree model interpretability and debiasing feature importance.
problem Improving interpretability and debiasing feature importance in tree-based models.
method Demonstrates a common thread among bias correction methods and local explanations for trees.
result Points out a bias in explainable AI for trees algorithms due to inbag data inclusion.
In this work, we consider to improve the model estimation efficiency by aggregating the neighbors' information as well as identify the subgroup membership for each node in the network. A tree-based l1 penalty is proposed to save the computation and communication cost. We design a decentralized generalized alternatin…
Optimal survival trees ensemble reduces tree count and improves predictive performance.
problem Improving predictive performance in survival analysis.
method Grows a forest of optimal survival trees by ranking and selecting the best trees based on out-of-bag error.
result Reduces the number of trees in the ensemble while improving predictive performance.
Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised…
We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as…
Proposes a generalized causal tree for handling multiple treatments in uplift modeling.
problem Handling multiple treatments in uplift modeling.
method Generalizes causal tree algorithm to handle multiple discrete and continuous-valued treatments.
result Demonstrates improved performance over existing methods in experiments and real data examples.
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-bas…
P-Trees improve investment performance by optimizing the efficient frontier.
problem Optimizing investment performance in complex financial markets.
method Introducing P-Trees, a new tree-based model for analyzing panel data.
result P-Trees significantly advance the efficient frontier and outperform existing models.
We present a method for incorporating missing data in non-parametric statistical learning without the need for imputation. We focus on a tree-based method, Bayesian Additive Regression Trees (BART), enhanced with "Missingness Incorporated in Attributes," an approach recently proposed incorporating missingness into deci…
Develops monotone tree-based GAMI models using XGBoost.
problem Incorporating monotonicity into GAMI models based on boosted trees.
method Adapting XGBoost algorithm to fit monotone GAMI-Tree models, filtering interactions, and parsing results.
result Fits monotone GAMI-Tree models that are directly interpretable and visualizable.