Tree-based model averaging improves CATE estimation from diverse sites.
problem Limited sample size and privacy concerns prevent accurate personalized treatment effect estimation.
method Tree-based model averaging approach to estimate CATEs from multiple heterogeneous sites.
result Improved accuracy in estimating conditional average treatment effects (CATEs) across sites.
A new tree-based estimator, FastPD, efficiently estimates PD functions for machine learning models.
problem Efficiently estimating Partial Dependence functions for machine learning models.
method Proposes a new tree-based estimator, FastPD, to estimate PD functions.
result FastPD consistently estimates the desired population quantity and improves complexity from quadratic to linear.
LassoFlexNet improves deep learning performance on tabular data.
problem Deep learning underperforms tree-based models on tabular data.
method Incorporates five inductive biases and uses Tied Group Lasso for variable selection.
result LassoFlexNet matches or outperforms leading tree-based models on 52 datasets.
Local MDI+ improves feature importance for tree-based models, enhancing interpretability and performance.
problem Lack of instance-specific feature importance for tree-based models.
method Local MDI+ extends MDI+ framework to provide instance-specific feature importances.
result Local MDI+ outperforms existing baselines, improving predictive performance by 10%.
Partition Tree estimates conditional densities for mixed continuous and categorical variables.
problem Estimating conditional densities for mixed data types.
method Tree-based framework modeling conditional distributions as piecewise-constant densities on adaptive partitions, minimizing conditional negative log-likelihood.
result Improved probabilistic prediction compared to CART-style trees and state-of-the-art methods.
New trees-based models handle correlated data better.
problem Standard trees-based models ignore correlation structure.
method Explicitly accounts for correlation structure in splitting criterion, stopping rules, and fitted values.
result New approach superior to standard models in simulations and real data.
New method improves feature selection in tree-based models.
problem Previous feature selection methods in tree-based models lack sufficient regularization and sub-optimal performance.
method Developed a new gain penalization approach for tree-based models that allows for flexible feature-specific importance weights.
result The new method improves out-of-sample performance, especially with correlated features.
Study reviews tree-based methods and introduces new ensemble strategies.
problem Improving the efficiency and performance of tree-based machine learning models.
method Review of tree-based methods, introduction of ISLE framework, ARM model combination strategy, and modified ISLEs.
result Performance evaluation of modified ISLEs on real data sets.
The paper explores features from orderbooks to improve intraday electricity price forecasting.
problem Improving probabilistic forecasting of intraday electricity prices.
method Extracted 384 features from orderbooks, selected powerful features, and benchmarked models across two countries and product types.
result Revealed an asymmetric generalization phenomenon in electricity price forecasting models.
In short, our experiments suggest that yes, on average, rotation forest is better than the most common alternatives when all the attributes are real-valued. Rotation forest is a tree based ensemble that performs transforms on subsets of attributes prior to constructing each tree. We present an empirical comparison of c…
Tree-based models outperform deep learning on tabular data, especially for medium-sized datasets.
problem Understanding why tree-based models outperform deep learning on tabular data.
method Extensive benchmarks of tree-based and deep learning models on 45 datasets, accounting for hyperparameters.
result Tree-based models remain state-of-the-art on medium-sized tabular data, even without hyperparameter optimization.
Visualizes classification accuracy and label bias in neural nets and trees.
problem Identifying mislabeled cases in neural net and tree-based classifications.
method Silhouette plots and quasi residual plots of PAC (probability of alternative class).
result Comparison of different classifications using silhouette width and PAC plots.
Simulation study evaluates tree-based imputation methods for multi-level data.
problem Ignoring dependencies in hierarchical data can compromise imputation accuracy.
method Chained Random Forests and Extreme Gradient Boosting (mixgb) adapted for multi-level data.
result Adapted boosting methods outperform traditional MICE for Level-1 variables at higher missingness rates.
C-XGBoost estimates causal effects from observational data.
problem Estimating causal effects from observational data.
method Proposes C-XGBoost, a tree boosting model for causal effect estimation.
result Demonstrates effectiveness through performance profiles and statistical tests.
A new tree-based model for varying coefficients using CGBM.
problem Modeling varying coefficients with high dimensionality and complex interactions.
method Tree-based varying coefficient model with CGBM for varying coefficients, dimension-wise early stopping, and feature importance scores.
result The model produces comparable out-of-sample loss to neural networks, demonstrating effectiveness.
A novel tree algorithm improves time series forecasting accuracy.
problem Improving accuracy in non-linear time series forecasting.
method Developed a hierarchical TAR model as a regression tree that trains globally across series, introducing a forecasting-specific tree algorithm with cross-series learning.
result Significantly higher accuracy than state-of-the-art tree-based algorithms and benchmarks across four metrics.
Improved Shapley Values for tree-based models, more accurate than existing methods.
problem Inaccurate Shapley Values in tree-based models leading to poor explanations.
method Introduced two new estimators exploiting tree structure, derived correct approach for categorical variables.
result More accurate Shapley Values for tree-based models, demonstrated through simulations.
Develops fair feature importance scores for tree-based models to interpret fairness.
problem Ensuring fairness in machine learning models, especially tree-based ones.
method Inspired by decision trees, proposes a novel fair feature importance score based on mean decrease in group bias.
result Valid interpretations of fairness for tree-based ensembles and surrogates of other ML systems.
The main goal of this paper is to study the geometric structures associated with the representation of tensors in subspace based formats. To do this we use a property of the so-called minimal subspaces which allows us to describe the tensor representation by means of a rooted tree. By using the tree structure and the d…
FREEtree improves tree-based methods for correlated longitudinal data.
problem Poor performance of Random Forests in high dimensional longitudinal data with correlated features.
method FREEtree uses a piecewise random effects model and clustering with WGCNA to select features and maintain interpretability.
result FREEtree outperforms other tree-based methods in prediction and feature selection accuracy.
A new approach uses partial likelihood to improve tree-based density estimation and inference.
problem Inference on tree-based models suffers from overfitting and reduced efficiency due to data-independent partitioning.
method Proposes a partial likelihood approach to data-dependent partitioning of tree-based models.
result Significant gains in estimation accuracy and computational efficiency from adopting partial likelihood.
Enhanced tree-based classifiers use derivatives and geometry for better function classification.
problem Improving classification of high-dimensional time series data.
method Integrates Functional Data Analysis with tree-based ensemble techniques, leveraging derivative and geometric features.
result Significant improvements over traditional approaches in function classification.
A new tree-based model improves uncertainty estimation in sequential optimization.
problem Improving uncertainty estimation in sequential model-based optimization.
method Proposed a new ensemble of randomized trees (BwO forest) with bagging and oversampling.
result BwO forest outperforms existing tree-based models in various optimization scenarios.
In aircraft industry, market needs evolve quickly in a high competitiveness context. This requires adapting a given aircraft model in minimum time considering for example an increase of range or the number of passengers (cf A330 NEO family). The computation of loads and stress to resize the airframe is on the critical …
nTreeClus clusters categorical sequences using tree-based learners and k-mers.
problem Challenges in clustering categorical and sequential data.
method nTreeClus uses Tree-based Learners, k-mers, and autoregressive models for categorical time series.
result nTreeClus outperformed baseline methods in various validation metrics.
TreeDOX predicts chaotic systems without hyperparameter tuning.
problem Forecasting chaotic systems requires hyperparameter tuning, limiting adoption.
method TreeDOX uses time delay overembedding and Extra-Trees Regressors.
result TreeDOX achieves state-of-the-art performance on chaotic systems.
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
Time series analysis is a key component of machine learning, with applications in various fields.
problem Time series analysis in machine learning
method Basic concepts, classical statistical models, modern machine learning approaches
result Machine learning techniques for time series analysis
Machine learning models classify celestial objects like pulsars and black holes.
problem Classifying high-energy celestial objects using photometric data.
method Applied tree-based models and RNN to classify pulsars and black holes.
result RNN showed potential for real-time object discrimination and classification.
Ensembling improves performance when classifiers disagree more than average.
problem When do ensembles provide significant performance improvements in classification tasks?
method Theoretical and empirical analysis of ensemble improvement rate and disagreement-error ratio.
result Ensembling improves performance significantly when the disagreement rate is large relative to the average error rate.
Extends geometrical description of tensor manifolds in tree-based formats.
problem Geometrical description of tensor manifolds in tree-based formats.
method Provided a new geometrical description of manifolds of tensors in tree-based format.
result Geometrical description compatible with Tucker format.
We introduce an efficient method for training the linear ranking support vector machine. The method combines cutting plane optimization with red-black tree based approach to subgradient calculations, and has O(m*s+m*log(m)) time complexity, where m is the number of training examples, and s the average number of non-zer…
Tree-based models biased when trained on imbalanced data, requiring new calibration methods.
problem Bias in tree-based models trained on imbalanced datasets.
method Analytical calibration of random forest models, demonstrating bias in decision trees.
result Calibrating tree-based models on imbalanced data negatively impacts predictions, especially for the minority class.
New research finds tree-based GAMs are most trustworthy and fair.
problem Variability in GAM algorithms leads to inconsistent models.
method Quantitative and qualitative analysis of various GAM algorithms.
result Tree-based GAMs are the most trustworthy and fair.
Novel unsupervised random forests improve density estimation and data synthesis.
problem Density estimation and data synthesis for complex tabular data.
method Recursive unsupervised random forests with alternating generation and discrimination rounds.
result Provable consistency and smooth densities with fast execution.
This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse arc…
SCORE improves tree-based predictions with boosted residual extraTrees.
problem Improving tree-based prediction models with reduced errors.
method Inspired by representation learning, SCORE uses boosting, regularized regression, and variable selection.
result SCORE provides comparable or superior performance compared to other models.
The paper predicts travel times using tree-based ensembles.
problem Predicting travel times between urban points over short and long horizons.
method Tree-based ensemble methods trained on taxi trip records with additional features from weather and routing data.
result Adding routing data improves model performance and short-term predictions require less data.
The paper introduces a method to control false splits in tree-based data aggregation.
problem Identifying the correct subgroups to treat as a single entity in tree-based data.
method Introduces the 'false split rate' and proposes a multiple hypothesis testing algorithm for tree-based aggregation.
result The proposed algorithm controls the false split rate, demonstrating its effectiveness on stock volatility and taxi fare data.
Automated prediction of public speaking performance enables novel systems for tutoring public speaking skills. We use the largest open repository---TED Talks---to predict the ratings provided by the online viewers. The dataset contains over 2200 talk transcripts and the associated meta information including over 5.5 mi…
Study compares tree-based imputation methods to MICE PMM for missing data.
problem Missing data in empirical studies.
method Various imputation methods including MICE PMM, RF, missRanger, and MIXGBoost.
result Tree-based imputations, especially RF and missRanger with PMM, perform better.
We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density …
We present the first tree-based regressor whose convergence rate depends only on the intrinsic dimension of the data, namely its Assouad dimension. The regressor uses the RPtree partitioning procedure, a simple randomized variant of k-d trees.
Tree-based synthesis improves forecast accuracy in GDP and inflation.
problem Improving forecast accuracy in GDP and inflation.
method Developed a nonparametric synthesis function using regression trees.
result Tree-based synthesis leads to improved forecast accuracy.
Study optimizes tree-based models for better alignment of predicted scores and actual probabilities.
problem Traditional calibration metrics fail to align predicted scores with actual probabilities when score distributions deviate from the underlying data.
method Optimizes tree-based models (Random Forest, XGBoost) using Kullback-Leibler (KL) divergence to minimize the difference between predicted and true probability distributions.
result Optimized tree-based models yield superior alignment between predicted scores and actual probabilities without significant performance loss.
Decision trees improve performance in various fields.
problem Improving performance in machine learning and data mining.
method Introduce tree-based models and training techniques.
result Highlight probabilistic characteristics and practical benefits.
Optimal Survival Trees improve accuracy in medical data analysis.
problem Analyzing censored outcomes in medical data.
method Mixed-integer optimization and local search techniques.
result Improves accuracy in large datasets compared to existing methods.
This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.
problem Understanding the effectiveness of tree-based methods in finite-sample settings, especially symbolic feature selection.
method Local ranking perspective, finite-sample analysis, oracle bounds, posterior contraction results, concordant divergence statistics.
result New insights and statistics for evaluating symbolic feature mappings.