The paper introduces a method to control false splits in tree-based data aggregation.
problem Identifying the correct subgroups to treat as a single entity in tree-based data.
method Introduces the 'false split rate' and proposes a multiple hypothesis testing algorithm for tree-based aggregation.
result The proposed algorithm controls the false split rate, demonstrating its effectiveness on stock volatility and taxi fare data.
New method aggregates nodes in sparse graphical models.
problem Estimating edge-sparse graphical models.
method Tree-aggregated graphical lasso (tag-lasso) method.
result Aggregates nodes in a data-driven fashion using a tree.
In this work, we consider to improve the model estimation efficiency by aggregating the neighbors' information as well as identify the subgroup membership for each node in the network. A tree-based l1 penalty is proposed to save the computation and communication cost. We design a decentralized generalized alternatin…
Paper addresses robust federated linear bandits against Byzantine attacks.
problem Byzantine attacks on a small fraction of agents in federated learning.
method Proposes a geometric median-based robust aggregation oracle.
result Achieves sublinear regret bound of ildeO(T3/4) robust to fewer than half Byzantine agents. New trees-based models handle correlated data better.
problem Standard trees-based models ignore correlation structure.
method Explicitly accounts for correlation structure in splitting criterion, stopping rules, and fitted values.
result New approach superior to standard models in simulations and real data.
Study reviews tree-based methods and introduces new ensemble strategies.
problem Improving the efficiency and performance of tree-based machine learning models.
method Review of tree-based methods, introduction of ISLE framework, ARM model combination strategy, and modified ISLEs.
result Performance evaluation of modified ISLEs on real data sets.
The main goal of this paper is to study the geometric structures associated with the representation of tensors in subspace based formats. To do this we use a property of the so-called minimal subspaces which allows us to describe the tensor representation by means of a rooted tree. By using the tree structure and the d…
Efficient method classifies locally stationary time series based on second-order characteristics.
problem Classifying locally stationary time series for various applications.
method Autoregressive approximation, ensemble aggregation, distance-based threshold.
result Zero misclassification error rate asymptotically for mildly differing second-order characteristics.
New method improves feature selection in tree-based models.
problem Previous feature selection methods in tree-based models lack sufficient regularization and sub-optimal performance.
method Developed a new gain penalization approach for tree-based models that allows for flexible feature-specific importance weights.
result The new method improves out-of-sample performance, especially with correlated features.
Enhanced tree-based classifiers use derivatives and geometry for better function classification.
problem Improving classification of high-dimensional time series data.
method Integrates Functional Data Analysis with tree-based ensemble techniques, leveraging derivative and geometric features.
result Significant improvements over traditional approaches in function classification.
Simulation study evaluates tree-based imputation methods for multi-level data.
problem Ignoring dependencies in hierarchical data can compromise imputation accuracy.
method Chained Random Forests and Extreme Gradient Boosting (mixgb) adapted for multi-level data.
result Adapted boosting methods outperform traditional MICE for Level-1 variables at higher missingness rates.
Extends geometrical description of tensor manifolds in tree-based formats.
problem Geometrical description of tensor manifolds in tree-based formats.
method Provided a new geometrical description of manifolds of tensors in tree-based format.
result Geometrical description compatible with Tucker format.
TQ separates sampling and integration for high-dimensional integrals.
problem High-dimensional integration challenges in science.
method Tree Quadrature (TQ) constructs a surrogate model using regression trees.
result TQ outperforms existing methods in up to 15 dimensions.
Study forecasts vegetable prices in Nepal using a novel index and ensemble model.
problem High volatility and cultural influences on agricultural commodity prices.
method Developed KVPI, created features, evaluated multiple models, introduced Momentum-Corrected Online Stacking Ensemble.
result Achieved RMSE of 1.771, MAPE of 0.68%, and R-squared of 0.845 at 90-day horizon.
Develops RKHS framework for analyzing tree ensembles.
problem Analyzing the theoretical properties of tree ensembles.
method Reproducing Kernel Hilbert Spaces (RKHS) for tree ensembles.
result Characterizes Random Forest predictor as minimizer of a penalized empirical risk functional in RKHS.
Tree-based models outperform deep learning on tabular data, especially for medium-sized datasets.
problem Understanding why tree-based models outperform deep learning on tabular data.
method Extensive benchmarks of tree-based and deep learning models on 45 datasets, accounting for hyperparameters.
result Tree-based models remain state-of-the-art on medium-sized tabular data, even without hyperparameter optimization.
Develops fair feature importance scores for tree-based models to interpret fairness.
problem Ensuring fairness in machine learning models, especially tree-based ones.
method Inspired by decision trees, proposes a novel fair feature importance score based on mean decrease in group bias.
result Valid interpretations of fairness for tree-based ensembles and surrogates of other ML systems.
We present the first tree-based regressor whose convergence rate depends only on the intrinsic dimension of the data, namely its Assouad dimension. The regressor uses the RPtree partitioning procedure, a simple randomized variant of k-d trees.
FREEtree improves tree-based methods for correlated longitudinal data.
problem Poor performance of Random Forests in high dimensional longitudinal data with correlated features.
method FREEtree uses a piecewise random effects model and clustering with WGCNA to select features and maintain interpretability.
result FREEtree outperforms other tree-based methods in prediction and feature selection accuracy.
Tree-based model averaging improves CATE estimation from diverse sites.
problem Limited sample size and privacy concerns prevent accurate personalized treatment effect estimation.
method Tree-based model averaging approach to estimate CATEs from multiple heterogeneous sites.
result Improved accuracy in estimating conditional average treatment effects (CATEs) across sites.
Improved Shapley Values for tree-based models, more accurate than existing methods.
problem Inaccurate Shapley Values in tree-based models leading to poor explanations.
method Introduced two new estimators exploiting tree structure, derived correct approach for categorical variables.
result More accurate Shapley Values for tree-based models, demonstrated through simulations.
A new tree-based model for varying coefficients using CGBM.
problem Modeling varying coefficients with high dimensionality and complex interactions.
method Tree-based varying coefficient model with CGBM for varying coefficients, dimension-wise early stopping, and feature importance scores.
result The model produces comparable out-of-sample loss to neural networks, demonstrating effectiveness.
This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.
problem Understanding the effectiveness of tree-based methods in finite-sample settings, especially symbolic feature selection.
method Local ranking perspective, finite-sample analysis, oracle bounds, posterior contraction results, concordant divergence statistics.
result New insights and statistics for evaluating symbolic feature mappings.
TreeDOX predicts chaotic systems without hyperparameter tuning.
problem Forecasting chaotic systems requires hyperparameter tuning, limiting adoption.
method TreeDOX uses time delay overembedding and Extra-Trees Regressors.
result TreeDOX achieves state-of-the-art performance on chaotic systems.
A new approach uses partial likelihood to improve tree-based density estimation and inference.
problem Inference on tree-based models suffers from overfitting and reduced efficiency due to data-independent partitioning.
method Proposes a partial likelihood approach to data-dependent partitioning of tree-based models.
result Significant gains in estimation accuracy and computational efficiency from adopting partial likelihood.
Study compares tree-based imputation methods to MICE PMM for missing data.
problem Missing data in empirical studies.
method Various imputation methods including MICE PMM, RF, missRanger, and MIXGBoost.
result Tree-based imputations, especially RF and missRanger with PMM, perform better.
TQF models multivariate uncertainty by learning conditional quantiles.
problem Challenges in fully nonparametric estimation of multivariate conditional distributions.
method Tomographic Quantile Forests (TQF) learns conditional quantiles of directional projections.
result TQF reconstructs multivariate conditional distribution efficiently without convexity restrictions.
SCORE improves tree-based predictions with boosted residual extraTrees.
problem Improving tree-based prediction models with reduced errors.
method Inspired by representation learning, SCORE uses boosting, regularized regression, and variable selection.
result SCORE provides comparable or superior performance compared to other models.
A new tree-based model improves uncertainty estimation in sequential optimization.
problem Improving uncertainty estimation in sequential model-based optimization.
method Proposed a new ensemble of randomized trees (BwO forest) with bagging and oversampling.
result BwO forest outperforms existing tree-based models in various optimization scenarios.
nTreeClus clusters categorical sequences using tree-based learners and k-mers.
problem Challenges in clustering categorical and sequential data.
method nTreeClus uses Tree-based Learners, k-mers, and autoregressive models for categorical time series.
result nTreeClus outperformed baseline methods in various validation metrics.
We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as…
Code generation maps a program description to executable source code in a programming language. Existing approaches mainly rely on a recurrent neural network (RNN) as the decoder. However, we find that a program contains significantly more tokens than a natural language sentence, and thus it may be inappropriate for RN…
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
Online news media provides aggregated news and stories from different sources all over the world and up-to-date news coverage. The main goal of this study is to have a solution that considered as a homogeneous source for the news and to represent the news in a new conceptual framework. Furthermore, the user can easily …
The paper predicts travel times using tree-based ensembles.
problem Predicting travel times between urban points over short and long horizons.
method Tree-based ensemble methods trained on taxi trip records with additional features from weather and routing data.
result Adding routing data improves model performance and short-term predictions require less data.
Machine learning models classify celestial objects like pulsars and black holes.
problem Classifying high-energy celestial objects using photometric data.
method Applied tree-based models and RNN to classify pulsars and black holes.
result RNN showed potential for real-time object discrimination and classification.
Automated classification of metadata of research data by their discipline(s) of research can be used in scientometric research, by repository service providers, and in the context of research data aggregation services. Openly available metadata of the DataCite index for research data were used to compile a large traini…
New research finds tree-based GAMs are most trustworthy and fair.
problem Variability in GAM algorithms leads to inconsistent models.
method Quantitative and qualitative analysis of various GAM algorithms.
result Tree-based GAMs are the most trustworthy and fair.
Tree-based models biased when trained on imbalanced data, requiring new calibration methods.
problem Bias in tree-based models trained on imbalanced datasets.
method Analytical calibration of random forest models, demonstrating bias in decision trees.
result Calibrating tree-based models on imbalanced data negatively impacts predictions, especially for the minority class.
This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse arc…
Tree-based synthesis improves forecast accuracy in GDP and inflation.
problem Improving forecast accuracy in GDP and inflation.
method Developed a nonparametric synthesis function using regression trees.
result Tree-based synthesis leads to improved forecast accuracy.
Local MDI+ improves feature importance for tree-based models, enhancing interpretability and performance.
problem Lack of instance-specific feature importance for tree-based models.
method Local MDI+ extends MDI+ framework to provide instance-specific feature importances.
result Local MDI+ outperforms existing baselines, improving predictive performance by 10%.
Decision trees improve performance in various fields.
problem Improving performance in machine learning and data mining.
method Introduce tree-based models and training techniques.
result Highlight probabilistic characteristics and practical benefits.
Enhances functional classifier performance with new tree-based methods and unbiased feature importance assessment.
problem Challenges of high-dimensional functional data and biased feature importance assessment.
method Augmented functional classification trees and random forests with ad-hoc conditional permutations for unbiased feature importance.
result Significant enhancement in predictive power of functional classifiers through new feature importance assessment.
A theory for approximating complex concepts with simple decision trees.
problem Approximating complex concepts with simple decision trees.
method Introducing interpretable approximations, studying binary concept approximation by decision trees.
result A trichotomy of cases for approximating a binary concept by decision trees based on a simple class.
We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density …
K-nearest neighbor (kNN) search has wide applications in many areas, including data mining, machine learning, statistics and many applied domains. Inspired by the success of ensemble methods and the flexibility of tree-based methodology, we propose random projection forests (rpForests), for kNN search. rpForests finds …
Study optimizes tree-based models for better alignment of predicted scores and actual probabilities.
problem Traditional calibration metrics fail to align predicted scores with actual probabilities when score distributions deviate from the underlying data.
method Optimizes tree-based models (Random Forest, XGBoost) using Kullback-Leibler (KL) divergence to minimize the difference between predicted and true probability distributions.
result Optimized tree-based models yield superior alignment between predicted scores and actual probabilities without significant performance loss.