PILOT is a fast algorithm for linear model trees that outperforms existing methods.
problem Fitting linear model trees to large datasets efficiently and accurately.
method Greedy training with L2 boosting and model selection rule. result PILOT outperforms standard decision trees and other linear model trees on various datasets.
A new decision tree variant improves linear model performance.
problem Improving decision tree performance on non-linear data.
method Extremely random tree with non-linear data transformation and linear observer.
result Outperforms linear models on benchmark dataset.
BART and MOTR-BART improve tree-based predictions with local linear models.
problem Non-linearity and high-order interactions in data.
method Bayesian Additive Regression Trees (BART) and Model Trees BART (MOTR-BART) using piecewise linear functions.
result MOTR-BART achieves equal or better performance with fewer trees than BART.
A new tree-based model for multivariate responses interprets piecewise linear regimes.
problem Recovering piecewise multivariate linear regimes in complex data.
method Twoblock clustering trees with coskewness-based dimension reduction.
result Recovery of piecewise linear regimes in data.
GLMM trees identify subgroups with different growth patterns in longitudinal data.
problem Identifying subgroups with distinct growth trajectories in longitudinal studies.
method Extended GLMM trees for longitudinal data.
result Extended GLMM trees outperform other methods in accuracy and speed.
We present a detailed analysis of the class of regression decision tree algorithms which employ a regulized piecewise-linear node-splitting criterion and have regularized linear models at the leaves. From a theoretic standpoint, based on Rademacher complexity framework, we present new high-probability upper bounds for …
This paper develops a new method to model treatment effects that are heterogeneous across different quantiles.
problem Modeling treatment effects that vary across different quantiles of the outcome distribution.
method The paper combines quantile classification with local polynomial estimation to build a decision tree and forest.
result The proposed QLPRT and QLPRF methods provide a new way to estimate and infer heterogeneous treatment effects.
New method estimates root-directed tree from extreme data.
problem Discovering causality in river networks from extreme flow data.
method Qualitative max-linear Bayesian network approach to estimate bivariate scores and root-directed spanning tree.
result The new estimator is consistent under max-linear Bayesian network model with noise.
Enhanced ODT with Feature Concatenation boosts learning efficiency.
problem Insufficient learning efficiency of ODT due to linear projections not being transmitted to child nodes.
method Feature Concatenation ( exttt{FC-ODT}) to transmit linear projections along decision paths.
result Experiments show exttt{FC-ODT} outperforms state-of-the-art decision trees with a limited tree depth.
New method identifies causal parameters in tree-shaped linear models using cycles.
problem Identifying causal parameters from correlations in tree-shaped linear models.
method Investigates tree-shaped linear models, uses missing cycles to identify causal parameters, solves quadratic equations.
result Shows how missing cycles can be combined to obtain a unique solution for causal parameters.
LGB+ improves macroeconomic forecasting by combining linear and tree models.
problem Efficiency in small samples for forecasting with mixed linear and nonlinear dynamics.
method LGB+ is a boosting procedure that evaluates both tree and linear candidates at each step, advancing only the winner. It decomposes forecasts into linear and nonlinear contributions.
result LGB+ delivers strong gains for targets with pronounced autoregressive dynamics or mixed signals.
Researchers improve tree model recovery from noisy data.
problem Learning tree structured models from corrupted data.
method Linear latent tree models and continuous corruption model.
result Chow-Liu algorithm consistently learns tree from noisy data.
A novel tree algorithm improves time series forecasting accuracy.
problem Improving accuracy in non-linear time series forecasting.
method Developed a hierarchical TAR model as a regression tree that trains globally across series, introducing a forecasting-specific tree algorithm with cross-series learning.
result Significantly higher accuracy than state-of-the-art tree-based algorithms and benchmarks across four metrics.
Enhances GBDT robustness with one-hot encoding and regularization.
problem Low robustness of GBDT models against covariate perturbation.
method One-hot encoding to linear framework, risk decomposition, L1 or L2 regularization. result Regularization enhances GBDT robustness.
This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolutionary trees. The …
Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-bas…
Proposes extensions to semi-parametric models using BART for shared covariates.
problem Avoiding poor coverage properties and reducing bias in linear predictor estimates.
method Modifies tree-generation moves in BART to handle shared covariates between linear and non-parametric components.
result Competitive performance in modelling complex interactions and predicting student achievements.
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
Paper proposes a novel SVM method for creating survival trees.
problem Creating non-linear survival trees for right-censored data.
method L2-regularized dipole splitting criteria with kernel methods.
result Non-linear splits using polynomial and Gaussian kernels show similar predictive power but often smaller tree sizes.
Improves local model explanations using GANs and Linear Model Trees.
problem Need for accurate and intuitive explanations of complex machine learning models.
method Generative Adversarial Network (GAN) for synthetic data generation and Linear Model Trees for surrogate model training.
result Significantly improved local model explanations with contextual information.
We introduce block-tree graphs as a framework for deriving efficient algorithms on graphical models. We define block-tree graphs as a tree-structured graph where each node is a cluster of nodes such that the clusters in the graph are disjoint. This differs from junction-trees, where two clusters connected by an edge al…
Ensemble of regression trees have become popular statistical tools for the estimation of conditional mean given a set of predictors. However, quantile regression trees and their ensembles have not yet garnered much attention despite the increasing popularity of the linear quantile regression model. This work proposes a…
Paper simplifies complex sports analytics models for better understanding.
problem Difficulty in interpreting deep learning models for sports analytics.
method Developed a linear model tree that mimics deep learning models and explains its knowledge.
result The linear model tree achieves high accuracy and provides actionable insights.
BAVART model combines VAR and BART for non-linear forecasting.
problem Overly restrictive linearity assumption in VAR models.
method Combining VAR with Bayesian additive regression trees (BART).
result BAVART model yields highly competitive forecasts.
We study the robustness verification problem for tree-based models, including decision trees, random forests (RFs) and gradient boosted decision trees (GBDTs). Formal robustness verification of decision tree ensembles involves finding the exact minimal adversarial perturbation or a guaranteed lower bound of it. Existin…
Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these re…
Several classification methods assume that the underlying distributions follow tree-structured graphical models. Indeed, trees capture statistical dependencies between pairs of variables, which may be crucial to attain low classification errors. The resulting classifier is linear in the log-transformed univariate and b…
Paper proposes a new method for density estimation using tree tensor-network states.
problem Density estimation for complex graphical models with loops.
method Determines tree topology with Chow-Liu algorithm and uses sketching techniques to define tensor-network components.
result Sample complexity guarantees and empirical validation provided.
AGBoost uses attention weights to improve GBM for regression problems.
problem Improving gradient boosting machine for regression tasks.
method Attention-based modification of GBM with trainable attention weights.
result AGBoost achieves better performance on regression datasets.
This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tr…
Regression Trees analyze stock returns, revealing market excess return as the most informative factor.
problem Understanding informational content of three factors in stock returns.
method Joint regression tree analysis of daily stock return data for 5 major US corporations.
result The market excess return factor is always the most informative in all cases (solo and joint).
In this paper, we investigate adaptive nonlinear regression and introduce tree based piecewise linear regression algorithms that are highly efficient and provide significantly improved performance with guaranteed upper bounds in an individual sequence manner. We use a tree notion in order to partition the space of regr…
TRUST improves tree models' accuracy while maintaining interpretability.
problem Piecewise-constant regression trees lack in predictive accuracy compared to black-box models.
method Combines Random Forest accuracy with interpretability of shallow trees and sparsity of linear models, using LLMs for explanations.
result TRUST outperforms other interpretable models in predictive accuracy and matches Random Forest's accuracy.
Study finds a linear lower bound on conformal dimension for random hyperbolic groups.
problem Understanding conformal dimension in random hyperbolic groups.
method Building undistorted round trees from lower density groups.
result Achieves a linear lower bound in l at all densities 0<d<1/2. Estimates high-dimensional distributions using tree tensor networks.
problem Estimating high-dimensional probability distributions from i.i.d. samples.
method Tree-based tensor formats, empirical risk minimization, L2 contrast, orthogonal bases.
result Effective approximation of classical probabilistic models like Gaussian and graphical models.
Kauri is a novel unsupervised binary tree for clustering that outperforms existing methods.
problem Learning a tree end-to-end for clustering without labels is an open challenge.
method Greedy maximization of the kernel KMeans objective without centroids.
result Kauri often outperforms existing unsupervised clustering methods, especially with non-linear kernels.
The theme in this paper is the recombining binomial tree to price American put option when the underlying stock follows constant elasticity of variance(CEV) process. Recombining nodes of binomial tree are decided from finite difference scheme to emulate CEV process and the tree has a linear complexity. Also it is deriv…
Tree-AMP simplifies inference in complex tree-structured models.
problem Inference in high-dimensional tree-structured models.
method Approximate Message Passing algorithms for various machine learning tasks.
result Theoretical performance predictions and automated entropy estimation.
dtControl uses decision trees to represent controllers efficiently and explainably.
problem Representing controllers concisely and explainably.
method dtControl uses decision tree learning algorithms to represent controllers. Novel techniques for determinizing controllers are introduced.
result Novel techniques for determinizing controllers during decision tree construction are extremely efficient, yielding small decision trees.
Novel algorithm optimizes decision trees for nonlinear metrics.
problem Optimizing decision trees for nonlinear metrics like F1-score.
method Bi-objective optimisation approach to find optimal trees on Pareto frontier.
result The optimal tree for nonlinear metrics lies on the Pareto frontier.
Decision trees perform well in complex interactions, even when interactions are not fully accounted for.
problem Interpreting complex interactions in machine learning models.
method Experiments on datasets and two methods for robust GLMs.
result Tree depth compensates for model misspecification, enhancing performance in complex scenarios.
New framework detects model weaknesses in decision tree ensembles.
problem Detecting feature sensitivity in decision tree ensembles.
method Data-aware sensitivity framework using MILP and SMT.
result Realistic and interpretable examples of model weaknesses.
Learning a regression function using censored or interval-valued output data is an important problem in fields such as genomics and medicine. The goal is to learn a real-valued prediction function, and the training output labels indicate an interval of possible values. Whereas most existing algorithms for this task are…
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also…
Despite outstanding contribution to the significant progress of Artificial Intelligence (AI), deep learning models remain mostly black boxes, which are extremely weak in explainability of the reasoning process and prediction results. Explainability is not only a gateway between AI and society but also a powerful tool t…
Linear algebra algorithms are used widely in a variety of domains, e.g machine learning, numerical physics and video games graphics. For all these applications, loop-level parallelism is required to achieve high performance. However, finding the optimal way to schedule the workload between threads is a non-trivial prob…
The kernel method is a potential approach to analyzing structured data such as sequences, trees, and graphs; however, unordered trees have not been investigated extensively. Kimura et al. (2011) proposed a kernel function for unordered trees on the basis of their subpaths, which are vertical substructures of trees resp…
DTE uses tree leaf means to embed data, balancing accuracy and speed.
problem High variance in decision tree splits and computational inefficiency of ensembles.
method DTE constructs an interpretable feature representation using leaf means of a trained tree.
result DTE strikes a balance between accuracy and computational efficiency, outperforming ensembles.