arXiv research

This work improves mixing rates for Bayesian CART, a key component of BART.

Bayesian CART models improve insurance claims frequency prediction and interpretation.

We propose a novel "tree-averaging" model that utilizes the ensemble of classification and regression trees (CART). Each constituent tree is estimated with a subset of similar data. We treat this grouping of subsets as Bayesian ensemble trees (BET) and model them as an infinite mixture Dirichlet process. We show that B…

The paper introduces BCART models for aggregate claim amount, improving frequency-severity and joint modeling.

New split rules improve subpopulation targeting in policy-making.

This paper explains CART random forests using stochastic control theory.

Covariance-Driven Regression Trees reduce overfitting in CART.

Bregman perspective on CART provides a unified framework for impurity measures.

Data mining and machine learning techniques such as classification and regression trees (CART) represent a promising alternative to conventional logistic regression for propensity score estimation. Whereas incomplete data preclude the fitting of a logistic regression on all subjects, CART is appealing in part because s…

Undirected graphical models encode in a graph $G$ the dependency structure of a random vector $Y$. In many applications, it is of interest to model $Y$ given another random vector $X$ as input. We refer to the problem of estimating the graph $G(x)$ of $Y$ conditioned on $X=x$ as ``graph-valued regression.'' In this pap…

The paper analyzes the convergence of CART under a SID condition, improving previous results.

The paper studies statistical properties of CART regression trees.

Risk bounds for Classification and Regression Trees (CART, Breiman et. al. 1984) classifiers are obtained under a margin condition in the binary supervised classification framework. These risk bounds are obtained conditionally on the construction of the maximal deep binary tree and permit to prove that the linear penal…

Study uses AI techniques to predict bank customer solvency.

Proposed by Donoho (1997), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the contex…

The paper develops a cross-validation method for improving signal denoising techniques.

Selective inference framework for CART trees to control error rates and coverage.

Decision tree learning is a popular approach for classification and regression in machine learning and statistics, and Bayesian formulations---which introduce a prior distribution over decision trees, and formulate learning as posterior inference given data---have been shown to produce competitive performance. Unlike c…

A single slow-growing tree matches Random Forest's performance.

Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these t…

We propose the use of Bayesian networks, which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies under circumstances in which a test-time input differs significantly from the training set. Our algorithm combines reinforcement learning and end-to-end imita…

Decision trees with binary splits are popularly constructed using Classification and Regression Trees (CART) methodology. For binary classification and regression models, this approach recursively divides the data into two near-homogenous daughter nodes according to a split point that maximizes the reduction in sum of …

A new randomized tree classifier outperforms traditional CARTs.

Decision trees improve performance in various fields.

Paper detects bias in AI medical models using CART.

Paper explores grafting consistent estimators to improve Random Forest consistency.

This study uses machine learning to predict sovereign credit ratings and identifies key factors.

We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…

This paper explores the use of Column Generation (CG) techniques in constructing univariate binary decision trees for classification tasks. We propose a novel Integer Linear Programming (ILP) formulation, based on root-to-leaf paths in decision trees. The model is solved via a Column Generation based heuristic. To spee…

AF improves classification models by adaptively weighting trees.

Previous algorithms for constructing regression tree models for longitudinal and multiresponse data have mostly followed the CART approach. Consequently, they inherit the same selection biases and computational difficulties as CART. We propose an alternative, based on the GUIDE approach, that treats each longitudinal d…

We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present GapTV, an approach that …

We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present Maximum Variance Total V…

Kauri is a novel unsupervised binary tree for clustering that outperforms existing methods.

Recursive partitioning approaches producing tree-like models are a long standing staple of predictive modeling, in the last decade mostly as ``sub-learners'' within state of the art ensemble methods like Boosting and Random Forest. However, a fundamental flaw in the partitioning (or splitting) rule of commonly used tre…

Decision trees are consistent for regression and classification tasks even with many predictors.

WRAAC uses Wasserstein distance for robust reinforcement learning.

One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy …

The infinitesimal jackknife (IJ) has recently been applied to the random forest to estimate its prediction variance. These theorems were verified under a traditional random forest framework which uses classification and regression trees (CART) and bootstrap resampling. However, random forests using conditional inferenc…

In this paper, we introduce a new approach to multiclass classification problem. We decompose the problem into a series of regression tasks, that are solved with CART trees. The proposed method works significantly faster than state-of-the-art solutions while giving the same level of model quality. The algorithm is also…

With the rapid growth in fashion e-commerce and customer-friendly product return policies, the cost to handle returned products has become a significant challenge. E-tailers incur huge losses in terms of reverse logistics costs, liquidation cost due to damaged returns or fraudulent behavior. Accurate prediction of prod…

Model selection is a crucial issue in machine-learning and a wide variety of penalisation methods (with possibly data dependent complexity penalties) have recently been introduced for this purpose. However their empirical performance is generally not well documented in the literature. It is the goal of this paper to in…

Max-Cut decision tree improves classification accuracy and reduces computation time.

This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.

Improves test set performance and reduces out-of-sample disappointment for unstable models.

Causal trees struggle with accuracy in estimating treatment effects.

TMLE improves causal effect estimation in missing data scenarios with various positivity violations.

TabSODA improves imputation of surveys with skips and ordinal data.