Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
problem Creating highly accurate and easily interpretable classifiers for practical applications.
method Representing classifiers as assemblies of simple mathematical rules using NLDTs with evolutionary bilevel optimization.
result The approach ensures interpretability while achieving high accuracy on various classification problems.
In this work, we propose a novel node splitting method for regression trees and incorporate it into the regression forest framework. Unlike traditional binary splitting, where the splitting rule is selected from a predefined set of binary splitting rules via trial-and-error, the proposed node splitting method first fin…
New split rules improve subpopulation targeting in policy-making.
problem Improving binary classification for subpopulation targeting in policy-making.
method MDFS, PFS, wEFS for maximizing distance and penalizing final splits.
result Proposed methods target more vulnerable subpopulations than classic CART/KD-CART.
A new type of distributional regression tree uses soft split rules for better predictive performance.
problem Estimating complete conditional distributions in regression.
method Distributional adaptive soft regression trees using multivariate soft split rules.
result The method outperforms various benchmark methods, especially in complex non-linear interactions.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
Tree-based algorithm for functional data analysis reduces generalization error.
problem Classification and regression problems with functional data.
method Constrained convex optimization for weighted functional L2 space, multiple splitting rules, and weighted integral features. result Reduces generalization error while maintaining interpretability.
A novel gradient-based method optimizes decision trees for complex tasks.
problem Training decision trees with arbitrary differentiable loss functions.
method Gradient-based optimization using first and second derivatives of loss functions.
result Improves accuracy and flexibility in decision tree optimization.
Simultaneous Latent Budget Trees for stratified classification
problem Classification with stratification factors
method Probabilistic machine learning framework
result Interpretation of latent components and conditional split rule
Recursive partitioning approaches producing tree-like models are a long standing staple of predictive modeling, in the last decade mostly as ``sub-learners'' within state of the art ensemble methods like Boosting and Random Forest. However, a fundamental flaw in the partitioning (or splitting) rule of commonly used tre…
DTE uses tree leaf means to embed data, balancing accuracy and speed.
problem High variance in decision tree splits and computational inefficiency of ensembles.
method DTE constructs an interpretable feature representation using leaf means of a trained tree.
result DTE strikes a balance between accuracy and computational efficiency, outperforming ensembles.
Study compares Cox model and RSF for predicting patient survival, finding RSF superior in certain scenarios.
problem Comparing predictive accuracy of Cox proportional hazards model and Random Survival Forest for patient-specific survival probabilities.
method Conducted a comprehensive comparison study using simulation scenarios and real-world datasets.
result RSF outperforms Cox model in nonproportional hazards settings and with treatment-covariate interactions.
Despite the impressive performance of random forests (RF), its theoretical properties have not been thoroughly understood. In this paper, we propose a novel RF framework, dubbed multinomial random forest (MRF), to analyze the \emph{consistency} and \emph{privacy-preservation}. Instead of deterministic greedy split rule…
Paper introduces a medoid-based approach for efficient Fréchet regression.
problem Regression in metric spaces with random objects.
method Adapted random forest algorithm with medoid-based splitting rule.
result Asymptotic equivalence and consistency of the regression estimator.
New Random Forest variants estimate heterogeneous treatment effects using Wasserstein distances.
problem Estimating heterogeneous treatment effects in complex situations.
method Proposes natural variants of Random Forests using Wasserstein distances.
result Natural variants of Random Forests are well-suited for estimating conditional distributions.
New method builds robust trees from noisy data.
problem Building accurate classification trees from noisy labeled data.
method Combines SVM-like splitting rules and label noise detection.
result Effective in detecting and mitigating label noise.
This paper introduces TNTK to study infinite soft tree ensembles.
problem Understanding the behavior of infinite soft tree ensembles.
method Introduced Tree Neural Tangent Kernel (TNTK) to analyze infinite soft tree ensembles.
result Identified several non-trivial properties of infinite soft tree ensembles.
CovRegRF estimates covariance matrix from covariates using random forests.
problem Estimating conditional covariances or correlations among multivariate responses.
method Random forest trees with a custom splitting rule to maximize covariance difference.
result Accurate covariance matrix estimates and controlled Type-1 error.
In many situations, the choice of an adequate similarity measure or metric on the feature space dramatically determines the performance of machine learning methods. Building automatically such measures is the specific purpose of metric/similarity learning. In Vogel et al. (2018), similarity learning is formulated as a …
The random forest algorithm (RF) has several hyperparameters that have to be set by the user, e.g., the number of observations drawn randomly for each tree and whether they are drawn with or without replacement, the number of variables drawn randomly for each split, the splitting rule, the minimum number of samples tha…
Paper analyzes soft tree ensembles using NTK, finding only leaf count matters.
problem Understanding impact of various tree architectures in ensemble learning.
method Formulated and analyzed Neural Tangent Kernel (NTK) for soft tree ensembles.
result Only the number of leaves at each depth is relevant for tree architecture in ensemble learning.
A new method estimates conditional canonical correlations using random forests.
problem Estimating relationships between two sets of variables given covariates.
method Random Forest with Canonical Correlation Analysis (RFCCA)
result RFCCA provides accurate canonical correlation estimations and well-controlled Type-1 error.
A new decision tree induction method using MIP for faster optimization.
problem Optimizing decision tree split rules for better classification performance.
method Mixed-integer programming (MIP) for Gini reduction maximization, efficient search algorithm.
result bsnsing trees outperform other tree models in new case discrimination.
This paper examines the stability of learned explanations for black-box predictions via model distillation with decision trees. One approach to intelligibility in machine learning is to use an understandable `student' model to mimic the output of an accurate `teacher'. Here, we consider the use of regression trees as a…
Causal trees struggle with accuracy in estimating treatment effects.
problem Estimating heterogeneous causal treatment effects using recursive decision trees.
method Adaptive recursive partitioning with and without sample splitting.
result Causal tree estimators can have uniform-norm errors decreasing more slowly than any power of the sample size.
The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.
problem Understanding the generalization properties of decision trees on different types of features.
method Introducing partitioning functions, relating them to growth functions and VC dimension, and deriving bounds for decision stumps and trees of various structures.
result Exact VC dimension of decision stumps and improved pruning algorithms for binary trees.
Model identifies order splitting and liquidity replenishment as necessary for the square-root law of market impact.
problem Quantifying the square-root law of market impact and identifying its underlying mechanisms.
method Minimal limit-order-book model with heterogeneous interacting agents calibrated against real data. Counterfactual ablation to isolate mechanisms.
result Order splitting and liquidity replenishment are necessary for the square-root law of market impact.
This paper explains CART random forests using stochastic control theory.
problem Understanding the inner workings of CART random forests.
method Developed a stochastic-control perspective on CART random forests, interpreting feature subsampling as a random feasible action set and the split rule as a policy.
result Established that the CART policy is locally stabilizing but globally suboptimal for the forest objective.