A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
Improves decision tree performance by correcting split selection errors.
FoLDTree improves oblique decision trees with ULDA, enhancing accuracy and feature selection.
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near-optimal decision trees. ID3, C4.5 and CART are classical decision tree algorithms and the split criteria they…
HATT improves online decision tree ensembles by using a more eager splitting strategy.
New tree splitting criteria improve probabilistic predictions.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
Decision trees algorithms use a gain function to select the best split during the tree's induction. This function is crucial to obtain trees with high predictive accuracy. Some gain functions can suffer from a bias when it compares splits of different arities. Quinlan proposed a gain ratio in C4.5's information gain fu…
Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using…
Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setti…
It is widely believed that the prediction accuracy of decision tree models is invariant under any strictly monotone transformation of the individual predictor variables. However, this statement may be false when predicting new observations with values that were not seen in the training-set and are close to the location…
Improved decision tree learning guarantees for complex functions.
Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised…
Convex polytope trees expand decision trees with interpretable boundaries.
Max-Cut decision tree improves classification accuracy and reduces computation time.
Rectified decision trees improve machine learning interpretability and effectiveness.
A novel gradient-based method optimizes decision trees for complex tasks.
Conventional decision trees have a number of favorable properties, including interpretability, a small computational footprint and the ability to learn from little training data. However, they lack a key quality that has helped fuel the deep learning revolution: that of being end-to-end trainable, and to learn from scr…
CDF uses centroids to split features for high-dimensional classification.
A new decision tree induction method using MIP for faster optimization.
Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementatio…
Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
SketchBoost accelerates GBDT for multioutput problems up to 40x.
Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many r…
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-…
DTE uses tree leaf means to embed data, balancing accuracy and speed.
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression. Individual canonical correlation trees are binary decision trees with hyperplane splits based on local canonical correlation coefficients calculated during training. Unlike axis-aligned alternatives…
Enhanced ODT with Feature Concatenation boosts learning efficiency.
This work describes an outlier detection procedure (named "OutlierTree") loosely based on the GritBot software developed by RuleQuest research, which works by evaluating and following supervised decision tree splits on variables, in whose branches 1-d confidence intervals are constructed for the target variable and pot…
New techniques optimize decision trees for interpretable machine learning.
Recently proposed budding tree is a decision tree algorithm in which every node is part internal node and part leaf. This allows representing every decision tree in a continuous parameter space, and therefore a budding tree can be jointly trained with backpropagation, like a neural network. Even though this continuity …
A new randomized tree classifier outperforms traditional CARTs.
The paper studies statistical properties of CART regression trees.
Covariance-Driven Regression Trees reduce overfitting in CART.
Optimizes decision-making with uncertain variables using auxiliary observations.
We present an algorithm for learning decision trees using stochastic gradient information as the source of supervision. In contrast to previous approaches to gradient-based tree learning, our method operates in the incremental learning setting rather than the batch learning setting, and does not make use of soft splits…
A new type of distributional regression tree uses soft split rules for better predictive performance.
Decision trees with binary splits are popularly constructed using Classification and Regression Trees (CART) methodology. For binary classification and regression models, this approach recursively divides the data into two near-homogenous daughter nodes according to a split point that maximizes the reduction in sum of …
Kauri is a novel unsupervised binary tree for clustering that outperforms existing methods.
Causal trees struggle with accuracy in estimating treatment effects.
Machine learning algorithms aim at minimizing the number of false decisions and increasing the accuracy of predictions. However, the high predictive power of advanced algorithms comes at the costs of transparency. State-of-the-art methods, such as neural networks and ensemble methods, often result in highly complex mod…
New tree and forest methods use oblique splits for better risk bounds.
Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.
New method learns binary decision trees efficiently.
LoBoost improves local conformal prediction for gradient-boosted trees without extra data splits.
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …
This paper introduces TNTK to study infinite soft tree ensembles.
Despite its success and popularity, machine learning is now recognized as vulnerable to evasion attacks, i.e., carefully crafted perturbations of test inputs designed to force prediction errors. In this paper we focus on evasion attacks against decision tree ensembles, which are among the most successful predictive mod…