Sparse oblique decision tree improves security rules for renewable power systems.
problem Identifying secure operating conditions in power systems with high renewable energy.
method Sparse weighted oblique decision tree to learn and embed linear security rules.
result The method significantly increases secure states and reduces solution time.
FoLDTree improves oblique decision trees with ULDA, enhancing accuracy and feature selection.
problem Axis-orthogonal splits limit traditional decision trees' performance on oblique decision boundaries.
method Integrates ULDA into decision tree structure for efficient oblique splits, feature selection, and handling missing values.
result FoLDTree outperforms other methods in accuracy and feature selection, comparable to random forest.
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for s0-sparse oblique splits, proposing progressive trees for iterative refinement. result Demonstrated that SID function class expands with s0-sparsity, enabling capture of complex data-generating processes. Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using…
We show how neural models can be used to realize piece-wise constant functions such as decision trees. The proposed architecture, which we call locally constant networks, builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally …
New tree and forest methods use oblique splits for better risk bounds.
problem Improving risk bounds for regression algorithms.
method Randomized decision trees and forests with oblique splits.
result Oblique splits lead to better risk bounds for multi-index models.
Enhanced ODT with Feature Concatenation boosts learning efficiency.
problem Insufficient learning efficiency of ODT due to linear projections not being transmitted to child nodes.
method Feature Concatenation ( exttt{FC-ODT}) to transmit linear projections along decision paths.
result Experiments show exttt{FC-ODT} outperforms state-of-the-art decision trees with a limited tree depth.
Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier,…
Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many r…
Study integrates reliability constraints into generation planning models.
problem Challenges in integrating reliability constraints with generation planning models.
method Leverages a weighted oblique decision tree (WODT) technique to embed reliability verification constraints.
result Demonstrates effectiveness in achieving reliable and optimal planning solutions.
Conventional decision trees have a number of favorable properties, including interpretability, a small computational footprint and the ability to learn from little training data. However, they lack a key quality that has helped fuel the deep learning revolution: that of being end-to-end trainable, and to learn from scr…
Proposes oblique predictive clustering trees for faster, more efficient learning.
problem Learning time scales poorly with output space dimensionality and cannot exploit data sparsity.
method Design and implement oblique splits using linear combinations of features.
result Achieves performance on-par with state-of-the-art methods and is orders of magnitude faster.
Efficient oblique RSF method improves prediction and interpretability.
problem Limited computational efficiency and difficulty in interpreting oblique RSF ensembles.
method Newton-Raphson scoring for computational efficiency and negation importance for variable importance estimation.
result The method reduces computational overhead by 450 times and improves prediction accuracy.
Resource-efficient oblique trees reduce neural signal classification costs.
problem Implementing efficient neural signal classifiers on resource-constrained devices.
method Integrating model compression, probabilistic routing, and cost-aware learning.
result Significant reduction in model size and feature extraction cost compared to state-of-the-art models.
In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more…
We present a new way of constructing an ensemble classifier, named the Guided Random Forest (GRAF) in the sequel. GRAF extends the idea of building oblique decision trees with localized partitioning to obtain a global partitioning. We show that global partitioning bridges the gap between decision trees and boosting alg…
Single tree outperforms random forest in testing accuracy.
problem The challenge of improving single decision tree performance.
method Gradient-based entire tree optimization framework, scaled sigmoid approximation, numerical stability algorithm, subtree polish strategy.
result Optimized single tree outperforms classic random forest by 2.03% on average.
Paper proposes a novel SVM method for creating survival trees.
problem Creating non-linear survival trees for right-censored data.
method L2-regularized dipole splitting criteria with kernel methods.
result Non-linear splits using polynomial and Gaussian kernels show similar predictive power but often smaller tree sizes.
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression. Individual canonical correlation trees are binary decision trees with hyperplane splits based on local canonical correlation coefficients calculated during training. Unlike axis-aligned alternatives…
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
This work presents an approach to automatically induction for non-greedy decision trees constructed from neural network architecture. This construction can be used to transfer weights when growing or pruning a decision tree, allowing non-greedy decision tree algorithms to automatically learn and adapt to the ideal arch…
NSOTree combines neural networks and trees for better survival analysis interpretability.
problem Survival analysis event prediction with interpretability.
method Proposes NSOTree integrating neural networks and trees for better interpretability.
result NSOTree improves both performance and interpretability in survival analysis.
AGBoost uses attention weights to improve GBM for regression problems.
problem Improving gradient boosting machine for regression tasks.
method Attention-based modification of GBM with trainable attention weights.
result AGBoost achieves better performance on regression datasets.
The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.
problem Random forests' uniform voting fails to correct errors in regions where incorrect tree representations outnumber correct ones.
method The paper introduces using the structural pattern of each tree's decision path as an instance-adaptive reliability signal to identify and weight more reliable trees.
result Using the proposed method yields a statistically significant accuracy improvement over RF on 36 binary classification benchmarks.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
Bayesian Decision Trees are known for their probabilistic interpretability. However, their construction can sometimes be costly. In this article we present a general Bayesian Decision Tree algorithm applicable to both regression and classification problems. The algorithm does not apply Markov Chain Monte Carlo and does…
Convex polytope trees expand decision trees with interpretable boundaries.
problem High accuracy often requires many nodes in decision trees, reducing interpretability.
method CPT uses logical disjunction of weighted linear decision-makers, geometrically a convex polytope.
result CPT achieves high accuracy with fewer nodes compared to existing methods.
The Mondrian process represents an elegant and powerful approach for space partition modelling. However, as it restricts the partitions to be axis-aligned, its modelling flexibility is limited. In this work, we propose a self-consistent Binary Space Partitioning (BSP)-Tree process to generalize the Mondrian process. Th…
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
We present an alternative layer to convolution layers in convolutional neural networks (CNNs). Our approach reduces the complexity of convolutions by replacing it with binary decisions. Those binary decisions are used as indexes to conditional distributions where each weight represents a leaf in a decision tree. This m…
This paper describes experiments, on two domains, to investigate the effect of averaging over predictions of multiple decision trees, instead of using a single tree. Other authors have pointed out theoretical and commonsense reasons for preferring the multiple tree approach. Ideally, we would like to consider predictio…
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …
DTE uses tree leaf means to embed data, balancing accuracy and speed.
problem High variance in decision tree splits and computational inefficiency of ensembles.
method DTE constructs an interpretable feature representation using leaf means of a trained tree.
result DTE strikes a balance between accuracy and computational efficiency, outperforming ensembles.
G-FIGS uses instance weights to create interpretable models from diverse data.
problem Generalizing to diverse data distributions while maintaining interpretability.
method Estimates group membership probabilities, uses as instance weights in FIGS to grow decision trees.
result Achieves state-of-the-art prediction performance and maintains interpretability.
Decision forests (Forests), in particular random forests and gradient boosting trees, have demonstrated state-of-the-art accuracy compared to other methods in many supervised learning scenarios. In particular, Forests dominate other methods in tabular data, that is, when the feature space is unstructured, so that the s…
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
Proposes PredVAR model for reduced-dimensional dynamics from noisy data.
problem Extracting low-dimensional dynamics from high-dimensional noisy data.
method Probabilistic reduced-dimensional vector autoregressive model with oblique projection.
result Iterative algorithm yields dynamic latent variables with rank-ordered predictability.
Machine learning classifiers often stumble over imbalanced datasets where classes are not equally represented. This inherent bias towards the majority class may result in low accuracy in labeling minority class. Imbalanced learning is prevalent in many real-world applications, such as medical research, network intrusio…
Unified theory and debiasing framework for random oblique projections in high dimensions.
problem Systematic statistical bias in random oblique projections induced by sampling.
method Unified non-asymptotic theory and debiasing framework.
result Sharp bias--variance characterizations and improved approximation accuracy.
Machine learning tools are commonly used in modern high energy physics (HEP) experiments. Different models, such as boosted decision trees (BDT) and artificial neural networks (ANN), are widely used in analyses and even in the software triggers. In most cases, these are classification models used to select the "signal"…
Decision Machines embeds decision trees into vector spaces for improved optimization.
problem Overfitting and difficulty in finding optimal decision tree structure.
method Embedding Boolean tests into a binary vector space and representing tree structure as matrices.
result Optimized decision trees with enhanced predictive power.
Boosting meta-trees improve decision tree performance.
problem Overfitting in decision trees.
method Boosting approach to construct multiple meta-trees.
result Ensembles of meta-trees prevent overfitting.
Infinite BART model selects number of trees and allows different functions for clusters.
problem Regression and classification analysis with automatic tree selection and cluster-specific functions.
method Incorporates an Indian Buffet process prior to select a subset of decision trees for each observation.
result Infinite BART model outperforms classic BART on simulated and real datasets.
REALITrees uses a Rashomon ensemble approach for active learning in sparse decision trees.
problem Active learning reduces labeling costs by selecting informative samples, but current methods often sacrifice model diversity and direct characterization of the hypothesis space.
method REALITrees constructs a committee of all near-optimal sparse decision tree models using a Rashomon Set and a Gibbs posterior to weight them by empirical risk.
result REALITrees outperforms randomized ensembles, especially in noisy environments, by leveraging expanded model multiplicity.
dtControl uses decision trees to represent controllers efficiently and explainably.
problem Representing controllers concisely and explainably.
method dtControl uses decision tree learning algorithms to represent controllers. Novel techniques for determinizing controllers are introduced.
result Novel techniques for determinizing controllers during decision tree construction are extremely efficient, yielding small decision trees.
Develops an oblique projection technique to approximate a foliation for non-normal dynamics.
problem Modeling dynamics far from a primary Spectral Submanifold (SSM) in non-normal systems.
method Oblique projection technique based on experimental data.
result Approximates a stable invariant foliation for non-normal dynamics efficiently.
The paper develops a decision support system for hierarchical text classification of conference proceedings.
problem Classifying documents with a fixed hierarchical structure of topics.
method Developed a weighted hierarchical similarity function to calculate topic relevance, using entropy of words to estimate weights.
result The weighted hierarchical similarity function improves ranking accuracy compared to other methods.