DNDT combines neural networks and decision trees for tabular data.
problem Tabular data processing with interpretability and efficiency.
method Deep Neural Decision Trees (DNDT) using neural networks to model decision trees.
result DNDT achieves both interpretability and efficiency in tabular data processing.
L1-orthogonal regularization improves decision tree explainability of deep neural networks.
problem Lack of explainability in deep neural networks.
method L1-orthogonal regularization during training of decision trees.
result Decision trees closely approximate trained deep neural networks with improved accuracy and fidelity.
Proposes regional tree regularization for interpretable deep models.
problem Lack of interpretability in deep neural networks.
method Encourages deep models to be well-approximated by separate decision trees for predefined regions of the input space.
result Regional tree regularization delivers more accurate predictions than training separate decision trees for each region, while producing simpler explanations.
Knowledge distillation simplifies deep models into interpretable decision trees.
problem Interpretability of deep neural networks is challenging and important for practical deployment.
method Knowledge distillation applied to transform deep models into decision trees.
result The student model achieves better accuracy than vanilla decision trees.
Neural models can realize decision trees with parameter sharing and improved performance.
problem Training and optimizing oblique decision trees.
method Locally constant networks based on ReLU gradients, parameter sharing, and neural tools.
result Locally constant networks can implicitly model oblique decision trees with fewer neurons.
Tree regularization makes deep models interpretable by approximating them with simple decision trees.
problem Lack of interpretability in deep neural networks.
method Tree regularization to train deep models to resemble compact, axis-aligned decision trees.
result Tree regularized models are easier for humans to interpret without sacrificing accuracy.
Deep neural networks predict walking, biking, and driving from Wi-Fi signals.
problem Predicting human mobility modes using Wi-Fi signals.
method Deployed Wi-Fi sensors at four locations, developed and tested multiple classifiers (MLP, Decision Tree, Bagged Decision Tree, Random Forest).
result Multilayer Perceptron achieved 86.52% correct predictions of mobility modes.
DBDT uses deep boosting decision trees for fraud detection.
problem Fraud detection in imbalanced data.
method Gradient boosting with neural networks (SDT), AUC maximization.
result DBDT significantly improves fraud detection performance.
Simplifies neural networks into easier-to-explain decision trees.
problem Difficulty in explaining neural network decisions.
method Converts neural network into a soft decision tree.
result The resulting soft decision tree generalizes better.
ANTs integrate neural networks and decision trees for better performance.
problem Combining neural networks and decision trees for improved performance.
method Adaptive Neural Trees (ANTs) integrate representation learning into decision tree structures, using backpropagation for adaptive growth.
result ANTs achieve competitive performance on classification and regression datasets, with benefits in lightweight inference, feature separation, and adaptable architecture.
NODE improves deep learning on tabular data, outperforming GBDT.
problem Limited performance of deep learning on tabular data compared to gradient boosting decision trees.
method Introducing Neural Oblivious Decision Ensembles (NODE), a deep learning architecture that generalizes ensembles of oblivious decision trees.
result NODE outperforms leading GBDT packages on most tabular tasks.
End-to-end learning of deterministic decision trees improves performance.
problem Lack of end-to-end trainable deterministic decision trees.
method Probabilistic decision trees with deterministic annealing, trained using Expectation-Maximization.
result End-to-end trainable deterministic decision trees achieve performance comparable to or better than existing methods.
DOFEN improves DNN performance on tabular data benchmarks.
problem DOFEN tackles the performance gap between DNNs and tree-based models on tabular data.
method DOFEN uses a two-level rODT forest ensembling process inspired by oblivious decision trees.
result DOFEN achieves state-of-the-art results on the Tabular Benchmark.
This paper improves deep forest models with soft routing and topology learning.
problem Expensive computational costs and lack of interpretability in deep neural networks.
method Soft routing in probabilistic trees and topology learning for joint optimization.
result Empowered deep forests achieve better performance with reduced model complexity.
This work automates decision tree construction from neural networks.
problem Creating optimal decision tree architectures from neural networks.
method Automatic induction of non-greedy decision trees using weights transfer from neural networks.
result Improved model performance over fixed hyperparameters.
Bayesian GBMs improve predictive uncertainty calibration for tabular data.
problem Lack of well-calibrated predictive uncertainties in gradient boosting machines.
method Variational inference with soft decision trees.
result Variational soft GBMs provide useful uncertainty estimates and maintain good predictive performance.
Simple machine learning models achieve high accuracy in toxicity prediction.
problem Toxicity prediction of chemical compounds using complex models.
method Using shallow neural networks and decision trees with 2D features.
result Achieves similar or better performance than deep neural networks with less computing time.
Proposes a simple neural network model similar to gradient boosted decision trees.
problem Building a neural network equivalent to gradient boosted decision trees.
method Converts an ensemble of decision trees to a neural network, relaxes properties, and trains a simple neural network model.
result The proposed Hammock model achieves similar performance to gradient boosted decision trees.
eForest uses tree ensembles for auto-encoding with faster training and lower error.
problem Auto-encoding using deep neural networks (DNNs).
method eForest uses decision paths of trees to enable backward reconstruction.
result eForest achieves lower reconstruction error with faster training and is reusable.
Deep tabular models outperform GBDT in medical diagnosis tasks.
problem Transfer learning for tabular data in medical diagnosis.
method Proposes a pseudo-feature method for transfer learning between different feature sets.
result Tabular neural networks outperform GBDT in medical diagnosis tasks.
Paper analyzes soft tree ensembles using NTK, finding only leaf count matters.
problem Understanding impact of various tree architectures in ensemble learning.
method Formulated and analyzed Neural Tangent Kernel (NTK) for soft tree ensembles.
result Only the number of leaves at each depth is relevant for tree architecture in ensemble learning.
Automated medical protocol uses neural networks and decision trees.
problem Improving healthcare delivery through automated decision-making.
method Hybrid model combining neural networks and decision trees.
result Effective early decisions for patient care.
Study explores loss design for decision trees to improve robustness against noisy labels.
problem Improving decision tree robustness to noisy labels.
method Investigated loss correction and symmetric losses, found ineffective.
result Other loss design directions need exploration for robust decision trees.
Converts GBDT trees to neural networks for online updates.
problem Performance loss in converting GBDT trees to neural networks.
method Converts existing GBDT implementations to neural network architectures, allowing online updates of decision splits.
result Learning bounds for neural network architecture with updated splits.
VIPER learns verifiable decision tree policies from deep reinforcement learning.
problem Ensuring safety of learned reinforcement learning policies.
method VIPER combines model compression and imitation learning to train decision tree policies.
result VIPER learns decision tree policies that are provably robust and stable.
Deep imagination optimizes decision-making in large trees with limited resources.
problem Optimal planning in large decision trees with limited resources and time.
method Analytical solutions and numerical analysis of sampling capacity allocation.
result Optimal policy is to allocate few samples per level for deep exploration, favoring depth over breadth.
Integrates differentiable decision trees into neural networks for faster training and inference.
problem Combining differentiability and conditional computation in tree ensembles for neural networks.
method Sparse activation function and specialized forward/backward propagation algorithms for efficient training and inference.
result 10x speed-ups and 20x reduction in parameters compared to existing methods, while maintaining performance.
Proposes mGBDTs for learning hierarchical representations in gradient boosting decision trees.
problem Inability of gradient boosting decision trees to learn hierarchical representations.
method Introduces multi-layered GBDT forest (mGBDTs) with explicit emphasis on hierarchical learning.
result Jointly trained mGBDTs can learn hierarchical representations effectively without backpropagation.
Extracts decision trees from CNNs to explain concept importance.
problem Understanding how CNNs make decisions about human-understandable concepts.
method Inferring labeled concept data from CNN hidden layer activations and creating a shallow decision tree.
result Extracted decision trees accurately represent CNN classifications.
Optimized decision trees for reinforcement learning, improving sample complexity and interpretability.
problem Updating decision trees online in reinforcement learning.
method Gradient update over differentiable decision trees, including theoretical justification and empirical validation.
result Our approach outperforms neural networks in sample complexity and achieves higher rewards online.
Tree regularization improves deep model interpretability without sacrificing accuracy.
problem Lack of interpretability in deep models hinders their adoption.
method Explicitly regularizes deep models to be closely modeled by decision trees with few nodes.
result Tree-regularized models are easier for humans to simulate than simpler penalties without sacrificing accuracy.
Transform ANNs into interpretable decision trees.
problem Lack of interpretability in ANNs.
method Developed two MDT algorithms: EC-DT and Extended C-Net.
result Extended C-Net generates the most compact and effective trees.
Enhances multi-class classification using neural networks and decision trees.
problem Improving multi-class classification accuracy.
method Combines neural networks, decision trees, and random vector functional link networks with oblique decision surfaces.
result Superior performance on multi-class datasets compared to state-of-the-art classifiers.
Residual Networks are shown to be equivalent to boosting feature representation.
problem Improving feature representation in deep learning models.
method Proved ResNet's equivalence to Online Gradient Boosting and proposed decision tree residual modules.
result ResNet can achieve Online Gradient Boosting regret bounds through architectural changes.
New random forest algorithms for PU learning minimize risk directly.
problem Learning from positive and unlabeled data.
method Recursive greedy risk minimization for decision trees.
result Efficient PU random forest algorithm with robustness and low hyperparameter tuning.
Proposes DVC for better variable selection in non-grid data.
problem Challenges of identifying important variables in non-grid data.
method Imposes chain structure on blocks of variables using step-wise greedy search.
result Outperforms other generic DNNs and classifiers.
DBT combines diffusion models and boosting for supervised learning.
problem Supervised learning problems.
method Diffusion Boosting paradigm and Diffusion Boosted Trees (DBT).
result DBT outperforms deep neural network-based diffusion models and is effective on real-world classification tasks.
Tree-LIME explains deep learning models using decision trees.
problem Deep learning models are black boxes, making them hard to explain and prone to biases.
method Developed a Tree-LIME approach using decision trees to explain predictions of deep learning models.
result Tree-LIME can capture nonlinear interactions and creates more reliable explanations.
Paper interprets deep learning using decision trees and Haar wavelets.
problem Understanding the function approximation capabilities of ReLU deep learning.
method Constructing a deep learning structure equivalent to a forest and approximating Haar wavelet functions with ReLU deep learning.
result ReLU deep learning can be considered as decision trees and approximates Haar wavelet functions with arbitrary precision.
Paper proposes ECOC for deep neural network ensembles to improve performance.
problem Designing an ensemble of deep networks is time-consuming and often not beneficial.
method ECOC framework applied to deep networks, with design strategies to balance accuracy and complexity.
result Proposed combinatory technique achieves highest classification performance.
Neural Additive Models combine neural nets with interpretable models for high stakes tasks.
problem Inability to understand how neural networks make decisions.
method Combines neural nets with generalized additive models to create Neural Additive Models (NAMs).
result NAMs are more accurate than intelligible models and as accurate as state-of-the-art generalized additive models.
Develops a fair tree boosting method for tabular data.
problem Lack of fair classifiers based on decision trees in tabular data.
method Adversarial gradient tree boosting that minimizes adversarial neural network's ability to predict sensitive attributes.
result Achieves higher accuracy while maintaining fairness.
New method learns binary decision trees efficiently.
problem Learning binary decision trees for data partitioning.
method Argmin differentiation for discrete and continuous parameters.
result Produces competitive binary trees with fast training.
The paper studies how neural policies can be interpreted using decision trees.
problem Understanding how machine learning controllers make decisions in complex environments.
method The approach involves disentangled representation using decision trees to interpret neural policies.
result The paper shows that disentanglement of learned neural dynamics improves interpretability.
This review covers methods for autonomous driving including tracking, prediction, and decision making.
problem Improving autonomous driving through better tracking, prediction, and decision making.
method Approaches based on neural networks, stochastic techniques, and reinforcement learning are discussed.
result Effective methods for autonomous driving are identified and compared.
Proposes a method to speed up model selection for classification.
problem Time-consuming model selection process and lack of dataset-specific insights.
method Relaxes decision boundaries of neural decision trees to find equivalent or better models.
result Reduces the scope of exploration needed for model selection.
This work uses decision trees to encode relevant features and their interactions into neural networks, improving model performance.
problem Overfitting in neural networks with many irrelevant variables.
method Defines a mapping to encode decision tree extracted relationships into a neural network.
result The approach outperforms fully connected neural networks and tree-based methods.
MoËT combines decision trees and a gating function for verifiable reinforcement learning.
problem Lack of safety guarantees and explainability in deep learning models for safety-critical applications.
method Mixture of Experts with decision tree experts and a gating function.
result MoËT outperforms previous techniques in reinforcement learning and is verifiable.