ANTs integrate neural networks and decision trees for better performance.
problem Combining neural networks and decision trees for improved performance.
method Adaptive Neural Trees (ANTs) integrate representation learning into decision tree structures, using backpropagation for adaptive growth.
result ANTs achieve competitive performance on classification and regression datasets, with benefits in lightweight inference, feature separation, and adaptable architecture.
This work automates decision tree construction from neural networks.
problem Creating optimal decision tree architectures from neural networks.
method Automatic induction of non-greedy decision trees using weights transfer from neural networks.
result Improved model performance over fixed hyperparameters.
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
Automated medical protocol uses neural networks and decision trees.
problem Improving healthcare delivery through automated decision-making.
method Hybrid model combining neural networks and decision trees.
result Effective early decisions for patient care.
The paper tackles high-dimensional function approximation using tree-based tensor formats.
problem Approximating high-dimensional functions in statistical learning.
method Empirical risk minimization over tree-based tensor formats, exploiting multilinear models and sparsity.
result Numerical stability and reliability of the proposed algorithms for learning.
Combines neural networks and decision trees for supervised learning.
problem Challenges in optimizing hierarchical parameters and constructing tree structures.
method Probabilistic approach with modified gradient ascent and adaptive tree construction.
result Novel classification and regression technique that combines strengths of neural networks and decision trees.
New framework uses tree ensembles for contextual bandits.
problem Optimizing decisions in dynamic environments with contextual information.
method Adapts Upper Confidence Bound and Thompson Sampling to tree ensemble methods.
result Tree ensemble methods outperform traditional methods in regret minimization and runtime.
Locally adaptive clustering for tree delineation.
problem Tree delineation from distance data.
method Locally adaptive hierarchical cluster termination.
result Multi-scale alternative to conventional termination criteria.
New method reduces energy consumption of Hoeffding trees by up to 92%.
problem Inefficient energy consumption of Hoeffding trees due to fixed parameters.
method nmin adaptation for Hoeffding trees to adapt nmin parameter dynamically.
result VFDT-nmin consumes up to 92% less energy than CVFDT, trading off a few percent of accuracy.
This research adapts scoring rules for training survival models, improving predictive performance.
problem Training survival models with traditional methods struggles with censoring.
method Adapting scoring rules for survival analysis, creating a flexible framework for model training.
result Scoring rules can be successfully incorporated into model training, yielding competitive performance.
Linearized neural networks provide a fast and interpretable way to adapt models to new settings.
problem Difficulty in understanding and adapting inductive biases of trained neural networks.
method Linearization of neural networks and embedding these biases into Gaussian processes through a kernel designed from the Jacobian.
result Domain adaptation becomes interpretable posterior inference with analytic and scalable computational speed-ups.
SMART combines decision trees and MARS for better regression modeling.
problem High variance in decision trees for continuous relationships, poor performance in MARS for discontinuities.
method SMART uses a decision tree to identify subsets with distinct continuous relationships, then applies MARS to fit these relationships independently.
result SMART improves regression performance over state-of-the-art methods in capturing discontinuities and continuous relationships.
Paper proposes learnable topological features for efficient phylogenetic inference.
problem Finding appropriate topological structures for phylogenetic inference tasks requires significant design effort and domain expertise.
method Combines raw node features with graph neural networks to automatically adapt to different tasks.
result Demonstrates effectiveness and efficiency on simulated and real data phylogenetic inference tasks.
Proposes DVC for better variable selection in non-grid data.
problem Challenges of identifying important variables in non-grid data.
method Imposes chain structure on blocks of variables using step-wise greedy search.
result Outperforms other generic DNNs and classifiers.
AF improves classification models by adaptively weighting trees.
problem Improving classification model performance.
method AF combines OP2T for input-dependent weights and MIO for dynamic refinement.
result AF consistently outperforms RF, XGBoost, and other weighted RF.
We study the convergence of the predictive surface of regression trees and forests. To support our analysis we introduce a notion of adaptive concentration for regression trees. This approach breaks tree training into a model selection phase in which we pick the tree splits, followed by a model fitting phase where we f…
A new tree-based method for adaptive dictionary learning.
problem Sparse encoding and multiscale structure in data.
method Hierarchical clustering and binary partition tree for adaptive dictionary learning.
result Dictionary atoms are organized in a multiscale structure, leading to better reconstruction.
In this paper, we present machine learning approaches for characterizing and forecasting the short-term demand for on-demand ride-hailing services. We propose the spatio-temporal estimation of the demand that is a function of variable effects related to traffic, pricing and weather conditions. With respect to the metho…
A new random forest algorithm improves tree construction for optimal performance.
problem Improving the performance of random forests, especially in complex and smooth scenarios.
method Adaptive split-balancing method using permutation-based splitting criterion.
result Achieves minimax optimality under various Lipschitz and Hölder classes.
Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees…
LassoFlexNet improves deep learning performance on tabular data.
problem Deep learning underperforms tree-based models on tabular data.
method Incorporates five inductive biases and uses Tied Group Lasso for variable selection.
result LassoFlexNet matches or outperforms leading tree-based models on 52 datasets.
Paper proposes a method to improve prediction intervals for neural networks.
problem Improving prediction intervals for neural network models.
method Adapting extremely randomized trees to neural networks to create ensembles.
result The method yields gains in out-of-sample accuracy and is superior to existing methods.
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
TP-AIS improves sampling efficiency over existing methods.
problem Efficient sampling from complex probability distributions.
method Iterative sampling using a tree pyramid structure.
result TP-AIS outperforms DM-PMC, M-PMC, and LAIS.
Alpha-trimming prunes trees in random forests to improve predictive performance.
problem Improving predictive performance of random forests by locally adaptive tree pruning.
method Alpha-trimming is a fast pruning algorithm that prunes trees in a random forest based on signal-to-noise ratio, controlled by a tuning parameter.
result Alpha-trimming often lowers mean squared prediction error compared to fully grown random forests.
Study reviews tree-based methods and introduces new ensemble strategies.
problem Improving the efficiency and performance of tree-based machine learning models.
method Review of tree-based methods, introduction of ISLE framework, ARM model combination strategy, and modified ISLEs.
result Performance evaluation of modified ISLEs on real data sets.
Simulation study evaluates tree-based imputation methods for multi-level data.
problem Ignoring dependencies in hierarchical data can compromise imputation accuracy.
method Chained Random Forests and Extreme Gradient Boosting (mixgb) adapted for multi-level data.
result Adapted boosting methods outperform traditional MICE for Level-1 variables at higher missingness rates.
Deep networks adapt to function regularity and data distribution.
problem Understanding deep learning's adaptability to function regularity and data distribution.
method Developed nonparametric approximation and estimation theories for a broad class of functions using deep ReLU networks.
result Deep neural networks are adaptive to different regularity of functions and nonuniform data distributions.
Improved query complexity for adaptive learning of decision trees.
problem Learning decision trees of depth at most d from membership queries.
method Randomized and deterministic polynomial time algorithms with improved query complexity.
result Queries reduced for both randomized and deterministic algorithms.
Recent breakthrough results in compressive sensing (CS) have established that many high dimensional signals can be accurately recovered from a relatively small number of non-adaptive linear observations, provided that the signals possess a sparse representation in some basis. Subsequent efforts have shown that the perf…
The paper studies bias and adaptivity of CART regression trees.
problem Bias and adaptivity of CART regression trees.
method Derives an interesting connection between bias and MDI measure of variable importance.
result Decision trees with CART have small bias and are adaptive to signal strength and direction.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
Paper presents adaptive Lipschitz bandit framework for efficient optimization.
problem Optimizing rewards and minimizing regret in stochastic Lipschitz bandit problems.
method Adaptive learning of partitions in context- and arm-space using hierarchical Bayesian models.
result Achieves state-of-the-art performance in real-world tasks like neural network hyperparameter tuning.
Paper proposes a new adaptive multiscale value function approximation for reinforcement learning.
problem Value function approximation in reinforcement learning with varying complexity.
method Adaptive multiscale approximation using multiresolution analysis and tree approximation.
result Convergence rate of the multiscale approximation is independent of basis function regularity.
We introduce inference trees (ITs), a new class of inference methods that build on ideas from Monte Carlo tree search to perform adaptive sampling in a manner that balances exploration with exploitation, ensures consistency, and alleviates pathologies in existing adaptive methods. ITs adaptively sample from hierarchica…
Improves decision tree performance by correcting split selection errors.
problem Invalid statistical guarantees in split selection for decision trees.
method Introduces anytime-valid inference to provide valid statistical guarantees.
result Provides anytime-valid control of false splits under arbitrary data streams.
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, suc…
Proposes a simple neural network model similar to gradient boosted decision trees.
problem Building a neural network equivalent to gradient boosted decision trees.
method Converts an ensemble of decision trees to a neural network, relaxes properties, and trains a simple neural network model.
result The proposed Hammock model achieves similar performance to gradient boosted decision trees.
Converts GBDT trees to neural networks for online updates.
problem Performance loss in converting GBDT trees to neural networks.
method Converts existing GBDT implementations to neural network architectures, allowing online updates of decision splits.
result Learning bounds for neural network architecture with updated splits.
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
In this technical note, we adapt an idea of Gabai to construct non-uniquely ergodic, non-geometric, arational trees.
Deep neural networks have been proven powerful at processing perceptual data, such as images and audio. However for tabular data, tree-based models are more popular. A nice property of tree-based models is their natural interpretability. In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised…
Neural Architecture Search (NAS) has shown great success in automating the design of neural networks, but the prohibitive amount of computations behind current NAS methods requires further investigations in improving the sample efficiency and the network evaluation cost to get better results in a shorter time. In this …
A new model predicts spatio-temporal data using adaptive decision trees and point processes.
problem Predicting spatio-temporal data with real-life applications.
method Hawkes process, adaptive decision tree, joint optimization algorithm.
result Significant improvement in predictions compared to standard methods.
A new algorithm, Regular Tree Search, tackles non-convex simulation optimization problems.
problem Non-convex objective functions in simulation optimization.
method Integrates adaptive sampling with recursive partitioning of the search space.
result Proves global convergence and reliably identifies the global optimum.
Paper presents a self-adaptive learning model for robust classification and regression.
problem Dealing with various datasets of different complexity.
method Combines DNDN and DSP, an end-to-end training approach with multiple randomly initialized softmax layers and adaptive soft pruning.
result The model demonstrates no performance loss compared with unpruned models and higher robustness over different data and feature distributions.
A new modeling framework CSN simplifies and interprets machine learning models.
problem Complexity and interpretability issues in machine learning models.
method Combines spline transformation and cross-network to create CSN.
result CSN is as performant and interpretable as XGBoost and FCNN.
Adaptive Bayesian model for covariate-dependent power spectra analysis.
problem Estimating complex relationships and interactions between covariates and power spectra.
method Bayesian sum of trees model with local power spectrum estimation and reversible-jump MCMC for tree modifications.
result The method can accurately recover both smooth and abrupt changes in power spectra across multiple covariates.