iMondrian forest combines isolation forest and Mondrian forest for better anomaly detection.
problem Anomaly detection in batch and online settings.
method Hybrid of isolation forest and Mondrian forest, using depth in Mondrian forest structure.
result iMondrian forest outperforms existing methods in batch and online settings.
Random Hinge Forests are a new decision forest method that can be integrated into neural networks.
problem Training and optimizing neural networks efficiently and effectively.
method Random Hinge Forests are a novel variant of decision forests that can be integrated into neural networks and optimized end-to-end.
result Random Hinge Forests can be efficiently optimized end-to-end with stochastic gradient descent.
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for r…
HDI-Forest improves regression prediction intervals using Random Forest.
problem Improving the quality of prediction intervals in regression tasks.
method HDI-Forest is a novel quality-based PI estimation method based on Random Forest, optimizing PI quality metrics directly from standard tree-based models.
result HDI-Forest significantly reduces PI width by over 20% compared to previous methods, while maintaining or improving coverage probability.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
RFpredInterval package builds prediction intervals for random forests and boosted forests.
problem Quantifying uncertainty in random forest and boosted forest point predictions.
method 16 methods to build prediction intervals with random forests and boosted forests.
result The proposed method outperforms existing methods in building prediction intervals.
Boosting random forests reduces bias and improves predictive performance.
problem Reducing bias in random forest predictions.
method Extract residuals from random forest, fit another random forest to residuals, sum predictions.
result One-step boosted forest has reduced bias and improved predictive performance.
Improved random forest proximities capture data geometry.
problem Inaccurate random forest proximities do not reflect learned data geometry.
method Introduce RF-GAP: Geometry- and Accuracy-Preserving proximities.
result RF-GAP improves geometric representation in tasks like data imputation.
Deep forests enhance expressiveness exponentially with depth, not width or tree size.
problem Understanding the role of depth, width, and tree size in deep forest performance.
method Provided upper and lower bounds on deep forest approximation complexity.
result Depth exponentially enhances deep forest expressiveness.
This paper improves forest pruning to balance accuracy and interpretability.
problem Limited interpretability of regression forests.
method Lasso-pruning and theoretical analysis of regression forests.
result Pruned regression forests can achieve equal or better accuracy than unpruned ones, with significant size reduction.
This paper is a comment on the survey paper by Biau and Scornet (2016) about random forests. We focus on the problem of quantifying the impact of each ingredient of random forests on their performance. We show that such a quantification is possible for a simple pure forest , leading to conclusions that could apply more…
Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
This paper studies Mondrian Forests, proving they achieve optimal rates in arbitrary dimensions.
problem Understanding optimal rates for Mondrian Forests in arbitrary dimensions.
method Theoretical study based on new results about Mondrian partitions.
result Mondrian Forests achieve minimax optimal rates for s-Hölder functions in arbitrary dimensions. Improves time series classification with forest proximities.
problem Time series classification accuracy and efficiency.
method PF-GAP, an extension of RF-GAP proximities to proximity forests, combined with Multi-Dimensional Scaling and Local Outlier Factors.
result Forest proximities show stronger connection between misclassified points and outliers.
Random forests reduce bias and variance, especially in low SNR settings.
problem Reducing bias and variance in machine learning models, particularly in low SNR scenarios.
method Empirical study of random forests and bagging ensembles, focusing on the importance of mtry tuning. result Random forests reduce both bias and variance, outperforming bagging ensembles in high SNR settings.
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
Study examines survival models for ALS, focusing on proportional hazards assumption.
problem Impact of proportional hazards assumption on survival models for ALS.
method Theoretical and empirical investigation of survival forests and their variants.
result Alternative split procedures can improve model performance in non-proportional hazards situations.
Enhances random forest consistency and introduces DMRF for improved performance.
problem Improving the consistency and efficiency of random forest algorithms.
method Strengthened proof methods and propose DMRF algorithm.
result DMRF achieves better theoretical and experimental performance than previous variants.
Random forests improve probability estimates through kernel regression.
problem Improving the principled approach to random forest probability estimation.
method Forge a connection between random forests and kernel regression, develop a proximity kernel model.
result Improves statistical footing of random forest probability estimation.
Online random forests improve Q-learning performance in specific gym environments.
problem Improving Q-learning performance in reinforcement learning tasks.
method Proposed online random forests as Q-function approximators and growing them as learning progresses.
result Improved performance over state-of-the-art Deep Q-Networks in specific gym environments.
New method learns representations for decision forests using input perturbation.
problem Decision forests struggle with raw structured data and lack effective representations.
method Approximate decision forest gradients through input perturbation.
result Effective representation learning for decision forests without structural changes.
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
Proposes a more accurate random forest algorithm by selecting the best-performing trees.
problem Improving accuracy in binary classification problems.
method Best-scored random forest: selects the best-performing tree from random candidates.
result The best-scored random forest can be more accurate than the original random forest.
Local Linear Forests improve random forests for smooth signals and causal inference.
problem Random forests struggle with smooth signals and poor predictive performance in smooth effects.
method Pairing forest kernel with local linear regression adjustment.
result Improves asymptotic rates of convergence and accuracy on real and simulated data.
DDBF improves random forest for imbalanced data.
problem Learning from imbalanced data.
method Incorporates hard example mining into random forest, dynamically removing easy examples.
result DDBF outperforms random forest on multiple datasets.
RPSF improves shapelet-based random forest accuracy and speed.
problem Limitations of random shapelet forest, including high training cost and limited information per node.
method Combines pairs of shapelets, omits threshold searching, and uses DMDI for interpretability.
result Improves accuracy and training speed of shapelet-based random forest.
Many scientific and engineering challenges -- ranging from personalized medicine to customized marketing recommendations -- require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely us…
Decision forests, including Random Forests and Gradient Boosting Trees, have recently demonstrated state-of-the-art performance in a variety of machine learning settings. Decision forests are typically ensembles of axis-aligned decision trees; that is, trees that split only along feature dimensions. In contrast, many r…
A fast method for finding counterfactual explanations for decision forests.
problem Finding counterfactual explanations for decision forests efficiently.
method Constrain optimization to data-populated regions, reducing to nearest-neighbor search.
result Very fast and realistic counterfactual explanations for decision forests.
Improves random survival forest model by weighted averaging.
problem Improving the performance of random survival forest.
method Modifies random forest by weighted averaging of trees, optimizing weights via quadratic optimization to maximize Harrell's C-index.
result The weighted random survival forest outperforms the original model in numerical examples.
We improve random forest consistency and performance with DMRF, a new variant.
problem Improving the consistency and performance of random forest models.
method Developed DMRF, a data-driven multinomial random forest, by modifying proof methods and improving data utilization.
result DMRF achieves strong consistency with probability 1, surpassing previous models in classification tasks.
Rotation forest is superior to other classifiers for problems with continuous features.
problem Classifying problems with real-valued features.
method Empirical comparison of classifiers from three families: SVM, tree-based ensembles, and neural networks.
result Rotation forest is significantly more accurate than competing techniques on average.
Proves bounds on spanning two-forests and random cut sizes.
problem Counting spanning two-forests and estimating random cut sizes.
method Uses pairwise effective resistances and potential theory.
result Establishes bounds on the number of spanning two-forests and average cut size.
Paper explores grafting consistent estimators to improve Random Forest consistency.
problem Ensuring Random Forests are consistent despite their performance.
method Grafting consistent estimators onto a shallow CART.
result Grafted estimators provide a consistency guarantee and perform well empirically.
Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under which their predictions are consistent, but these results do not provide practical estimates of ran…
Transforms random forests into efficient neural networks using imitation learning.
problem Inefficient architectures of existing methods for transforming random forests into neural networks.
method Generates training data from a random forest and learns a neural network to imitate its behavior.
result Implicit transformation creates efficient neural networks with better generalization.
Theoretical study of random forests for nonlinear time series.
problem Theoretical justification for using random forests in time series modeling.
method Uniform concentration inequality for regression trees and random forests consistency proof.
result Consistency of random forests for nonlinear autoregressive processes.
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
MORF improves Forests' performance on manifold data by considering feature indices.
problem Forest methods struggle with structured data like images and text.
method MORF incorporates feature locality by sampling random matrices from manifold-aware distributions.
result MORF outperforms ConvNets and other methods on manifold data.
New methods estimate survival functions with time-varying covariates.
problem Estimating survival functions with time-varying covariates.
method Generalized conditional inference and relative risk forests, adapted transformation forest.
result Proposed methods outperform traditional models in estimating survival functions.
Alpha-trimming prunes trees in random forests to improve predictive performance.
problem Improving predictive performance of random forests by locally adaptive tree pruning.
method Alpha-trimming is a fast pruning algorithm that prunes trees in a random forest based on signal-to-noise ratio, controlled by a tuning parameter.
result Alpha-trimming often lowers mean squared prediction error compared to fully grown random forests.
We study graph estimation and density estimation in high dimensions, using a family of density estimators based on forest structured undirected graphical models. For density estimation, we do not assume the true distribution corresponds to a forest; rather, we form kernel density estimates of the bivariate and univaria…
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
GrCAN combines autoencoder and neural decision forest for improved classification.
problem Combining robust random forest and deep neural network advantages for high-dimensional data.
method Gradient Boost Convolutional Autoencoder with Neural Decision Forest.
result GrCAN achieves good efficiency and prediction performance compared to baseline methods.
Forest-based kernels improve on existing methods for high-dimensional testing.
problem Developing interpretable kernels for high-dimensional data.
method Kernel Mean Embedding Random Forests (KMERF) using leaf-node proximity.
result KMERF kernels are asymptotically characteristic and outperform existing methods.
Random forests are a type of ensemble method which makes predictions by combining the results of several independent trees. However, the theory of random forests has long been outpaced by their application. In this paper, we propose a novel random forests algorithm based on cooperative game theory. Banzhaf power index …
Study uses satellite and lidar data to map forest height and biomass in France.
problem Mapping forest resources and carbon in large areas.
method Machine learning approach using Sentinel-1, Sentinel-2, ALOS-2, and GEDI Lidar data.
result High-resolution maps of forest height and biomass produced with good accuracy.
Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to lar…