HDI-Forest improves regression prediction intervals using Random Forest.
problem Improving the quality of prediction intervals in regression tasks.
method HDI-Forest is a novel quality-based PI estimation method based on Random Forest, optimizing PI quality metrics directly from standard tree-based models.
result HDI-Forest significantly reduces PI width by over 20% compared to previous methods, while maintaining or improving coverage probability.
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
We investigate the effect of the proportional hazards assumption on prognostic and predictive models of the survival time of patients suffering from amyotrophic lateral sclerosis (ALS). We theoretically compare the underlying model formulations of several variants of survival forests and implementations thereof, includ…
New method learns representations for decision forests using input perturbation.
problem Decision forests struggle with raw structured data and lack effective representations.
method Approximate decision forest gradients through input perturbation.
result Effective representation learning for decision forests without structural changes.
RFpredInterval package builds prediction intervals for random forests and boosted forests.
problem Quantifying uncertainty in random forest and boosted forest point predictions.
method 16 methods to build prediction intervals with random forests and boosted forests.
result The proposed method outperforms existing methods in building prediction intervals.
GeFs use deep generative models to enhance prediction robustness and uncertainty.
problem Lack of principled methods to manipulate uncertainty in decision trees and random forests.
method Exploits Generative Forests (GeFs), a deep probabilistic model that extends Random Forests to represent full joint distributions.
result GeFs are uncertainty-aware classifiers capable of measuring robustness and detecting out-of-distribution samples.
Theoretical study of random forests for nonlinear time series.
problem Theoretical justification for using random forests in time series modeling.
method Uniform concentration inequality for regression trees and random forests consistency proof.
result Consistency of random forests for nonlinear autoregressive processes.
Forest-based methods estimate heterogeneous treatment effects, blending strengths for better performance.
problem Estimating heterogeneous treatment effects in randomized and observational studies.
method Causal forests and model-based forests, blending strengths for better performance.
result Local centering of treatment indicator and propensities is crucial for good performance in randomized trials.
Improves time series classification with forest proximities.
problem Time series classification accuracy and efficiency.
method PF-GAP, an extension of RF-GAP proximities to proximity forests, combined with Multi-Dimensional Scaling and Local Outlier Factors.
result Forest proximities show stronger connection between misclassified points and outliers.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
Random forests can be slow or inconsistent in certain models.
problem Performance issues of random forests in specific data-generating models.
method Intuitive arguments and numerical experiments, combined with variable use and importance statistics.
result Simple methods can create a better predictor using a forced random forest.
We study graph estimation and density estimation in high dimensions, using a family of density estimators based on forest structured undirected graphical models. For density estimation, we do not assume the true distribution corresponds to a forest; rather, we form kernel density estimates of the bivariate and univaria…
Despite widespread interest and practical use, the theoretical properties of random forests are still not well understood. In this paper we contribute to this understanding in two ways. We present a new theoretically tractable variant of random regression forests and prove that our algorithm is consistent. We also prov…
Online random forests improve Q-learning performance in specific gym environments.
problem Improving Q-learning performance in reinforcement learning tasks.
method Proposed online random forests as Q-function approximators and growing them as learning progresses.
result Improved performance over state-of-the-art Deep Q-Networks in specific gym environments.
Develops a method to estimate quantiles in censored data using random forests.
problem Inability of random forests to handle randomly censored observations.
method Regression adjustment for quantile regression models based on a new estimating equation.
result Consistent estimation of quantiles without parametric modeling assumptions.
Paper proposes a new financial fraud detection system using improved RF and GBM.
problem Detecting financial fraud with high accuracy and efficiency.
method Introduces GBM-SSRF, a novel model combining GBM and SSRF.
result GBM-SSRF model outperforms traditional methods in fraud detection.
A weighted random survival forest is presented in the paper. It can be regarded as a modification of the random forest improving its performance. The main idea underlying the proposed model is to replace the standard procedure of averaging used for estimation of the random survival forest hazard function by weighted av…
The paper improves random forest models for non-Gaussian responses.
problem Improving random forest models for non-Gaussian responses.
method Extends boosting random forests to model exponential family responses using residuals and weights.
result Generalized boosted forests reduce bias and provide conservative confidence intervals.
Introduced by Breiman, Random Forests are widely used classification and regression algorithms. While being initially designed as batch algorithms, several variants have been proposed to handle online learning. One particular instance of such forests is the \emph{Mondrian Forest}, whose trees are built using the so-cal…
A random forest is a popular tool for estimating probabilities in machine learning classification tasks. However, the means by which this is accomplished is unprincipled: one simply counts the fraction of trees in a forest that vote for a certain class. In this paper, we forge a connection between random forests and ke…
iMondrian forest combines isolation forest and Mondrian forest for better anomaly detection.
problem Anomaly detection in batch and online settings.
method Hybrid of isolation forest and Mondrian forest, using depth in Mondrian forest structure.
result iMondrian forest outperforms existing methods in batch and online settings.
SARF improves stock market prediction by integrating sentiment analysis.
problem Enhancing stock market prediction accuracy with sentiment data.
method Sentiment-Augmented Random Forest (SARF) using FinGPT.
result SARF outperforms conventional models with 9.23% accuracy improvement.
Projected random forests improve circular data prediction with adaptive arc length and finite-sample coverage.
problem Regression with circular responses.
method Adapting linear-response models to circular data using projection and random forest out-of-bag mechanism.
result Projected random forest out-of-bag conformal prediction sets are more efficient and shorter than alternative methods.
Random forest (Leo Breiman 2001a) (RF) is a non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF is a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. Random survival forests (RSF) …
Hybrid model improves forest growth predictions.
problem Misspecified assumptions in mechanistic models.
method Forest Informed Neural Networks (FINN) combining DVM and DNN.
result DNN learned improved growth process functional form.
Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under which their predictions are consistent, but these results do not provide practical estimates of ran…
Transforms random forests into efficient neural networks using imitation learning.
problem Inefficient architectures of existing methods for transforming random forests into neural networks.
method Generates training data from a random forest and learns a neural network to imitate its behavior.
result Implicit transformation creates efficient neural networks with better generalization.
Random forest models systematically bias predictions; a numerical transform corrects this.
problem Systematic bias in random forest regression predictions.
method Defined a numerical transformation using training data to correct bias.
result Improved predictions across real-world and synthetic datasets.
Quantile regression using random forest proximities improves prediction and uncertainty quantification.
problem Forecasting corporate bond volume with uncertainty quantification.
method Introduced a novel approach to compute quantile regressions from random forests using proximity metrics.
result Superior performance in approximating conditional target distributions and prediction intervals.
This paper describes structuring data and constructing plots to explore forest classification models interactively. A forest classifier is an example of an ensemble, produced by bagging multiple trees. The process of bagging and combining results from multiple trees, produces numerous diagnostics which, with interactiv…
Simplifies random forests by breaking down trees into rules for better interpretability.
problem Balancing model complexity and accuracy in random forests.
method Breaking down random forest trees into individual classification rules and selecting a subset.
result A few selected rules can achieve acceptable accuracy similar to the original model, leading to simpler models.
Random forest and deep neural network are two schools of effective classification methods in machine learning. While the random forest is robust irrespective of the data domain, the deep neural network has advantages in handling high dimensional data. In view that a differentiable neural decision forest can be added to…
Random Forests [Breiman:2001] (RF) are a fully non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF are a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. The randomForestSRC packag…
New method improves feature importance assessment in random forests.
problem Improving feature importance measures for random forests.
method Hypothesis testing via self-normalized feature-residual correlation test (FACT).
result The method provides theoretically justified feature importance tests with controlled type I error and appealing power.
Excellent ranking power along with well calibrated probability estimates are needed in many classification tasks. In this paper, we introduce a technique, Calibrated Boosting-Forest that captures both. This novel technique is an ensemble of gradient boosting machines that can support both continuous and binary labels. …
Random forests use randomness to improve model performance in noisy data.
problem Improving model performance in low signal-to-noise ratio settings.
method Demonstrates that randomness in random forests acts as implicit regularization, similar to shrinkage penalties in regularized regression.
result Random forests achieve strong performance by implicitly regularizing model complexity, especially in noisy data.
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for r…
Forest tree species mapped with high accuracy using satellite data.
problem Classifying dominant tree species in Swedish forests.
method Extreme gradient boosting model with Bayesian optimization, combining Sentinel-1/2 satellite data and field observations.
result Overall accuracy of 85%, F1 score of 0.82, Matthews correlation coefficient of 0.81.
The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for survival prediction is random forests for survival outcomes. The standard split criterion for random…
New methods estimate survival functions with time-varying covariates.
problem Estimating survival functions with time-varying covariates.
method Generalized conditional inference and relative risk forests, adapted transformation forest.
result Proposed methods outperform traditional models in estimating survival functions.
A new ensemble learning method called Residual Likelihood Forests improves performance and reduces model size.
problem Improving machine learning classification performance with compact models.
method Sequential optimization of conditional likelihoods in a boosting-like framework, combining multiplicatively.
result Significant performance improvements and reduced model size compared to other ensemble methods.
Random forest is widely exploited as an ensemble learning method. In many practical applications, however, there is still a significant challenge to learn from imbalanced data. To alleviate this limitation, we propose a deep dynamic boosted forest (DDBF), a novel ensemble algorithm that incorporates the notion of hard …
Machine learning identifies chimera states in complex dynamical systems.
problem Chimera states are hard to identify due to their varied appearance and peculiar nature.
method Machine learning techniques, specifically random forest and oblique random forest with null space regularization.
result High accuracy in identifying chimera states across different dynamical models.
Paper proposes a new method combining random forests and Lasso selection.
problem Improving random forest performance by applying Lasso regression.
method Adaptive Lasso weighting applied to random forest predictions.
result Unified framework strictly outperforms other methods in simulations and real-world datasets.
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
Attention augments forest for tabular data accuracy.
problem Training tabular data models with high accuracy and efficiency.
method Tree Attention Block (TAB) in differentiable forest framework.
result Attention augmented differentiable forest achieves comparable and sometimes higher accuracy than GBDT models.
Real world systems typically feature a variety of different dependency types and topologies that complicate model selection for probabilistic graphical models. We introduce the ensemble-of-forests model, a generalization of the ensemble-of-trees model. Our model enables structure learning of Markov random fields (MRF) …