This paper is a comment on the survey paper by Biau and Scornet (2016) about random forests. We focus on the problem of quantifying the impact of each ingredient of random forests on their performance. We show that such a quantification is possible for a simple pure forest , leading to conclusions that could apply more…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Generative Forests”
GeFs use deep generative models to enhance prediction robustness and uncertainty.
problem Lack of principled methods to manipulate uncertainty in decision trees and random forests.
method Exploits Generative Forests (GeFs), a deep probabilistic model that extends Random Forests to represent full joint distributions.
result GeFs are uncertainty-aware classifiers capable of measuring robustness and detecting out-of-distribution samples.
This paper improves forest pruning to balance accuracy and interpretability.
problem Limited interpretability of regression forests.
method Lasso-pruning and theoretical analysis of regression forests.
result Pruned regression forests can achieve equal or better accuracy than unpruned ones, with significant size reduction.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge…
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
New methods estimate survival functions with time-varying covariates.
problem Estimating survival functions with time-varying covariates.
method Generalized conditional inference and relative risk forests, adapted transformation forest.
result Proposed methods outperform traditional models in estimating survival functions.
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
Many scientific and engineering challenges -- ranging from personalized medicine to customized marketing recommendations -- require an understanding of treatment effect heterogeneity. In this paper, we develop a non-parametric causal forest for estimating heterogeneous treatment effects that extends Breiman's widely us…
We derive Gaussian approximations for random forest predictions using region-based stabilization.
problem Improving the accuracy of random forest predictions for Poisson process data.
method Region-based stabilization and Malliavin-Stein method for multivariate Gaussian approximation.
result Established Gaussian approximation bounds for random forest predictions under Poisson process.
iMondrian forest combines isolation forest and Mondrian forest for better anomaly detection.
problem Anomaly detection in batch and online settings.
method Hybrid of isolation forest and Mondrian forest, using depth in Mondrian forest structure.
result iMondrian forest outperforms existing methods in batch and online settings.
We present Neural Random Forest Imitation - a novel approach for transforming random forests into neural networks. Existing methods propose a direct mapping and produce very inefficient architectures. In this work, we introduce an imitation learning approach by generating training data from a random forest and learning…
Random forest training yields confidence intervals for generalization error.
problem Computing accurate confidence intervals for random forest generalization error.
method Directly computes confidence intervals from training data without data splitting.
result Confidence intervals have good coverage and appropriate width.
We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…
Random forest is widely exploited as an ensemble learning method. In many practical applications, however, there is still a significant challenge to learn from imbalanced data. To alleviate this limitation, we propose a deep dynamic boosted forest (DDBF), a novel ensemble algorithm that incorporates the notion of hard …
Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statisticsstat.ML
Random forests remain among the most popular off-the-shelf supervised learning algorithms. Despite their well-documented empirical success, however, until recently, few theoretical results were available to describe their performance and behavior. In this work we push beyond recent work on consistency and asymptotic no…
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for r…
Improved accuracy in machine learning with Cross-Cluster Weighted Forests.
problem Improving accuracy in machine learning algorithms for datasets with clusters.
method Ensembling Random Forest learners trained on clusters determined by k-means.
result Significant improvements in accuracy and generalizability over traditional Random Forest.
Local Linear Forestsstat.ML
Random forests are a powerful method for non-parametric regression, but are limited in their ability to fit smooth signals, and can show poor predictive performance in the presence of strong, smooth effects. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local li…
Projected random forests improve circular data prediction with adaptive arc length and finite-sample coverage.
problem Regression with circular responses.
method Adapting linear-response models to circular data using projection and random forest out-of-bag mechanism.
result Projected random forest out-of-bag conformal prediction sets are more efficient and shorter than alternative methods.
The paper develops a theory for random forests, separating variance components and providing methods for estimating prediction intervals.
problem Understanding the variance and uncertainty in random forest predictions.
method Design-based theory, Monte Carlo averaging, PASR resampling.
result The floor of prediction uncertainty is positive and persists even without observation overlap, providing conservative prediction intervals.
arfpy simplifies data generation with adversarial random forests.
problem Synthesizing new data that matches given data.
method Adversarial Random Forests (ARF) integrated into a Python package.
result Effective and user-friendly data generation for various fields.
RFpredInterval: An R Package for Prediction Intervals with Random Forests and Boosted Forestsstat.ML
RFpredInterval package builds prediction intervals for random forests and boosted forests.
problem Quantifying uncertainty in random forest and boosted forest point predictions.
method 16 methods to build prediction intervals with random forests and boosted forests.
result The proposed method outperforms existing methods in building prediction intervals.
Improved random forest proximities capture data geometry.
problem Inaccurate random forest proximities do not reflect learned data geometry.
method Introduce RF-GAP: Geometry- and Accuracy-Preserving proximities.
result RF-GAP improves geometric representation in tasks like data imputation.
A new imputation method MissARF uses adversarial random forests for fast and accurate missing value imputation.
problem Handling missing values in biostatistical analyses.
method Adversarial Random Forests (ARF) for density estimation and data synthesis.
result MissARF performs comparably to state-of-the-art methods in imputation quality and runtime.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
New method improves feature importance assessment in random forests.
problem Improving feature importance measures for random forests.
method Hypothesis testing via self-normalized feature-residual correlation test (FACT).
result The method provides theoretically justified feature importance tests with controlled type I error and appealing power.
SARF improves stock market prediction by integrating sentiment analysis.
problem Enhancing stock market prediction accuracy with sentiment data.
method Sentiment-Augmented Random Forest (SARF) using FinGPT.
result SARF outperforms conventional models with 9.23% accuracy improvement.
Deep forests enhance expressiveness exponentially with depth, not width or tree size.
problem Understanding the role of depth, width, and tree size in deep forest performance.
method Provided upper and lower bounds on deep forest approximation complexity.
result Depth exponentially enhances deep forest expressiveness.
Forest-based methods estimate heterogeneous treatment effects, blending strengths for better performance.
problem Estimating heterogeneous treatment effects in randomized and observational studies.
method Causal forests and model-based forests, blending strengths for better performance.
result Local centering of treatment indicator and propensities is crucial for good performance in randomized trials.
Random forest (Leo Breiman 2001a) (RF) is a non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF is a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. Random survival forests (RSF) …
Random forests are a statistical learning method widely used in many areas of scientific research because of its ability to learn complex relationships between input and output variables and also its capacity to handle high-dimensional data. However, current random forest approaches are not flexible enough to handle he…
Novel unsupervised random forests improve density estimation and data synthesis.
problem Density estimation and data synthesis for complex tabular data.
method Recursive unsupervised random forests with alternating generation and discrimination rounds.
result Provable consistency and smooth densities with fast execution.
Forest Guided Smoothingstat.ML
Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
In this paper we propose using the principle of boosting to reduce the bias of a random forest prediction in the regression setting. From the original random forest fit we extract the residuals and then fit another random forest to these residuals. We call the sum of these two random forests a \textit{one-step boosted …
Random forests perform bootstrap-aggregation by sampling the training samples with replacement. This enables the evaluation of out-of-bag error which serves as a internal cross-validation mechanism. Our motivation lies in using the unsampled training samples to improve each decision tree in the ensemble. We study the e…
Research compares ML and Time Series methods for generating trading signals.
problem Efficiency of on-line learning Algorithms in generating trading signals.
method Used technical indicators and ensemble of Random Forests, also Kalman Filter.
result Kalman Filter outperformed Random Forests in on-line learning predictions of stock prices.
Financial fraud detection system based on improved random forest and gradient boosting machine (GBM)q-fin.ST
Paper proposes a new financial fraud detection system using improved RF and GBM.
problem Detecting financial fraud with high accuracy and efficiency.
method Introduces GBM-SSRF, a novel model combining GBM and SSRF.
result GBM-SSRF model outperforms traditional methods in fraud detection.
New methods improve prediction performance and reduce computation time in boosting and random forest models.
problem Improving prediction performance and reducing computation time in boosting and random forest models.
method Random tree depth injection approach for Boosting and Random Forests.
result The new methods can improve prediction performance and reduce computation time by up to 40%.
Improved GAS models using trees and forests for better forecasts.
problem Improving forecasts from GAS models to avoid curse of dimensionality.
method Localized parameters using decision trees and random forests.
result Significantly outperform baseline GAS model in empirical analyses.
Generalized Random Forestsstat.ME
We propose generalized random forests, a method for non-parametric statistical estimation based on random forests (Breiman, 2001) that can be used to fit any quantity of interest identified as the solution to a set of local moment equations. Following the literature on local maximum likelihood estimation, our method co…
By seeking the narrowest prediction intervals (PIs) that satisfy the specified coverage probability requirements, the recently proposed quality-based PI learning principle can extract high-quality PIs that better summarize the predictive certainty in regression tasks, and has been widely applied to solve many practical…
Information Forestscs.LG
We describe Information Forests, an approach to classification that generalizes Random Forests by replacing the splitting criterion of non-leaf nodes from a discriminative one -- based on the entropy of the label distribution -- to a generative one -- based on maximizing the information divergence between the class-con…
Improves time series classification with forest proximities.
problem Time series classification accuracy and efficiency.
method PF-GAP, an extension of RF-GAP proximities to proximity forests, combined with Multi-Dimensional Scaling and Local Outlier Factors.
result Forest proximities show stronger connection between misclassified points and outliers.
Forest tree species mapped with high accuracy using satellite data.
problem Classifying dominant tree species in Swedish forests.
method Extreme gradient boosting model with Bayesian optimization, combining Sentinel-1/2 satellite data and field observations.
result Overall accuracy of 85%, F1 score of 0.82, Matthews correlation coefficient of 0.81.
Extends RF proximities to all supervised distance-based machine learning contexts.
problem Limited utility of RF proximities in various machine learning tasks.
method Introduces generalized Proximity Forest (PF) model and variant for regression.
result Demonstrates unique advantages over RF and k-nearest neighbors models.
This paper presents a basic property of region dividing of ReLU (rectified linear unit) deep learning when new layers are successively added, by which two new perspectives of interpreting deep learning are given. The first is related to decision trees and forests; we construct a deep learning structure equivalent to a …
Random forests reduce bias and variance, especially in low SNR settings.
problem Reducing bias and variance in machine learning models, particularly in low SNR scenarios.
method Empirical study of random forests and bagging ensembles, focusing on the importance of tuning.
result Random forests reduce both bias and variance, outperforming bagging ensembles in high SNR settings.