A novel unsupervised outlier detection method using Randomized PCA Forest.
problem Unsupervised outlier detection in datasets.
method Randomized Principal Component Analysis (RPCA) Forest for deriving an outlier score.
result Superior performance compared to classical and state-of-the-art methods.
We present Random Partition Kernels, a new class of kernels derived by demonstrating a natural connection between random partitions of objects and kernels between those objects. We show how the construction can be used to create kernels from methods that would not normally be viewed as random partitions, such as Random…
Paper proposes a new descriptor for early trajectory characterization in matrix iterations.
problem Comparing early behavior of high-dimensional trajectories in nonlinear matrix iterations.
method Develops a two-channel fuzzy coordinate system using F-transform for compact representation.
result The descriptor achieves high R^2 values (mean = 0.6480) in approximating convergence lengths.
Study improves traffic prediction intervals for minor roads.
problem Uncertainty in traffic data for underrepresented minor roads.
method Quantile Random Forest with PCA for interval prediction.
result Achieved 88.22% interval coverage and Winkler Score of 7,468.47.
Bayesian calibration speeds up ABM for pandemic modeling.
problem Calibrating stochastic ABMs for accurate pandemic predictions is computationally intensive.
method Random forest surrogate modeling for accelerated ABM evaluation.
result Improved predictive performance with random forest calibration compared to previous methods.
Study uses random forest to detect unlawful insider trading in financial data.
problem Detecting and identifying unlawful insider trading in complex financial data.
method Integrates PCA-RF and standalone RF models with semi-manually labeled transactions.
result 96.43% accurate classification of transactions, 95.47% lawful, 98.00% unlawful.
Ensembles of randomized decision trees, usually referred to as random forests, are widely used for classification and regression tasks in machine learning and statistics. Random forests achieve competitive predictive performance and are computationally efficient to train and test, making them excellent candidates for r…
This paper uses PCA and FA for feature selection in credit rating.
problem Selecting important features for credit rating prediction.
method Principal Component Analysis and Factor Analysis.
result Factor Analysis reduces feature set significantly without losing much accuracy.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
Improved random forest proximities capture data geometry.
problem Inaccurate random forest proximities do not reflect learned data geometry.
method Introduce RF-GAP: Geometry- and Accuracy-Preserving proximities.
result RF-GAP improves geometric representation in tasks like data imputation.
We propose random hinge forests, a simple, efficient, and novel variant of decision forests. Importantly, random hinge forests can be readily incorporated as a general component within arbitrary computation graphs that are optimized end-to-end with stochastic gradient descent or variants thereof. We derive random hinge…
Random forests reduce bias and variance, especially in low SNR settings.
problem Reducing bias and variance in machine learning models, particularly in low SNR scenarios.
method Empirical study of random forests and bagging ensembles, focusing on the importance of mtry tuning. result Random forests reduce both bias and variance, outperforming bagging ensembles in high SNR settings.
Improved random forest models enhance machine learning predictions.
problem Equal weights for random forest base decision trees are not optimal.
method Proposes algorithms to modify weighting strategy of regular random forest.
result Numerical results show significant improvements over regular random forest.
Enhances random forest consistency and introduces DMRF for improved performance.
problem Improving the consistency and efficiency of random forest algorithms.
method Strengthened proof methods and propose DMRF algorithm.
result DMRF achieves better theoretical and experimental performance than previous variants.
New random forest variants achieve optimal performance in high dimensions.
problem Handling dependencies between features in high-dimensional data.
method Using oblique splits in random forests with general split directions.
result Achieved minimax optimal convergence rates in arbitrary dimension.
This paper is a comment on the survey paper by Biau and Scornet (2016) about random forests. We focus on the problem of quantifying the impact of each ingredient of random forests on their performance. We show that such a quantification is possible for a simple pure forest , leading to conclusions that could apply more…
RFpredInterval package builds prediction intervals for random forests and boosted forests.
problem Quantifying uncertainty in random forest and boosted forest point predictions.
method 16 methods to build prediction intervals with random forests and boosted forests.
result The proposed method outperforms existing methods in building prediction intervals.
One technique to visualize the training of neural networks is to perform PCA on the parameters over the course of training and to project to the subspace spanned by the first few PCA components. In this paper we compare this technique to the PCA of a high dimensional random walk. We compute the eigenvalues and eigenvec…
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
We improve random forest consistency and performance with DMRF, a new variant.
problem Improving the consistency and performance of random forest models.
method Developed DMRF, a data-driven multinomial random forest, by modifying proof methods and improving data utilization.
result DMRF achieves strong consistency with probability 1, surpassing previous models in classification tasks.
Online random forests improve Q-learning performance in specific gym environments.
problem Improving Q-learning performance in reinforcement learning tasks.
method Proposed online random forests as Q-function approximators and growing them as learning progresses.
result Improved performance over state-of-the-art Deep Q-Networks in specific gym environments.
Paper explores grafting consistent estimators to improve Random Forest consistency.
problem Ensuring Random Forests are consistent despite their performance.
method Grafting consistent estimators onto a shallow CART.
result Grafted estimators provide a consistency guarantee and perform well empirically.
Random forests have proven to be reliable predictive algorithms in many application areas. Not much is known, however, about the statistical properties of random forests. Several authors have established conditions under which their predictions are consistent, but these results do not provide practical estimates of ran…
Transforms random forests into efficient neural networks using imitation learning.
problem Inefficient architectures of existing methods for transforming random forests into neural networks.
method Generates training data from a random forest and learns a neural network to imitate its behavior.
result Implicit transformation creates efficient neural networks with better generalization.
Theoretical study of random forests for nonlinear time series.
problem Theoretical justification for using random forests in time series modeling.
method Uniform concentration inequality for regression trees and random forests consistency proof.
result Consistency of random forests for nonlinear autoregressive processes.
We propose an algorithm named best-scored random forest for binary classification problems. The terminology "best-scored" means to select the one with the best empirical performance out of a certain number of purely random tree candidates as each single tree in the forest. In this way, the resulting forest can be more …
In this paper we propose using the principle of boosting to reduce the bias of a random forest prediction in the regression setting. From the original random forest fit we extract the residuals and then fit another random forest to these residuals. We call the sum of these two random forests a \textit{one-step boosted …
Alpha-trimming prunes trees in random forests to improve predictive performance.
problem Improving predictive performance of random forests by locally adaptive tree pruning.
method Alpha-trimming is a fast pruning algorithm that prunes trees in a random forest based on signal-to-noise ratio, controlled by a tuning parameter.
result Alpha-trimming often lowers mean squared prediction error compared to fully grown random forests.
New methods improve prediction performance and reduce computation time in boosting and random forest models.
problem Improving prediction performance and reducing computation time in boosting and random forest models.
method Random tree depth injection approach for Boosting and Random Forests.
result The new methods can improve prediction performance and reduce computation time by up to 40%.
The current study proposes a dimension reduction method, stepwise support vector machine (SVM), to reduce the dimensions of large p small n datasets. The proposed method is compared with other dimension reduction methods, namely, the Pearson product difference correlation coefficient (PCCs), recursive feature eliminati…
Random forests are a type of ensemble method which makes predictions by combining the results of several independent trees. However, the theory of random forests has long been outpaced by their application. In this paper, we propose a novel random forests algorithm based on cooperative game theory. Banzhaf power index …
A random forest is a popular tool for estimating probabilities in machine learning classification tasks. However, the means by which this is accomplished is unprincipled: one simply counts the fraction of trees in a forest that vote for a certain class. In this paper, we forge a connection between random forests and ke…
Despite widespread interest and practical use, the theoretical properties of random forests are still not well understood. In this paper we contribute to this understanding in two ways. We present a new theoretically tractable variant of random regression forests and prove that our algorithm is consistent. We also prov…
Paper proposes a new method combining random forests and Lasso selection.
problem Improving random forest performance by applying Lasso regression.
method Adaptive Lasso weighting applied to random forest predictions.
result Unified framework strictly outperforms other methods in simulations and real-world datasets.
Develops a new random forest method for clustered data with improved prediction and inference.
problem Improving prediction and inference accuracy for clustered data with within-cluster dependence.
method Clustered Random Forests, using weighted least squares estimators for leaf predictions.
result Optimal prediction and inference weights vary under covariate shift, necessitating user-chosen weights.
Proves bounds on spanning two-forests and random cut sizes.
problem Counting spanning two-forests and estimating random cut sizes.
method Uses pairwise effective resistances and potential theory.
result Establishes bounds on the number of spanning two-forests and average cut size.
We optimize large Random Forests into faster, smaller decision diagrams.
problem Efficiency and size of large Random Forests.
method Aggregating large Random Forests into a single, semantically equivalent decision diagram.
result Significant speed-ups and reduction in data structure size.
Shapelet is a discriminative subsequence of time series. An advanced shapelet-based method is to embed shapelet into accurate and fast random forest. However, it shows several limitations. First, random shapelet forest requires a large training cost for split threshold searching. Second, a single shapelet provides limi…
This paper explains CART random forests using stochastic control theory.
problem Understanding the inner workings of CART random forests.
method Developed a stochastic-control perspective on CART random forests, interpreting feature subsampling as a random feasible action set and the split rule as a policy.
result Established that the CART policy is locally stabilizing but globally suboptimal for the forest objective.
Forest-guided smoothing uses random forest outputs for interpretable local smoothers.
problem Creating interpretable local smoothers from complex random forest outputs.
method Uses random forest outputs to define spatially adaptive bandwidth matrices for a linear smoother.
result Improves interpretability and applicability of random forest outputs for various analyses.
Paper assesses error estimates of Random Forests classification.
problem Quantitative assessment of Random Forests error estimates.
method Theoretical and empirical investigation of various error estimation methods.
result Random Forests' error estimates are closer to true error rate than average prediction error.
Random forests classify Pokemon names based on evolutionary status.
problem Classifying Pokemon names into pre-evolution and post-evolution categories.
method Trained random forests using sound symbolism from Pokemon names, conducted experiments with human participants.
result Random forests outperform human participants in classifying Pokemon names.
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an ex…
As a testament to their success, the theory of random forests has long been outpaced by their application in practice. In this paper, we take a step towards narrowing this gap by providing a consistency result for online random forests.
Random forest is widely exploited as an ensemble learning method. In many practical applications, however, there is still a significant challenge to learn from imbalanced data. To alleviate this limitation, we propose a deep dynamic boosted forest (DDBF), a novel ensemble algorithm that incorporates the notion of hard …
A weighted random survival forest is presented in the paper. It can be regarded as a modification of the random forest improving its performance. The main idea underlying the proposed model is to replace the standard procedure of averaging used for estimation of the random survival forest hazard function by weighted av…
Random forests are stable and provide reliable prediction intervals.
problem Stability and reliability of random forest prediction intervals.
method Established stability under mild conditions and proved coverage bounds.
result Non-asymptotic lower and upper bounds for prediction interval coverage.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.