A new type of distributional regression tree uses soft split rules for better predictive performance.
problem Estimating complete conditional distributions in regression.
method Distributional adaptive soft regression trees using multivariate soft split rules.
result The method outperforms various benchmark methods, especially in complex non-linear interactions.
Five simple soft sensor methodologies with two update conditions were compared on two experimentally-obtained datasets and one simulated dataset. The soft sensors investigated were moving window partial least squares regression (and a recursive variant), moving window random forest regression, the mean moving window of…
This paper improves deep forest models with soft routing and topology learning.
problem Expensive computational costs and lack of interpretability in deep neural networks.
method Soft routing in probabilistic trees and topology learning for joint optimization.
result Empowered deep forests achieve better performance with reduced model complexity.
New framework optimizes decisions under uncertainty considering causal and continuous data.
problem Optimizing decisions under uncertain distributions with causal and continuous data structures.
method Developed a framework using Causal Sinkhorn DRO with Soft Regression Forest decision rules.
result Framework provides interpretable and tractable decision rules for optimizing under uncertainty.
To deal with various datasets over different complexity, this paper presents an self-adaptive learning model that combines the proposed Dynamic Connected Neural Decision Networks (DNDN) and a new pruning method--Dynamic Soft Pruning (DSP). DNDN is a combination of random forests and deep neural networks that enjoys bot…
ASBART accelerates Soft BART for faster Bayesian regression.
problem Slow computation in Soft BART.
method Proposed ASBART, a variant of Soft BART.
result ASBART is about 10 times faster than Soft BART with similar accuracy.
Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
This paper improves forest pruning to balance accuracy and interpretability.
problem Limited interpretability of regression forests.
method Lasso-pruning and theoretical analysis of regression forests.
result Pruned regression forests can achieve equal or better accuracy than unpruned ones, with significant size reduction.
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…
Regression, unlike classification, has lacked a comprehensive and effective approach to deal with cost-sensitive problems by the reuse (and not a re-training) of general regression models. In this paper, a wide variety of cost-sensitive problems in regression (such as bids, asymmetric losses and rejection rules) can be…
Quantile regression using random forest proximities improves prediction and uncertainty quantification.
problem Forecasting corporate bond volume with uncertainty quantification.
method Introduced a novel approach to compute quantile regressions from random forests using proximity metrics.
result Superior performance in approximating conditional target distributions and prediction intervals.
Study uses regression and ML for COVID-19 mortality forecasting.
problem Forecasting COVID-19 mortality during the first wave in Spain.
method Cyclical curve log-regression, multivariate time series spatial residual correlation analysis, Bayesian approach, machine learning.
result Empirical analysis shows ML regression models perform better than traditional methods.
Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to lar…
Mondrian random forests improve statistical inference for regression.
problem Improving statistical inference for regression functions.
method Combining bias and variance characterizations with debiasing and variance estimation.
result Valid statistical inference methods for unknown regression functions with error bounds.
The paper uses regression trees/random forests to price Bermudan options more efficiently.
problem Pricing Bermudan options with conditional expectation estimation.
method Estimates conditional expectations using regression trees or random forests instead of traditional regression methods.
result Regression trees/random forests provide better results in high dimensions.
Random forests are powerful non-parametric regression method but are severely limited in their usage in the presence of randomly censored observations, and naively applied can exhibit poor predictive performance due to the incurred biases. Based on a local adaptive representation of random forests, we develop its regre…
By seeking the narrowest prediction intervals (PIs) that satisfy the specified coverage probability requirements, the recently proposed quality-based PI learning principle can extract high-quality PIs that better summarize the predictive certainty in regression tasks, and has been widely applied to solve many practical…
Random forests are powerful non-parametric regression method but are severely limited in their usage in the presence of randomly censored observations, and naively applied can exhibit poor predictive performance due to the incurred biases. Based on a local adaptive representation of random forests, we develop its regre…
New random forest algorithm improves regression with missing data.
problem Regression with missing data values.
method New random forest algorithm compared to existing techniques.
result Improved performance in quadratic errors and bias compared to existing methods.
Random forests with attention and self-attention improve regression performance.
problem Improving regression model performance on various datasets.
method Proposes new models using attention and self-attention mechanisms to solve regression problems.
result The models improve model performance on many datasets.
TrIM improves gradient-based dimension reduction and regression.
problem Efficiently identifying relevant feature subspace for high-dimensional regression.
method Introduced TrIM forest, an iterative approach using Mondrian forest and EGOP estimate.
result Consistency guarantees and convergence rates for EGOP matrix and random forest estimator.
New method improves stability of soft FQI for offline RL.
problem Stability issues in soft FQI under function approximation.
method Stationary reweighting to align operator norms.
result Local linear convergence proved under certain conditions.
New random forest method provides optimal rates and confidence bands.
problem Improving random forest regression rates and constructing confidence bands.
method Proposed Ehrenfest centered purely random forests achieve optimal rates; used Gaussian approximation for supremum of empirical processes.
result Explicit asymptotic uniform confidence bands constructed for both random forest types.
Proposes a new random forest weighted local Fréchet regression method.
problem Complex metric space valued responses and curse of dimensionality in Fréchet regression.
method Locally adaptive kernel generated by random forests for local average and local linear Fréchet regression.
result Significantly improves existing Fréchet regression methods with theoretical guarantees.
We analyze the consistency of decision trees and random forests in regression.
problem Consistency of decision trees and random forests in regression.
method Elementary proofs following classical arguments for smoothing methods.
result Establish weak and almost sure convergence of honest trees and forest averages to the true regression function.
Random forests are a powerful method for non-parametric regression, but are limited in their ability to fit smooth signals, and can show poor predictive performance in the presence of strong, smooth effects. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local li…
Proposes a method to estimate drug sensitivity uncertainty using deep regression forests.
problem Lack of confidence intervals in deep learning models for critical tasks.
method Uses Deep Regression Forests to estimate variance and uncertainty for drug sensitivity prediction.
result Improves efficiency and coverage of uncertainty estimates for drug sensitivity predictions.
Improved random forest proximities capture data geometry.
problem Inaccurate random forest proximities do not reflect learned data geometry.
method Introduce RF-GAP: Geometry- and Accuracy-Preserving proximities.
result RF-GAP improves geometric representation in tasks like data imputation.
Paper introduces a medoid-based approach for efficient Fréchet regression.
problem Regression in metric spaces with random objects.
method Adapted random forest algorithm with medoid-based splitting rule.
result Asymptotic equivalence and consistency of the regression estimator.
Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…
We improve random forest consistency and performance with DMRF, a new variant.
problem Improving the consistency and performance of random forest models.
method Developed DMRF, a data-driven multinomial random forest, by modifying proof methods and improving data utilization.
result DMRF achieves strong consistency with probability 1, surpassing previous models in classification tasks.
Random forest (RF) methodology is one of the most popular machine learning techniques for prediction problems. In this article, we discuss some cases where random forests may suffer and propose a novel generalized RF method, namely regression-enhanced random forests (RERFs), that can improve on RFs by borrowing the str…
Random Forests [Breiman:2001] (RF) are a fully non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF are a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. The randomForestSRC packag…
Time series quantile regression using GRF for more accurate volatility estimation.
problem Estimating conditional quantiles for time series data accurately.
method Generalized Random Forests (GRF) for quantile regression on time series data.
result The tsQRF estimator is consistent under time series data assumptions.
Paper proposes real-time VaR estimation using quantile regression forest with conformal calibration.
problem Real-time estimation of Value at Risk (VaR) in rapidly changing markets.
method Quantile regression forest trained offline, real-time VaR estimates via observed risk factors, conformalized estimator for reliability.
result The proposed method provides reliable real-time VaR estimates.
Develops new algorithms for QRF to handle mixed-frequency and longitudinal data.
problem Handling mixed-frequency and longitudinal data in quantile regression.
method Mixed-Frequency Quantile Regression Forest (MIDAS-QRF) and Finite Mixture Quantile Regression Forest (FM-QRF).
result Valid and flexible models for complex empirical settings in financial risk management and climate-change impact evaluation.
CovRegRF estimates covariance matrix from covariates using random forests.
problem Estimating conditional covariances or correlations among multivariate responses.
method Random forest trees with a custom splitting rule to maximize covariance difference.
result Accurate covariance matrix estimates and controlled Type-1 error.
Random forest models systematically bias predictions; a numerical transform corrects this.
problem Systematic bias in random forest regression predictions.
method Defined a numerical transformation using training data to correct bias.
result Improved predictions across real-world and synthetic datasets.
Random Planted Forest interprets tree-based models by keeping some splits, leading to more interpretable predictions.
problem Estimating the unknown regression function from lower-order interaction terms.
method Modifying the random forest algorithm by keeping certain leaves instead of deleting them, resulting in non-binary trees called planted trees.
result The random planted forest achieves asymptotically optimal convergence rates up to a logarithmic factor when the interaction bound is low.
Paper proposes a new method combining random forests and Lasso selection.
problem Improving random forest performance by applying Lasso regression.
method Adaptive Lasso weighting applied to random forest predictions.
result Unified framework strictly outperforms other methods in simulations and real-world datasets.
E2Tree explains random forest models in regression tasks.
problem Lack of transparency in random forest models.
method E2Tree extends random forest to regression by explaining model predictions through graphical representation and dissimilarity measures.
result E2Tree provides a transparent explanation of random forest models in regression tasks.
Random forests is a common non-parametric regression technique which performs well for mixed-type data and irrelevant covariates, while being robust to monotonic variable transformations. Existing random forest implementations target regression or classification. We introduce the RFCDE package for fitting random forest…
We characterize and study variable importance (VIMP) and pairwise variable associations in binary regression trees. A key component involves the node mean squared error for a quantity we refer to as a maximal subtree. The theory naturally extends from single trees to ensembles of trees and applies to methods like rando…
Paper proposes a new descriptor for early trajectory characterization in matrix iterations.
problem Comparing early behavior of high-dimensional trajectories in nonlinear matrix iterations.
method Develops a two-channel fuzzy coordinate system using F-transform for compact representation.
result The descriptor achieves high R^2 values (mean = 0.6480) in approximating convergence lengths.
Paper studies ensemble probabilistic regression trees for smooth approximations.
problem Smooth approximations of regression functions.
method Ensemble versions of probabilistic regression trees.
result Ensemble probabilistic regression trees are consistent and perform well.
Bayesian GBMs improve predictive uncertainty calibration for tabular data.
problem Lack of well-calibrated predictive uncertainties in gradient boosting machines.
method Variational inference with soft decision trees.
result Variational soft GBMs provide useful uncertainty estimates and maintain good predictive performance.