VC-PCR improves prediction by clustering correlated variables.
problem Decreased prediction accuracy due to cluster structure in predictor variables.
method Supervised variable selection and clustering to integrate cluster information into a sparse modeling process.
result VC-PCR achieves better prediction, variable selection, and clustering performance.
The paper proposes a method to stabilize predictions by identifying causal variables using a seed variable.
problem Stable prediction across unknown test data with potential spurious correlations.
method Conditional independence test based algorithm using a seed variable to separate causal from non-causal variables.
result The algorithm precisely separates causal and non-causal variables for stable prediction across test data.
Variable importance is central to scientific studies, including the social sciences and causal inference, healthcare, and other domains. However, current notions of variable importance are often tied to a specific predictive model. This is problematic: what if there were multiple well-performing predictive models, and …
Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables i…
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Study proposes a new method for better price prediction using machine learning and metaheuristics.
problem Challenges in predicting prices due to correlated variables and computational efficiency.
method Introduces a novel decision fusion approach combining Elastic Net and MOPSO for variable selection and prediction.
result The proposed method outperforms traditional approaches in terms of accuracy and efficiency.
Electronic Medical Records (EMR) are a rich source of patient information, including measurements reflecting physiologic signs and administered therapies. Identifying which variables are useful in predicting clinical outcomes can be challenging. Advanced algorithms such as deep neural networks were designed to process …
missForestPredict fills missing data for prediction models quickly and accurately.
problem Missing data in input variables for prediction models.
method Iterative imputation using random forests until convergence.
result missForestPredict outperforms other imputation methods in prediction settings.
We propose a procedure for assigning a relevance measure to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check the out of sample performance. First, the individual relevance of each variable is computed by comparing the predictions in …
DSVNP uses global and local latent variables for improved neural process predictions.
problem Limited expressiveness of vanilla neural processes in capturing target-specific local variation.
method Introduces DSVNP combining global and local latent variables for prediction.
result Competitive prediction performance in multi-output regression and uncertainty estimation.
For recurrent neural networks trained on time series with target and exogenous variables, in addition to accurate prediction, it is also desired to provide interpretable insights into the data. In this paper, we explore the structure of LSTM recurrent neural networks to learn variable-wise hidden states, with the aim t…
Hybrid quantum neural networks predict continuous variables.
problem Predicting continuous variables using quantum computing.
method Quantum classical hybrid neural networks for continuous variable prediction.
result Quantum neural networks outperform classical methods in continuous variable prediction.
This paper deals with prediction of anopheles number, the main vector of malaria risk, using environmental and climate variables. The variables selection is based on an automatic machine learning method using regression trees, and random forests combined with stratified two levels cross validation. The minimum threshol…
Bayesian approach selects subsets of variables for interpretable prediction and identifies key factors in educational outcomes.
problem Challenges in subset selection for stability, regularization, and inference.
method Bayesian perspective on subset selection, deriving optimal subsets and variable importance metrics.
result Better prediction, interval estimation, and variable selection compared to competing methods.
Machine learning algorithms find frequent application in spatial prediction of biotic and abiotic environmental variables. However, the characteristics of spatial data, especially spatial autocorrelation, are widely ignored. We hypothesize that this is problematic and results in models that can reproduce training data …
The paper investigates causal relationships in heart failure prediction using machine learning.
problem Understanding the causal relationships between clinical variables and heart failure.
method Proposes a new computational framework for causal structure discovery (CSD) of mixed-type clinical variables for binary disease outcomes.
result Feature importance from nonlinear classifiers strongly correlates with causal strength of variables, but not differentiating cause and effect.
Tree ensemble methods such as random forests [Breiman, 2001] are very popular to handle high-dimensional tabular data sets, notably because of their good predictive accuracy. However, when machine learning is used for decision-making problems, settling for the best predictive procedures may not be reasonable since enli…
MCP extends conformal prediction to vector-valued score functions without data splitting.
problem Fixed prediction set shapes in scalar score functions limit coverage guarantees.
method MCP uses a single optimization problem for prediction set design and calibration, eliminating data splitting.
result RemMCP and RelMCP achieve target coverage with smaller or comparable prediction set sizes, reducing variance.
The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.
problem Uncertainty in predicting macroeconomic variables like prices and returns.
method Defines theoretical lower bounds of uncertainty and upper limits on forecast accuracy based on statistical moments and trade volumes.
result Accuracy of forecasts of probabilities of macroeconomic variables doesn't exceed Gaussian approximations.
SCORE improves tree-based predictions with boosted residual extraTrees.
problem Improving tree-based prediction models with reduced errors.
method Inspired by representation learning, SCORE uses boosting, regularized regression, and variable selection.
result SCORE provides comparable or superior performance compared to other models.
The paper proposes a new model for predicting and analyzing economic variables.
problem Predicting and analyzing economic variables in developed regions.
method Time-varying parameter global vector autoregressive (TVP-GVAR) framework combined with machine learning models.
result The proposed model provides high precision out-of-sample predictions and novel insights into economic variable connectedness.
New measure assesses predictive dependence between continuous variables, capturing non-functional relationships.
problem Quantifying the joint dependence between continuous random variables.
method Introduces a novel, fully non-parametric measure bounded [0,1] that assesses predictive accuracy loss.
result The measure captures a wide range of relationships, including non-functional ones, and is interpretable.
We introduce a new approach to variable selection, called Predictive Correlation Screening, for predictor design. Predictive Correlation Screening (PCS) implements false positive control on the selected variables, is well suited to small sample sizes, and is scalable to high dimensions. We establish asymptotic bounds f…
New method better identifies irrelevant variables for more accurate treatment effect estimation.
problem Handling irrelevant variables in treatment effect estimation with deep disentanglement.
method Deep embedding method to disentangle pre-treatment variables, explicitly identify and represent irrelevant variables, and orthogonalize them.
result Better identification and representation of irrelevant variables lead to more precise treatment effect prediction.
Bayesian model for multi-environment prediction with latent variable changes.
problem Prediction in environments with changing latent variable distributions.
method Bayesian model with empirical Bayes prior and amortized variational algorithm.
result Method outperforms previous approaches in new environments.
We reproduced the results of CheXNet with fixed hyperparameters and 50 different random seeds to identify 14 finding in chest radiographs (x-rays). Because CheXNet fine-tunes a pre-trained DenseNet, the random seed affects the ordering of the batches of training data but not the initialized model weights. We found subs…
Study improves paddy rice yield predictions in Peru using sparse regression and climatic variables.
problem Improving precision of paddy rice yield forecasts in Peru.
method Sparse regression, Elastic-Net regularization, climatic variables, dynamic transformations.
result Improved predictive performance of paddy rice yield forecasts.
Second-order economic theory considers new variables to improve price volatility predictions.
problem Current economic models focus on first-order variables, missing second-order variables that affect price volatility.
method Introduces second-order economic theory with new variables composed of sums of squares of agents' transactions.
result Second-order economic theory complements first-order variables and introduces new macroeconomic variables.
Tree ensemble models such as random forests and boosted trees are among the most widely used and practically successful predictive models in applied machine learning and business analytics. Although such models have been used to make predictions based on exogenous, uncontrollable independent variables, they are increas…
In life sciences, the experts generally use empirical knowledge to recode variables, choose interactions and perform selection by classical approach. The aim of this work is to perform automatic learning algorithm for variables selection which can lead to know if experts can be help in they decision or simply replaced …
Study evaluates multi-omics data's role in predicting cancer survival.
problem Determining the usefulness of multi-omics data for predicting disease outcomes.
method 5-fold cross-validation with 12 prediction methods applied to 18 cancer datasets.
result Multi-omics data generally improves prediction performance, but not consistently.
Researchers develop a new method to assess variable importance in spatial machine learning models for air pollution exposure prediction.
problem Understanding the mechanism captured by machine learning models in air pollution studies, especially with spatial correlation.
method Leave-one-out approach for variable importance measure applicable to models with separable mean and covariance components.
result The new method highlights differences in model mechanisms even for similar prediction accuracies.
Paper uses neural networks to predict NOx emissions from gas turbines.
problem Predicting NOx emissions from degrading gas turbines.
method Applied neural network algorithm to model NOx emissions from nine process variables.
result Neural network model optimizes process variables for minimal NOx emissions.
Decadal climate predictions, which are initialized with observed conditions, are characterized by two main sources of uncertainties--internal and model variabilities. Using an ensemble of climate model simulations from the CMIP5 decadal experiments, we quantified the total uncertainty associated with these predictions …
Paper introduces PCP for efficient, reliable predictive inference.
problem Developing reliable predictive inference methods for target variables.
method Probabilistic conformal prediction using conditional random samples.
result PCP provides sharper predictive sets compared to existing methods.
Recently there has been a significant interest in learning disentangled representations, as they promise increased interpretability, generalization to unseen scenarios and faster learning on downstream tasks. In this paper, we investigate the usefulness of different notions of disentanglement for improving the fairness…
We develop a novel "decouple-recouple" dynamic predictive strategy and contribute to the literature on forecasting and economic decision making in a data-rich environment. Under this framework, clusters of predictors generate different latent states in the form of predictive densities that are later synthesized within …
For joint inference over multiple variables, a variety of structured prediction techniques have been developed to model correlations among variables and thereby improve predictions. However, many classical approaches suffer from one of two primary drawbacks: they either lack the ability to model high-order correlations…
For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables. Importance can be assessed locally with sensitivity analysis using general methods that rely on the model's predictions or their deri…
A new method for variable importance measures without impossible data.
problem Using impossible data for variable importance measures in black box models.
method Cohort Shapley, a method grounded in economic game theory using only observed data.
result Cohort Shapley provides a more trustworthy explanation of black box models' decisions.
Random forest hyperparameters affect variable selection in omics studies.
problem Impact of hyperparameters on variable selection in random forests.
method Two simulation studies using theoretical and empirical data.
result Hyperparameters influence variable selection more than the splitting strategy and sample fraction.
VarPro selects features without model dependence, achieving balanced performance.
problem Finding a small set of features with high explanatory power.
method Rule-based variable priority approach, avoiding model-specific methods and artificial data.
result VarPro has a consistent filtering property for noise variables and achieves balanced performance.
Study predicts droughts using ANN models and hydro-meteorological data.
problem Accurate prediction of short and long-term droughts.
method Employed Artificial Neural Network (ANN) models to predict droughts using SPI at different time scales and various hydro-meteorological variables.
result Hydro-meteorological variables significantly improve SPI prediction at different time scales.
Hierarchical-CPI improves variable importance measurement for medical data.
problem Limited interpretability of complex medical models.
method Hierarchical-CPI measures conditional variable importance with statistical control, handling correlated data.
result Hierarchical-CPI outperforms existing methods in medical datasets.
SEMF predicts prediction intervals for ML models using latent variables.
problem Uncertainty quantification in ML models, especially for diverse data distributions.
method Supervised Expectation-Maximization Framework (SEMF) extending EM algorithm for latent variable modeling.
result SEMF produces narrower prediction intervals with desired coverage probability.
Bayesian deep learning accounts for input uncertainty using Errors-in-Variables models.
problem Uncertainty in deep regression models, especially from input data.
method Bayesian treatment with Errors-in-Variables model to decompose predictive uncertainty.
result The approach yields more complete and consistent uncertainty estimates.
ecpc R-package improves high-dimensional prediction with co-data.
problem High-dimensional prediction with more variables than samples.
method Adaptive ridge penalised models with co-data, including continuous co-data.
result Improved variable selection and prediction performance.
Study examines challenges in variable importance ranking due to feature correlation.
problem Challenges in variable importance ranking under correlation.
method Simulation study and theoretical analysis of feature knockoffs and conditional predictive impact (CPI).
result Highly correlated features increase the correlation of knockoff variables, posing a limitation for CPI.