Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Solar algorithm selects variables faster and more accurately in high-dimensional data.
problem Variable selection in high-dimensional data with high accuracy and stability.
method Subsample-ordered least-angle regression (solar) and its coordinate descent generalization (solar-cd) using L0 norm solution path averaging. result Solar selects variables with high accuracy and stability, reducing redundant variable selection.
A new method selects important variables for clustering from dependency networks.
problem Variable selection for clustering in high-cost data scenarios.
method Create dependency networks, rank variables by centrality, select top-n variables.
result Top-n variables improve clustering performance compared to existing methods.
Variable selection is of significant importance for classification and regression tasks in machine learning and statistical applications where both predictability and explainability are needed. In this paper, a Copula Entropy (CE) based method for variable selection which use CE based ranks to select variables is propo…
Proposes a two-stage method for selecting correlated predictors in high-dimensional data.
problem Selecting correlated predictors in high-dimensional data with unknown group structures.
method Two-stage approach: variable clustering followed by group selection.
result The two-stage method improves prediction accuracy and active predictor selection.
The paper develops a test for independence of selected Gaussian variables after thresholding correlations.
problem Testing independence of selected Gaussian variables after thresholding correlations.
method The approach involves conditioning on the selection event and using a new characterization of the conditioning event in terms of canonical correlation.
result The proposed test has higher power than a naive approach that ignores selection effects.
VC-PCR improves prediction by clustering correlated variables.
problem Decreased prediction accuracy due to cluster structure in predictor variables.
method Supervised variable selection and clustering to integrate cluster information into a sparse modeling process.
result VC-PCR achieves better prediction, variable selection, and clustering performance.
Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables i…
Private variable selection method controls FDR with simulations showing reasonable power.
problem Performing variable selection with privacy constraints.
method Private knockoff filter using Gaussian and Laplace mechanisms.
result Achieves controlled false discovery rate (FDR) in variable selection.
Random forest hyperparameters affect variable selection in omics studies.
problem Impact of hyperparameters on variable selection in random forests.
method Two simulation studies using theoretical and empirical data.
result Hyperparameters influence variable selection more than the splitting strategy and sample fraction.
Study improves model estimation and variable selection using GANs with Lasso penalty.
problem Variable selection in high-dimensional data with deep networks.
method Conditional Wasserstein Generative Adversarial Networks with Group Lasso penalization.
result Established convergence rate for variable selection in censored survival data.
Variable selection for optimal treatment regime in a clinical trial or an observational study is getting more attention. Most existing variable selection techniques focused on selecting variables that are important for prediction, therefore some variables that are poor in prediction but are critical for decision-making…
Efficient Bayesian variable selection for binomial and negative binomial data.
problem Computational challenges in Bayesian variable selection for complex models.
method Tempered Gibbs Sampling and MCMC scheme.
result Demonstrated effectiveness on cancer data with thousands of covariates.
Knoop enhances variable selection with over-parameterization and knockoffs.
problem Challenges of variable selection in high-dimensional datasets.
method Generates knockoff variables, integrates them into an over-parameterized model, and uses anomaly-based significance tests.
result Superior performance in variable selection compared to existing methods.
Bayesian method tackles variable selection in high-dimensional data.
problem Challenges in Bayesian variable selection with large P.
method Efficient MCMC scheme with sublinear cost per iteration, extended to generalized linear models.
result Demonstrated effectiveness on cancer and maize genomic data.
Proposes a gradient-based variable selection method for binary classification in RKHS.
problem Variable selection in high-dimensional data analysis.
method Gradient-based representation of large-margin classifier with group-lasso penalty.
result Selection consistency and risk bound of the estimated classifier.
We present a new variable selection method based on model-based gradient boosting and randomly permuted variables. Model-based boosting is a tool to fit a statistical model while performing variable selection at the same time. A drawback of the fitting lies in the need of multiple model fits on slightly altered data (e…
The paper introduces methods to identify key variables discriminating between two datasets.
problem Identifying variables that distinguish between two datasets.
method Introduces a mathematical notion of discriminating variables and proposes two methods for their selection.
result Proposed methods improve upon existing techniques in two-sample variable selection.
Proposes a method to select variables for kernel two-sample tests.
problem Determining whether two samples have the same distribution using informative variables.
method A framework based on kernel maximum mean discrepancy (MMD) for selecting a subset of variables.
result The sample size requirements for the three kernels depend on the number of selected variables, not the data dimension.
Lasso and other regularization procedures are attractive methods for variable selection, subject to a proper choice of shrinkage parameter. Given a set of potential subsets produced by a regularization algorithm, a consistent model selection criterion is proposed to select the best one among this preselected set. The a…
Local learning method selects covariates for causal effect estimation in the presence of latent variables.
problem Estimating causal effects from nonexperimental data with latent variables.
method Local learning approach that identifies valid adjustment sets for causal relationships.
result Ensures soundness and completeness of causal effect estimation under standard assumptions.
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
Extends deep learning for nonlinear Cox regression variable selection.
problem Variable selection for nonlinear Cox regression model.
method Extends LassoNet to survival data for nonlinear Cox model.
result Valid and effective method demonstrated through simulations.
The most popular approach for analyzing survival data is the Cox regression model. The Cox model may, however, be misspecified, and its proportionality assumption may not always be fulfilled. An alternative approach for survival prediction is random forests for survival outcomes. The standard split criterion for random…
Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…
Bayesian approach selects subsets of variables for interpretable prediction and identifies key factors in educational outcomes.
problem Challenges in subset selection for stability, regularization, and inference.
method Bayesian perspective on subset selection, deriving optimal subsets and variable importance metrics.
result Better prediction, interval estimation, and variable selection compared to competing methods.
Solar improves variable selection in high-dimensional data with complicated dependence structures.
problem Variable selection in ultrahigh dimensional data with severe multicollinearity and grouping effect issues.
method Subsample-ordered least angle regression (Solar) for ultrahigh dimensional data.
result Solar yields substantial improvements in sparsity, stability, and accuracy of variable selection compared to traditional methods.
Proposes a method to improve Lasso stability via variable decorrelation.
problem Stability of Lasso in high-dimensional settings with correlated variables.
method Variable decorrelation before applying the Lasso.
result Improves stability of variable selection regardless of predictor correlation.
In this paper, we introduce Adaptive Cluster Lasso(ACL) method for variable selection in high dimensional sparse regression models with strongly correlated variables. To handle correlated variables, the concept of clustering or grouping variables and then pursuing model fitting is widely accepted. When the dimension is…
In this article, we advocate the ensemble approach for variable selection. We point out that the stochastic mechanism used to generate the variable-selection ensemble (VSE) must be picked with care. We construct a VSE using a stochastic stepwise algorithm, and compare its performance with numerous state-of-the-art algo…
This paper deals with prediction of anopheles number, the main vector of malaria risk, using environmental and climate variables. The variables selection is based on an automatic machine learning method using regression trees, and random forests combined with stratified two levels cross validation. The minimum threshol…
Random Forest variable importance is improved by class balancing techniques.
problem Class imbalance problem in machine learning.
method Proposed a variable selection algorithm using RF variable importance and its confidence interval.
result Our algorithm efficiently selects an optimal feature set, leading to improved prediction performance.
Deep neural networks (DNNs) are famous for their high prediction accuracy, but they are also known for their black-box nature and poor interpretability. We consider the problem of variable selection, that is, selecting the input variables that have significant predictive power on the output, in DNNs. We propose a backw…
Unified framework for selecting variables with uncertainty quantification.
problem Uncertainty in nonlinear variable selection for various models.
method Develops a unified framework using integrated partial derivatives for quantifying variable importance and uncertainty.
result The approach provides a principled method for quantifying variable selection uncertainty and is generalizable to non-differentiable models.
Estimates causal effects with selection bias and confounding using regression.
problem Estimating causal effects in presence of selection bias and confounding.
method Two-step regression estimator (TSR) that corrects for selection bias and accounts for confounding.
result TSR estimator reduces variance and is validated in simulations.
While variable selection is essential to optimize the learning complexity by prioritizing features, automating the selection process is preferred since it requires laborious efforts with intensive analysis otherwise. However, it is not an easy task to enable the automation due to several reasons. First, selection techn…
Proposes a method for stable variable selection in high-dimensional data.
problem Challenges of variable selection in high-dimensional, correlated data.
method Resample-aggregate framework using diffusion models.
result Stable subset of predictors with calibrated stability scores.
In this article, we propose a new algorithm for supervised learning methods, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, an ideal selection method should have the potential of adding a supplementary level of regres…
VarPro selects features without model dependence, achieving balanced performance.
problem Finding a small set of features with high explanatory power.
method Rule-based variable priority approach, avoiding model-specific methods and artificial data.
result VarPro has a consistent filtering property for noise variables and achieves balanced performance.
New algorithm selects variables from large datasets.
problem Automatic selection of variables from large datasets.
method Uses Graphical Models and combines with OLS method.
result Outperforms LASSO method in forecasting models.
Penalized regression models are popularly used in high-dimensional data analysis to conduct variable selection and model fitting simultaneously. Whereas success has been widely reported in literature, their performances largely depend on the tuning parameters that balance the trade-off between model fitting and model s…
Study tackles variable selection with missing covariates and outcomes using machine learning and imputation.
problem Missing data in both covariates and outcomes complicates variable selection in health studies.
method Exploits machine learning flexibility and bootstrap imputation for variable selection, comparing multiple methods.
result XGBoost and BART perform best in variable selection with bootstrap imputation, achieving high F1 scores and low Type I errors. SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.
problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.
Sparse PCA selects variables with FDR control for improved performance.
problem Sparse PCA selects irrelevant variables when maximizing explained variance.
method Proposes FDR-controlled selection using T-Rex selector.
result Significant performance improvement over traditional sparse PCA.
Develops ELBD for efficient feature selection in VAE latent variables.
problem Feature selection in latent variables of VAE and its variants.
method ELBD score algorithm and weak convergence approximation for optimization.
result Effective feature selection and optimization of VAE models.
In this study, we propose an automatic learning method for variables selection based on Lasso in epidemiology context. One of the aim of this approach is to overcome the pretreatment of experts in medicine and epidemiology on collected data. These pretreatment consist in recoding some variables and to choose some inter…
This paper enhances stability selection by evaluating overall results robustness and identifying optimal regularization values.
problem Improving the robustness and reliability of high-dimensional variable selection.
method Developed a stability estimator to evaluate stability of stability selection results, calibrating key parameters.
result Identified optimal regularization value and improved stability of variable selection.
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.