FlowSelect uses normalizing flows to control FDR in feature selection.
problem Controlled feature selection with knockoffs often fails to control false discovery rate (FDR).
method FlowSelect uses normalizing flows for accurate feature modeling and a novel MCMC-based p-value calculation to enforce knockoff properties.
result FlowSelect consistently controls FDR and demonstrates greater power compared to competing methods.
Nonparametric IPSS selects features with false discovery control.
problem Feature selection in high-dimensional data with theoretical false discovery control.
method Integrated Path Stability Selection (IPSS) applied to nonparametric feature importance scores.
result IPSS accurately controls false discovery rate and detects more true positives than existing methods.
Flexible deep learning framework controls FDR for feature selection.
problem Controlling Type-I error in feature selection for deep neural networks.
method Approximates FDR control for a wide range of deep architectures using gradient-based feature-importance vectors.
result Theoretical guarantee of FDR control for feature selection in deep learning models.
Feature selection is central to contemporary high-dimensional data analysis. Grouping structure among features arises naturally in various scientific problems. Many methods have been proposed to incorporate the grouping structure information into feature selection. However, these methods are normally restricted to a li…
NGMs create mirrored features to assess neural network feature importance.
problem Lack of feature relevance information in DNNs limits their applicability.
method Structured perturbation and kernel-based conditional dependence measure for feature importance evaluation.
result Controls feature selection error rate and maintains high selection power with correlated features.
The paper proposes a method to test features selected by SeqFS-DA with controlled FPR.
problem Ensuring reliability of feature selection after domain adaptation in high-dimensional regression.
method Proposes a novel method to test features selected by SeqFS-DA with controlled FPR.
result The proposed method controls FPR below a significance level α (e.g., 0.05) and enhances statistical power. We revisit the problem of feature selection in linear discriminant analysis (LDA), that is, when features are correlated. First, we introduce a pooled centroids formulation of the multiclass LDA predictor function, in which the relative weights of Mahalanobis-transformed predictors are given by correlation-adjusted t…
DMFAW improves multi-view clustering with adaptive weights and feature selection.
problem Lack of effective feature selection and empirical hyperparameter selection in existing deep matrix factorization methods.
method Introduces Deep Matrix Factorization with Adaptive Weights (DMFAW) for multi-view clustering, incorporating feature selection and dynamically updating weights using Control Theory.
result DMFAW outperforms state-of-the-art methods in clustering performance.
GRIP2 improves deep learning feature selection robustness in correlated and noisy data.
problem Identifying predictive features in correlated and noisy data.
method Integrates first-layer feature activity over a two-dimensional regularization surface to control sparsity and geometry, using efficient block-stochastic sampling.
result Demonstrates improved robustness and power in high correlation and low signal-to-noise ratio regimes.
New causal measures improve feature selection in AI models.
problem Lack of causal interpretability in AI models.
method Introduces causal entropy and causal information gain to assess feature control.
result Demonstrates superiority of causal information gain in feature selection.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
The L1 regularization (Lasso) has proven to be a versatile tool to select relevant features and estimate the model coefficients simultaneously and has been widely used in many research areas such as genomes studies, finance, and biomedical imaging. Despite its popularity, it is very challenging to guarantee the feature…
New method improves feature selection by integrating stability paths.
problem Improving feature selection with tighter false positive control.
method Integrating stability paths to strengthen theoretical bounds on E(FP).
result Significantly more true positives with same E(FP) control.
Approach for selecting features by discarding nuisance and correlated ones.
problem Large datasets with correlated and nuisance features.
method Laplacian score criterion, autoencoder architecture, concrete layer.
result Outperforms similar approaches in clustering performance.
Proposes a method to select features for deep learning in noisy, high-dimensional data.
problem Feature selection for deep learning in ultra-high dimensional and highly correlated data.
method Data-adaptive multi-resolutional screening and cleaning with deep learning.
result Achieves high power while keeping false discovery rate low.
Deep learning has become increasingly popular in both supervised and unsupervised machine learning thanks to its outstanding empirical performance. However, because of their intrinsic complexity, most deep learning methods are largely treated as black box tools with little interpretability. Even though recent attempts …
CatNet controls FDR in LSTM models using SHAP feature importance and Gaussian mirrors.
problem Controlling False Discovery Rate (FDR) in LSTM models with feature selection.
method CatNet uses SHAP values for feature importance and Gaussian Mirror algorithm for FDR control. It introduces a kernel-based independence measure to handle feature correlations.
result CatNet reduces overfitting and improves model interpretability on simulated and real-world data.
DeepLINK-T uses deep learning and knockoffs for time series data.
problem Interpreting and reproducible deep learning models for high-dimensional time series data.
method Combines deep learning with knockoffs for FDR control in feature selection for time series models.
result DeepLINK-T effectively controls FDR while demonstrating superior feature selection for high-dimensional longitudinal time series data.
We present an Automatic Relevance Determination prior Bayesian Neural Network(BNN-ARD) weight l2-norm measure as a feature importance statistic for the model-x knockoff filter. We show on both simulated data and the Norwegian wind farm dataset that the proposed feature importance statistic yields statistically signific…
In this paper, we consider voxel selection for functional Magnetic Resonance Imaging (fMRI) brain data with the aim of finding a more complete set of probably correlated discriminative voxels, thus improving interpretation of the discovered potential biomarkers. The main difficulty in doing this is an extremely high di…
Paper introduces PTL-SI for statistical inference in TL-HDR, controlling FPR.
problem Quantifying statistical significance in TL-HDR with limited data.
method PTL-SI framework for valid p-values in TL-HDR feature selection. result Valid p-values and controlled FPR in TL-HDR feature selection. A new method for feature selection robust to noise and design variability.
problem Feature selection in high-dimensional regression under sampling variability and measurement error.
method Injects controlled additive noise into the design matrix, fits a base selector, and aggregates selection frequencies.
result Improved robustness compared to Stability Selection and standard base selectors.
Paper improves feature selection for predicting outcomes from observational data.
problem Feature selection for post-intervention outcome prediction from pre-intervention variables in healthcare settings.
method Extends Markov boundary concept to treatment-outcome pairs, uses observational and experimental data.
result Combining observational and experimental data improves feature selection and effect estimation.
New method uses DNN for genetic variant identification, controlling randomness and improving interpretability.
problem Challenges in interpreting deep neural networks for genetic variant identification.
method Interpretable neural network model with controlled variable selection using ensembling, knockoffs, and de-randomization.
result The proposed method leads to more discoveries compared to conventional methods.
In machine learning applications for online product offerings and marketing strategies, there are often hundreds or thousands of features available to build such models. Feature selection is one essential method in such applications for multiple objectives: improving the prediction accuracy by eliminating irrelevant fe…
Paper relaxes symmetry conditions for universal feature selection in noisy data.
problem Feature selection in noisy data with weak symmetry.
method Developed a universal feature selection framework using singular value decomposition of canonical dependence matrix.
result Selected features achieve asymptotically optimal error exponents up to a residual term.
Proposes a new method to find features affecting treatment effect distribution.
problem Existing methods fail to detect differences in treatment effect distribution parameters other than the mean.
method Formulates and estimates a feature importance measure that quantifies feature influence on potential outcome distribution discrepancies. Develops a feature selection algorithm to control type I error rate.
result Successfully discovers important features and outperforms existing mean-based methods.
A new feature selection method for cost-sensitive classification in Random Forests.
problem Feature selection in Random Forests is challenging due to the complexity of ensemble trees.
method Shallow Tree Selection method for feature selection from small tree structures, adapted for cost-sensitive learning.
result BCR criterion improves performance of feature selection methods.
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Generative AI improves stock selection by synthesizing features from diverse data sources.
problem Automating feature discovery in stock market data.
method Used large language models with retrieval-augmented generation and structured prompting to synthesize features from various data sources.
result AI-generated features consistently outperform baselines, with Sharpe improvements ranging from 14% to 91%.
Feature selection is among the most important components because it not only helps enhance the classification accuracy, but also or even more important provides potential biomarker discovery. However, traditional multivariate methods is likely to obtain unstable and unreliable results in case of an extremely high dimen…
The goal of feature selection is to identify important features that are relevant to explain an outcome variable. Most of the work in this domain has focused on identifying globally relevant features, which are features that are related to the outcome using evidence across the entire dataset. We study a more fine-grain…
Novel privatization framework for high-dimensional variable selection with differential privacy.
problem High-dimensional controlled variable selection with rigorous FDR control under differential privacy constraints.
method Gaussian Johnson-Lindenstrauss Transformation for privatizing the knockoff matrix.
result The proposed private variable selection procedure maintains statistical power even under strict privacy budgets.
A novel feature selection method using noise-based hypothesis testing improves feature selection accuracy.
problem Challenges in feature selection for complex, high-dimensional datasets.
method Introduces multiple random noise features and evaluates feature importance against noise feature maxima using non-parametric bootstrap-based hypothesis testing.
result Outperforms existing methods in simulated and real-world datasets.
New method reduces memory usage for high-dimensional variable selection.
problem Scalability issues in high-dimensional variable selection, especially in genomics.
method Adaptive sampling of null features to eliminate dummy matrix materialization.
result Reduces memory and runtime by several orders of magnitude while preserving FDR control.
Proposes HSIC-Lasso for selective inference in non-linear data.
problem Detecting influential features in non-linear and high-dimensional data.
method Model-free HSIC-Lasso based on truncated Gaussians and polyhedral lemma.
result Tight control of type-I error even for small sample sizes.
This paper studies simultaneous feature selection and extraction in supervised and unsupervised learning. We propose and investigate selective reduced rank regression for constructing optimal explanatory factors from a parsimonious subset of input features. The proposed estimators enjoy sharp oracle inequalities, and w…
There are many problems in machine learning and data mining which are equivalent to selecting a non-redundant, high "quality" set of objects. Recommender systems, feature selection, and data summarization are among many applications of this. In this paper, we consider this problem as an optimization problem that seeks …
We connect high-dimensional subset selection and submodular maximization. Our results extend the work of Das and Kempe (2011) from the setting of linear regression to arbitrary objective functions. For greedy feature selection, this connection allows us to obtain strong multiplicative performance bounds on several meth…
gOMP algorithm selects features for various types of data.
problem Feature selection for scalable molecular data.
method Generalized Orthogonal Matching Pursuit algorithm for multiple types of data.
result gOMP performs similarly or better than LASSO on various datasets.
Datasets with hundreds to tens of thousands features is the new norm. Feature selection constitutes a central problem in machine learning, where the aim is to derive a representative set of features from which to construct a classification (or prediction) model for a specific task. Our experimental study involves micro…
Optimizes subgroup selection in clinical trials.
problem Identifying regions in feature space where a regression function exceeds a threshold.
method Formulates subgroup selection as constrained optimisation, determining minimax optimal rate for regret.
result Determines the minimax optimal rate for regret in sample size and Type I error probability.
A new knockoff statistic using conditional prediction function improves variable selection in complex models.
problem Controlling false discovery rate in complex models with nonlinear relationships.
method Introducing a knockoff statistic based on the conditional prediction function for use with machine learning models.
result The CPF statistics provide superior power in detecting prognostic variables over existing knockoff statistics.
SFS-DA method statistically tests FS reliability under domain adaptation.
problem Feature selection reliability under domain adaptation with limited target data.
method Selective Inference framework to control false positive rate and enhance true positive rate.
result SFS-DA method controls FPR below a pre-specified level α (e.g., 0.05) while maximizing true positive rate. Mild Traumatic Brain Injury (mTBI) is a significant public health problem. The most troubling symptoms after mTBI are cognitive complaints. Studies show measurable differences between patients with mTBI and healthy controls with respect to tissue microstructure using diffusion MRI. However, it remains unclear which dif…
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
ECCIT improves conditional independence tests by calibrating for miscalibration.
problem Inaccurate frequentist guarantees in CITs, especially in small samples and misspecified models.
method Empirically Calibrated Conditional Independence Tests (ECCIT) that optimize and correct for miscalibration.
result ECCIT achieves valid FDR with higher power than existing calibration strategies.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.