Develops a more powerful selective inference method for stepwise feature selection.
problem Loss of power in existing conditional SI methods due to over-conditioning.
method Uses homotopy continuation approach to overcome over-conditioning.
result Shows improved power and efficiency in selective inference for feature selection.
The paper develops a method to identify conditionally relevant features with statistical guarantees.
problem Identifying features that are relevant given the values of other features.
method A generalization of the knockoff procedure that controls a generalized FDR for conditional feature selection.
result The method provides a statistical guarantee for conditional feature selection.
A new method for dynamic feature selection outperforms existing approaches.
problem Sequentially selecting features based on current information in machine learning.
method Greedy selection of features based on conditional mutual information, combined with a learning approach for optimization.
result The method outperforms existing feature selection methods in experiments.
Proposes a new method to improve selective inference for Lasso models.
problem Over-conditioning due to conditioning on feature signs in selective inference for Lasso.
method Parametric programming approach to avoid conditioning on signs and identify feature selection events.
result Improves power and practicality of selective inference for Lasso models.
Paper relaxes symmetry conditions for universal feature selection in noisy data.
problem Feature selection in noisy data with weak symmetry.
method Developed a universal feature selection framework using singular value decomposition of canonical dependence matrix.
result Selected features achieve asymptotically optimal error exponents up to a residual term.
More powerful feature selection tests using selective inference.
problem Selection bias in feature selection leading to specious analysis.
method Conditioning on minimal selection event using Maximum Mean Discrepancy and Hilbert Schmidt Independence Criterion with multiscale bootstrap.
result Proposed test is more powerful in most scenarios.
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
In this paper, a novel feature selection method is presented, which is based on Class-Separability (CS) strategy and Data Envelopment Analysis (DEA). To better capture the relationship between features and the class, class labels are separated into individual variables and relevance and redundancy are explicitly handle…
Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to pro…
Novel approach uses neural networks to enhance CI testing for feature selection.
problem Challenges in implementing Markov blanket feature selection due to CI testing limitations.
method Two-step approach: feature mapping followed by CI testing using k-NN. result The method boosts CI testing performance, leading to improved feature selection.
Learning a distribution conditional on a set of discrete-valued features is a commonly encountered task. This becomes more challenging with a high-dimensional feature set when there is the possibility of interaction between the features. In addition, many frequently applied techniques consider only prediction of the me…
Gaussian OBFS proves strong consistency in feature selection with correlations.
problem Feature selection consistency in the presence of correlations.
method Proves strong consistency of Gaussian OBFS under mild conditions.
result Identifies selected features and rates of convergence for different feature types.
A new method selects robust features for ML models using causal discovery.
problem Challenges in feature selection for ML models with limited domain knowledge.
method Multidata causal feature selection using PC1 or PCMCI algorithms.
result The method improves model performance and provides interpretable drivers.
New theoretical insights improve feature selection using mutual information.
problem Improving feature selection methods with theoretical guarantees.
method Proposed novel stopping condition for greedy feature selection methods.
result Ideal prediction error remains bounded by a threshold.
NGMs create mirrored features to assess neural network feature importance.
problem Lack of feature relevance information in DNNs limits their applicability.
method Structured perturbation and kernel-based conditional dependence measure for feature importance evaluation.
result Controls feature selection error rate and maintains high selection power with correlated features.
A feature selection algorithm should ideally satisfy four conditions: reliably extract relevant features; be able to identify non-linear feature interactions; scale linearly with the number of features and dimensions; allow the incorporation of known sparsity structure. In this work we propose a novel feature selection…
New algorithms for model selection in linear contextual bandits without feature diversity conditions.
problem Model selection in linear contextual bandits without feature diversity conditions.
method Data-adaptive algorithms that provide model selection guarantees without feature diversity conditions.
result O(d^α T^{1-α}) model selection guarantees with no feature diversity conditions.
We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building on past work in kernel dimension reduction, we show how to perform feature selection via a constrained optimization problem involving the tra…
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Feature selection and reducing the dimensionality of data is an essential step in data analysis. In this work, we propose a new criterion for feature selection that is formulated as conditional information between features given the labeled variable. Instead of using the standard mutual information measure based on Kul…
Online selection of dynamic features has attracted intensive interest in recent years. However, existing online feature selection methods evaluate features individually and ignore the underlying structure of feature stream. For instance, in image analysis, features are generated in groups which represent color, texture…
DiSC detects feature clusters that differentiate between conditions.
problem Identifying subsets of features that differentiate between two conditions.
method Construct feature graphs, compute connectivity differences using spectral clustering.
result DiSC uncovers features that better differentiate between conditions.
Fast feature selection for SHM using canonical correlation.
problem Feature selection for structural health monitoring.
method Greedy search of sum of squared canonical correlation coefficients.
result Extremely fast feature selection with good performance.
Feature selection, which searches for the most representative features in observed data, is critical for health data analysis. Unlike feature extraction, such as PCA and autoencoder based methods, feature selection preserves interpretability, meaning that the selected features provide direct information about certain h…
Variable screening is a fast dimension reduction technique for assisting high dimensional feature selection. As a preselection method, it selects a moderate size subset of candidate variables for further refining via feature selection to produce the final model. The performance of variable screening depends on both com…
SYNTHONY selects tabular synthesizers based on stress profiling and user intent.
problem Non-uniform performance of tabular generative models across datasets.
method Stress profiling and intent-conditioned tabular synthesis selection.
result Meta-features predict synthesizer performance, improving selection accuracy.
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite …
This paper selects features in deep neural networks with theoretical guarantees.
problem Feature selection in deep neural networks with unknown nonlinear functions.
method Reformulate neural networks as index models, estimate feature sets using Stein's formula, and apply screening-and-selection mechanism.
result Consistent feature selection with theoretical guarantees, even in high-dimensional settings.
A new method for feature selection robust to noise and design variability.
problem Feature selection in high-dimensional regression under sampling variability and measurement error.
method Injects controlled additive noise into the design matrix, fits a base selector, and aggregates selection frequencies.
result Improved robustness compared to Stability Selection and standard base selectors.
Paper improves feature selection for predicting outcomes from observational data.
problem Feature selection for post-intervention outcome prediction from pre-intervention variables in healthcare settings.
method Extends Markov boundary concept to treatment-outcome pairs, uses observational and experimental data.
result Combining observational and experimental data improves feature selection and effect estimation.
Cost-efficient feature selection for multi-label classification in medicine.
problem Feature selection in multi-label classification with cost constraints.
method Sequential feature selection maximizing conditional mutual information, followed by cost-free feature selection using shadow features.
result The method effectively reduces prediction costs in medical applications.
A new feature selection method using random forest and Kolmogorov filter.
problem Ultra-high dimensional data feature selection.
method Fused Kolmogorov filter with random forest based recursive feature elimination.
result Selection and L2 consistency under weak conditions. A new algorithm difFOCI improves feature learning from data.
problem Feature selection and learning from data.
method Parametric, differentiable approximation of FOCI method.
result Improves feature learning with better management of spurious correlations.
Method evaluates classification uncertainty with adaptively chosen features.
problem Finding a balance between model efficiency and fairness.
method Adaptively selects features for equalized coverage in classification.
result Valid and effective method demonstrated on simulated and real data.
New method reduces computational cost for selective inference.
problem Over-conditioning in selective inference.
method Parametric programming-based selective inference (PP-based SI) with bounded p-values.
result Reduced computational cost while maintaining desired precision.
Proposes a new method to find features affecting treatment effect distribution.
problem Existing methods fail to detect differences in treatment effect distribution parameters other than the mean.
method Formulates and estimates a feature importance measure that quantifies feature influence on potential outcome distribution discrepancies. Develops a feature selection algorithm to control type I error rate.
result Successfully discovers important features and outperforms existing mean-based methods.
Identifying small subsets of features that are relevant for prediction and/or classification tasks is a central problem in machine learning and statistics. The feature selection task is especially important, and computationally difficult, for modern datasets where the number of features can be comparable to, or even ex…
This paper presents an original approach for jointly fitting survival times and classifying samples into subgroups. The Coxlogit model is a generalized linear model with a common set of selected features for both tasks. Survival times and class labels are here assumed to be conditioned by a common risk score which depe…
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Identifying measurable genetic indicators (or biomarkers) of a specific condition of a biological system is a key element of precision medicine. Indeed it allows to tailor diagnostic, prognostic and treatment choice to individual characteristics of a patient. In machine learning terms, biomarker discovery can be framed…
OBF optimally filters features under independent Gaussian models.
problem Biomarker discovery from complex data.
method Optimal Bayesian feature selection under independent Gaussian models.
result OBF is consistent and optimal under mild conditions.
ECCIT improves conditional independence tests by calibrating for miscalibration.
problem Inaccurate frequentist guarantees in CITs, especially in small samples and misspecified models.
method Empirically Calibrated Conditional Independence Tests (ECCIT) that optimize and correct for miscalibration.
result ECCIT achieves valid FDR with higher power than existing calibration strategies.
A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selecti…
We present a novel event embedding algorithm for crime data that can jointly capture time, location, and the complex free-text component of each event. The embedding is achieved by regularized Restricted Boltzmann Machines (RBMs), and we introduce a new way to regularize by imposing a ℓ1 penalty on the conditiona…
Paper automates variable selection for network anomaly detection.
problem Automating variable selection for network anomaly detection is challenging.
method Ensemble approach combining well-known selection techniques.
result Identifies a small number of core variables that approximate full-variable performance.
e-values rank features for model performance.
problem Feature selection for parametric models.
method Data depths and resampling-based algorithm.
result e-values distinguish essential features.
Study selective classification with halfspaces, achieving error bounds under Gaussian distributions.
problem Modeling relationships in subsets of data defined by selection rules.
method Sparse linear classifiers for subsets defined by halfspaces, focusing on Gaussian feature distributions.
result First PAC-learning algorithm for homogeneous halfspace selectors with error guarantee $\bigO*{\sqrt{\mathrm{opt}}}$.
Cross-balancing improves causal inference by balancing features with outcome data.
problem Balancing features for valid causal inference when outcome data is available.
method Cross-balancing using sample splitting to separate feature construction and weight estimation errors.
result Cross-balancing produces consistent, asymptotically normal, and efficient estimators under mild conditions.