LFD method improves text classification by making features clearer and less label-leaking.
problem Creating interpretable text representations that are both predictive and understandable.
method LFD method: proposes lexical and semantic features from contrastive text pairs, screens candidates using κ, and selects features by residual gain. result LFD features achieve higher human-human and human-LLM agreement than baseline concepts and are less label-leaking.
LOLA uses LLMs to optimize content delivery, outperforming traditional methods.
problem Identifying the most engaging headlines for user engagement.
method LOLA integrates LLMs with adaptive experimentation to optimize content delivery.
result LOLA outperforms traditional methods in optimizing user engagement.
ContextBench benchmarks methods for generating linguistically fluent inputs that activate specific latent features in language models.
problem Identifying inputs that trigger specific behaviours or latent features in language models.
method Context modification and benchmarking methods like Evolutionary Prompt Optimisation (EPO) with LLM-assistance and diffusion model inpainting.
result Enhanced methods achieve state-of-the-art performance in balancing elicitation effectiveness and fluency.
As datasets grow richer, an important challenge is to leverage the full features in the data to maximize the number of useful discoveries while controlling for false positives. We address this problem in the context of multiple hypotheses testing, where for each hypothesis, we observe a p-value along with a set of feat…
Nonparametric IPSS selects features with false discovery control.
problem Feature selection in high-dimensional data with theoretical false discovery control.
method Integrated Path Stability Selection (IPSS) applied to nonparametric feature importance scores.
result IPSS accurately controls false discovery rate and detects more true positives than existing methods.
Diamond method controls FDR for trustworthy feature interaction discovery in ML models.
problem Limited interpretability of ML models due to black box nature.
method Diamond method integrates model-X knockoffs framework to control FDR for non-additive interactions.
result Diamond method ensures accurate discovery of feature interactions with FDR control.
New methods for Markov Blanket discovery using MML outperform existing approaches.
problem Causal discovery from large datasets.
method Developed three new methods of Markov Blanket discovery using Minimum Message Length.
result Our best MML method is consistently competitive and has advantageous features.
CatNet controls FDR in LSTM models using SHAP feature importance and Gaussian mirrors.
problem Controlling False Discovery Rate (FDR) in LSTM models with feature selection.
method CatNet uses SHAP values for feature importance and Gaussian Mirror algorithm for FDR control. It introduces a kernel-based independence measure to handle feature correlations.
result CatNet reduces overfitting and improves model interpretability on simulated and real-world data.
SOCRATES uses LLMs to automate simulation optimization of complex systems.
problem Optimizing complex, expensive-to-sample stochastic systems.
method Two-stage procedure: replica construction and meta-optimization.
result Adaptive hybrid optimization schedule for real systems.
SWA selects important features from large data sets, controlling false discovery rate.
problem Feature selection in large regression data, especially scaling to big data and matching target FDR.
method Subsampling Winner algorithm using subsampling and scoring features.
result SWA controls actual FDR better than benchmark procedures and randomForest.
Previous work in network analysis has focused on modeling the mixed-memberships of node roles in the graph, but not the roles of edges. We introduce the edge role discovery problem and present a generalizable framework for learning and extracting edge roles from arbitrary graphs automatically. Furthermore, while existi…
A new method selects robust features for ML models using causal discovery.
problem Challenges in feature selection for ML models with limited domain knowledge.
method Multidata causal feature selection using PC1 or PCMCI algorithms.
result The method improves model performance and provides interpretable drivers.
Three RFF-based methods for nonlinear causal discovery in mixed data.
problem Nonlinear causal discovery in mixed data with computational constraints.
method FFML, TRFF, and FFCI methods for score-based, constraint-based, and hybrid causal discovery.
result FFML and TRFF methods provide complementary performance in causal discovery.
DDLK uses deep learning to find important features in models.
problem Discovering important features in black box models like deep neural networks.
method DDLK directly minimizes KL divergence to generate knockoffs that obey the swap property.
result DDLK outperforms baselines in discovering important features while controlling false discovery rate.
We introduce a framework for dynamic adversarial discovery of information (DADI), motivated by a scenario where information (a feature set) is used by third parties with unknown objectives. We train a reinforcement learning agent to sequentially acquire a subset of the information while balancing accuracy and fairness …
FMI uses matching to mimic interventions for causal feature learning.
problem Challenges in causal discovery from observational data.
method Feature Matching Intervention (FMI) using matching to emulate perfect interventions.
result FMI outperforms in identifying causal features from observational data.
COLUMBUS discovers new features to improve domain generalization.
problem Improving machine learning models' ability to generalize to unseen domains.
method COLUMBUS uses targeted corruption of input and multi-level representations to discover new features.
result COLUMBUS achieves state-of-the-art performance on DG benchmarks.
Proposes a method to select features for deep learning in noisy, high-dimensional data.
problem Feature selection for deep learning in ultra-high dimensional and highly correlated data.
method Data-adaptive multi-resolutional screening and cleaning with deep learning.
result Achieves high power while keeping false discovery rate low.
Univariate and multivariate feature selection methods can be used for biomarker discovery in analysis of toxicant exposure. Among the univariate methods, differential expression analysis (DEA) is often applied for its simplicity and interpretability. A characteristic of methods for DEA is that they treat genes individu…
Generative AI improves stock selection by synthesizing features from diverse data sources.
problem Automating feature discovery in stock market data.
method Used large language models with retrieval-augmented generation and structured prompting to synthesize features from various data sources.
result AI-generated features consistently outperform baselines, with Sharpe improvements ranging from 14% to 91%.
Identifying measurable genetic indicators (or biomarkers) of a specific condition of a biological system is a key element of precision medicine. Indeed it allows to tailor diagnostic, prognostic and treatment choice to individual characteristics of a patient. In machine learning terms, biomarker discovery can be framed…
ECCIT improves conditional independence tests by calibrating for miscalibration.
problem Inaccurate frequentist guarantees in CITs, especially in small samples and misspecified models.
method Empirically Calibrated Conditional Independence Tests (ECCIT) that optimize and correct for miscalibration.
result ECCIT achieves valid FDR with higher power than existing calibration strategies.
Fast nonparametric conditional independence testing via two-stage regression
problem Fast nonparametric conditional independence testing
method BLITZ (Broad-to-Local Independence Testing via residualiZation)
result Better null calibration than fast kernel, random-feature, and regression-based competitors
ROOFS helps researchers select robust biomarker features from complex data.
problem Challenges in feature selection for biomarker discovery and clinical models.
method ROOFS is a Python package that benchmarks multiple feature selection methods on user data.
result ROOFS identifies a filter method as optimal for identifying predictors of lung cancer resistance.
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Feature selection is among the most important components because it not only helps enhance the classification accuracy, but also or even more important provides potential biomarker discovery. However, traditional multivariate methods is likely to obtain unstable and unreliable results in case of an extremely high dimen…
Causal discovery can be a powerful tool for investigating causality when a system can be observed but is inaccessible to experiments in practice. Despite this, it is rarely used in any scientific or medical fields. One of the major hurdles preventing the field of causal discovery from having a larger impact is that it …
Markov boundary improves tabular prediction but not as expected.
problem Improving tabular prediction using the Markov boundary.
method Evaluation on a synthetic SCM benchmark with feature counts from 40 to 1000.
result Restricting a regressor to the Markov boundary often improves prediction, but existing discovery and training pipelines do not fully exploit this.
In science and medicine, model interpretations may be reported as discoveries of natural phenomena or used to guide patient treatments. In such high-stakes tasks, false discoveries may lead investigators astray. These applications would therefore benefit from control over the finite-sample error rate of interpretations…
engGNN combines external and generated graphs to improve disease classification and biomarker discovery.
problem Challenges in integrating omics data due to high dimensionality and small sample sizes.
method Dual-graph framework that integrates external biological networks with data-driven generated graphs.
result engGNN outperforms state-of-the-art methods in disease classification and biomarker discovery.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
problem Local attribution methods fail to identify coherent model behavior across samples.
method Sparse Subspace Clustering (SSCC) for concept discovery.
result Empirically validated method for various image classification tasks.
We present an intriguing discovery related to Random Fourier Features: in Gaussian kernel approximation, replacing the random Gaussian matrix by a properly scaled random orthogonal matrix significantly decreases kernel approximation error. We call this technique Orthogonal Random Features (ORF), and provide theoretical…
The paper investigates causal relationships in heart failure prediction using machine learning.
problem Understanding the causal relationships between clinical variables and heart failure.
method Proposes a new computational framework for causal structure discovery (CSD) of mixed-type clinical variables for binary disease outcomes.
result Feature importance from nonlinear classifiers strongly correlates with causal strength of variables, but not differentiating cause and effect.
Bayesian network structure learning algorithms with limited data are being used in domains such as systems biology and neuroscience to gain insight into the underlying processes that produce observed data. Learning reliable networks from limited data is difficult, therefore transfer learning can improve the robustness …
NeurT-FDR controls FDR by incorporating feature hierarchy.
problem Controlling FDR in complex, large-scale hypothesis testing problems.
method NeurT-FDR uses a neural network to parametrize test-level covariates and a regression framework to adjust feature hierarchy.
result NeurT-FDR makes substantially more discoveries than competitive baselines.
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
An important problem in machine learning and statistics is to identify features that causally affect the outcome. This is often impossible to do from purely observational data, and a natural relaxation is to identify features that are correlated with the outcome even conditioned on all other observed features. For exam…
Study compares single vs ensemble feature selection for cancer diagnosis.
problem Identifying relevant variables for cancer diagnosis and prognosis.
method Comparison of single feature selection algorithms and ensemble of diverse algorithms.
result Ensemble approach did not improve predictive performance over individual algorithms.
PIMA autoencoders discover shared features in multimodal scientific data.
problem Discovering shared information in high-throughput scientific datasets.
method Physics-informed multimodal autoencoders (PIMA) with Gaussian mixture prior and product of experts formulation.
result Accurate cross-modal inference between images and mechanical stress-strain response in lattice metamaterials.
Paper proposes conditional multidimensional scaling for better data reduction.
problem Mapping high-dimensional data to low-dimensional space with known features.
method Developed a broad class of methods called conditional multidimensional scaling (MDS) with an optimization algorithm.
result Conditional MDS improves estimation quality and simplifies visualization and knowledge discovery.
FlowSelect uses normalizing flows to control FDR in feature selection.
problem Controlled feature selection with knockoffs often fails to control false discovery rate (FDR).
method FlowSelect uses normalizing flows for accurate feature modeling and a novel MCMC-based p-value calculation to enforce knockoff properties.
result FlowSelect consistently controls FDR and demonstrates greater power compared to competing methods.
Cluster LOCO: A model-agnostic feature importance score for interpreting cluster outputs
problem Interpreting and auditing cluster outputs
method Cluster LOCO (Leave-One-Covariate-Out)
result More reliably recovers informative features than existing methods
Genetic programming (GP) is the state-of-the-art in financial automated feature construction task. It employs reverse polish expression to represent features and then conducts the evolution process. However, with the development of deep learning, more powerful feature extraction tools are available. This paper proposes…
Improves decision-making by correcting feature selection bias.
problem Cofounding bias in feature selection affects machine learning predictions.
method Proposes a meta-algorithm using a novel adjustment criterion based on causal sufficiency.
result Corrects cofounding bias to improve prediction performance.
Automated detection of new, interesting, unusual, or anomalous images within large data sets has great value for applications from surveillance (e.g., airport security) to science (observations that don't fit a given theory can lead to new discoveries). Many image data analysis systems are turning to convolutional neur…
TimeGraph creates synthetic datasets for robust time-series causal discovery.
problem Lack of reliable synthetic benchmark datasets for robust time-series causal discovery.
method Developed comprehensive synthetic datasets with temporal properties, including trends, seasonality, and noise.
result Demonstrated significant variations in algorithm performance under realistic temporal conditions.
This paper introduces LR-FFS for robust feature screening in federated learning under label shift.
problem Label shift challenges in federated learning for high-dimensional classification.
method Unified feature screening framework, label-shift robust federated feature screening (LR-FFS), federated estimation procedure.
result LR-FFS outperforms existing methods in diverse client environments with varying class distributions, sample sizes, and missing data.
New method discovers causal structures from incomplete data.
problem Discovering causal structure from incomplete data.
method Encoder and reinforcement learning integrated approach.
result Our method outperforms existing methods by 43.2%.