Paper relaxes differential privacy for correlated features, improving privacy-utility trade-off.
problem Standard differential privacy ignores feature correlation, leading to suboptimal privacy-utility balance.
method Introduces CorrDP framework that accounts for feature correlation, using total variation distance for quantification.
result CorrDP algorithms outperform standard DP in synthetic and real-world datasets with insensitive features.
Approach for selecting features by discarding nuisance and correlated ones.
problem Large datasets with correlated and nuisance features.
method Laplacian score criterion, autoencoder architecture, concrete layer.
result Outperforms similar approaches in clustering performance.
Kernel methods linked to feature subspaces and maximal correlation kernels.
problem Understanding kernel methods and their relationship to feature extraction.
method Established a correspondence between feature subspaces and kernels, introduced maximal correlation kernels, and demonstrated their optimality.
result Kernel SVM on maximal correlation kernel achieves minimum prediction error.
Study examines challenges in variable importance ranking due to feature correlation.
problem Challenges in variable importance ranking under correlation.
method Simulation study and theoretical analysis of feature knockoffs and conditional predictive impact (CPI).
result Highly correlated features increase the correlation of knockoff variables, posing a limitation for CPI.
Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.
problem Understanding how feature correlations impact DAM's capacity.
method Developed an empirical framework to analyze DAM's capacity under varying feature correlations and pattern separations.
result Memory capacity scales exponentially with feature separation, unaffected by correlations.
FREEtree improves tree-based methods for correlated longitudinal data.
problem Poor performance of Random Forests in high dimensional longitudinal data with correlated features.
method FREEtree uses a piecewise random effects model and clustering with WGCNA to select features and maintain interpretability.
result FREEtree outperforms other tree-based methods in prediction and feature selection accuracy.
The paper compares LOCO and Shapley values for feature importance, highlighting their limitations and suggesting improvements.
problem Quantifying feature importance in the presence of feature correlation.
method LOCO and Shapley Values, critiquing their axioms and proposing new measures.
result Shapley values do not eliminate feature correlation, and a modified LOCO is recommended.
PFDL improves deep learning models' OOD generalization by decorrelating feature embeddings.
problem Out-of-distribution generalization in deep learning models.
method PFDL algorithm that optimizes feature decomposition network and image classification model.
result PFDL improves the accuracy of image classification models on OOD datasets.
We study graph matching with correlated Gaussian features and find thresholds for exact recovery.
problem Graph matching with correlated Gaussian features.
method Information-theoretic thresholds and conditions for exact and almost exact recovery.
result Contextual information introduces a richer structure, with thresholds for exact and almost exact recovery no longer coinciding.
Motivated by an abstract notion of low-level edge detector filters, we propose a simple method of unsupervised feature construction based on pairwise statistics of features. In the first step, we construct neighborhoods of features by regrouping features that correlate. Then we use these subsets as filters to produce n…
Losaw improves FI scores by decorrelating features in ML models.
problem Feature correlation distorts feature importance scores in ML models.
method Losaw uses local sample weighting to decorrelate features.
result Losaw consistently improves feature importance scores and prediction accuracy.
Paper studies vertex correspondence recovery in correlated graphs with node features.
problem Recovering hidden vertex correspondence between two correlated graphs with observed edge weights and node features.
method Introduced featured correlated Gaussian Wigner model and proposed QPAlign algorithm for quadratic programming relaxation.
result Characterized optimal information-theoretic thresholds for exact and partial recovery of latent mapping.
Interventional domain adaptation improves feature transferability by removing spurious correlations.
problem Improper feature transferability due to spurious correlations in domain adaptation.
method Intervention strategy using unlabeled target data to generate counterfactual features and train discriminability invariance.
result Consistent performance improvements over state-of-the-art approaches in various domain adaptation tasks.
New method corrects correlation bias in feature importance.
problem Correlations between features in high-dimensional data bias statistical and machine learning models.
method Pairwise Permutation Algorithm (PPA)
result PPA corrects correlation bias, identifying biological relevant biomarkers.
The paper extends Pearson correlation to multi-variables, useful for noise measurement and feature selection.
problem The standard Pearson correlation coefficient is limited to two variables and doesn't meet the needs for multi-variable analysis.
method The authors use random matrix theory to extend Pearson's correlation coefficient to an arbitrary number of variables.
result The extended correlation coefficient is useful for gauging noise and selecting features, particularly in classification.
Gradient descent aligns neural feature matrices with pre-activation tangent features.
problem Understanding neural feature learning mechanisms.
method Analytical proof of alignment between weight matrices and pre-activation tangent features.
result Derivative alignment occurs almost surely in high-dimensional settings.
CaLoNet integrates spatial and local correlations for multivariate time series classification.
problem Ignoring spatial and local correlations in multivariate time series classification.
method Model spatial correlations using causality modeling, extract local correlations, integrate into graph neural network.
result Competitive performance compared to state-of-the-art methods on UEA datasets.
Unified framework for calculating Shapley values with correlated features.
problem Inaccurate Shapley values due to multicollinearity.
method Unified framework with matrix formulation and mathematical proof.
result Shapley values become independent of feature correlations.
Deep models learn spurious features correlated with target, but can still perform well.
problem Spurious correlations in feature learning.
method Empirical risk minimization and specialized group robustness training.
result Feature representations learned by ERM are competitive with specialized methods.
CCP clusters correlated features and projects them to 1D for efficient dimensionality reduction.
problem Efficiency in handling large datasets with high intrinsic dimensions.
method CCP partitions features into correlated clusters and projects them to 1D based on sample correlations.
result CCP achieves efficient dimensionality reduction without matrix diagonalization.
Paper proposes BSP to find stable bimodules of cross-correlated features.
problem Identify groups of features from two data types with strong cross-correlation.
method Iterative-testing based bimodule search procedure (BSP).
result BSP outperforms existing methods in detecting stable bimodules.
Study evaluates feature ranking methods' faithfulness in ML models, improving with dimensionality reduction.
problem Quantifying and improving the faithfulness of feature ranking methods in ML models.
method Evaluation of multiple feature ranking methods, including SHAP, LIME, ALE variance, and LR coefficients, using permutation importance as a baseline.
result Dimensionality reduction improves the faithfulness of feature ranking methods, making permutation importance the most faithful method.
GRIP2 improves deep learning feature selection robustness in correlated and noisy data.
problem Identifying predictive features in correlated and noisy data.
method Integrates first-layer feature activity over a two-dimensional regularization surface to control sparsity and geometry, using efficient block-stochastic sampling.
result Demonstrates improved robustness and power in high correlation and low signal-to-noise ratio regimes.
Feature selection has attracted significant attention in data mining and machine learning in the past decades. Many existing feature selection methods eliminate redundancy by measuring pairwise inter-correlation of features, whereas the complementariness of features and higher inter-correlation among more than two feat…
Fast feature selection for SHM using canonical correlation.
problem Feature selection for structural health monitoring.
method Greedy search of sum of squared canonical correlation coefficients.
result Extremely fast feature selection with good performance.
It has been shown that instead of learning actual object features, deep networks tend to exploit non-robust (spurious) discriminative features that are shared between training and test sets. Therefore, while they achieve state of the art performance on such test sets, they achieve poor generalization on out of distribu…
A new algorithm removes unexpected correlations in biased data for better clustering.
problem Clustering with selection bias in data.
method Decorrelation regularized K-Means (DCKM) algorithm.
result DCKM achieves significant performance gains on real-world datasets.
Paper proposes a new stock price forecasting method using DRAGAN and feature matching.
problem Capturing correlations and training instability in GANs for stock price forecasting.
method Introduces DRAGAN and feature matching for improved training stability and correlation capture.
result Proposed method outperforms LSTM and basic GANs in stock price forecasting.
Proposes PSCCA for estimating correlations and canonical correlations in sparse count data.
problem Estimating correlations and canonical correlations in sparse count data from next-generation sequencing.
method Probabilistic approach for sparse count data sets (PSCCA).
result PSCCA outperforms other methods in estimating true correlations and canonical correlations at the natural parameter level.
Multimodal language analysis often considers relationships between features based on text and those based on acoustical and visual properties. Text features typically outperform non-text features in sentiment analysis or emotion recognition tasks in part because the text features are derived from advanced language mode…
New method handles correlated genes for better genomic prediction.
problem Technical issues with highly correlated genes in prediction models.
method Grouping algorithm that treats correlated genes as a group and uses their common patterns.
result Significantly outperforms standard models in prediction and feature selection.
Private method measures nonlinear correlations between data hosted across two entities.
problem Measuring nonlinear correlations between sensitive data hosted across multiple parties while preserving privacy.
method Differentially private estimator of distance correlation.
result First private estimator of nonlinear correlations in a multi-party setup.
We are often interested in explaining data through a set of hidden factors or features. When the number of hidden features is unknown, the Indian Buffet Process (IBP) is a nonparametric latent feature model that does not bound the number of active features in dataset. However, the IBP assumes that all latent features a…
In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cu…
Online algorithm detects and removes spurious features for strong generalization.
problem Predictive models learn spurious correlations from incomplete data.
method Online continual learning algorithm that detects and removes spurious features.
result Algorithm converges to solutions with strong generalization.
One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation becomes an appealing objective because of its operational meaning and desirable propert…
An important problem in machine learning and statistics is to identify features that causally affect the outcome. This is often impossible to do from purely observational data, and a natural relaxation is to identify features that are correlated with the outcome even conditioned on all other observed features. For exam…
The study identifies spurious correlations in high-dimensional regression and quantifies their impact.
problem Spurious correlations in high-dimensional regression models.
method Statistical characterization of spurious correlations, quantifying their amount via ridge regularization.
result The value of regularization strength that minimizes test loss is in an interval where spurious correlations increase.
Study feature representations induced by dependence between variables.
problem Learning feature representations from dependent random variables.
method Characterized sufficient and necessary conditions for dependence-induced representations, and provided a family of loss functions.
result Features learned from the family of loss functions can be expressed as the composition of a loss-dependent function and the maximal correlation function.
Paper evaluates and improves private feature selection methods.
problem Feature selection in high-dimensional datasets with privacy constraints.
method Correlations-based order statistic privatized for feature selection.
result Our method significantly outperforms established baseline for private feature selection.
Non-negative matrix factorization is a basic tool for decomposing data into the feature and weight matrices under non-negativity constraints, and in practice is often solved in the alternating minimization framework. However, it is unclear whether such algorithms can recover the ground-truth feature matrix when the wei…
We revisit the problem of feature selection in linear discriminant analysis (LDA), that is, when features are correlated. First, we introduce a pooled centroids formulation of the multiclass LDA predictor function, in which the relative weights of Mahalanobis-transformed predictors are given by correlation-adjusted t…
To date, the instability of prognostic predictors in a sparse high dimensional model, which hinders their clinical adoption, has received little attention. Stable prediction is often overlooked in favour of performance. Yet, stability prevails as key when adopting models in critical areas as healthcare. Our study propo…
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
New stability measures for similar features improve feature selection accuracy.
problem Existing stability measures fail to distinguish similar features in highly correlated datasets.
method Introduce new adjusted stability measures that consider feature similarities.
result One new stability measure considers highly similar features as interchangeable.
Proposes a method to select features for deep learning in noisy, high-dimensional data.
problem Feature selection for deep learning in ultra-high dimensional and highly correlated data.
method Data-adaptive multi-resolutional screening and cleaning with deep learning.
result Achieves high power while keeping false discovery rate low.
We introduce a method to predict which correlation matrix coefficients are likely to change their signs in the future in the high-dimensional regime, i.e. when the number of features is larger than the number of samples per feature. The stability of correlation signs, two-by-two relationships, is found to depend on thr…
In many high dimensional classification or regression problems set in a biological context, the complete identification of the set of informative features is often as important as predictive accuracy, since this can provide mechanistic insight and conceptual understanding. Lasso and related algorithms have been widely …