Mitigates biases in reward models using variational inference.
problem Spurious correlations in reward models that align large language models with human preferences.
method Formulates data-generating process, identifies non-spurious latent variables, and uses variational inference to recover them.
result Effective mitigation of spurious correlation issues, yielding more robust reward models.
Sequential regression procedures can include spurious variables early, even in sparse settings.
problem Sequential regression procedures can select spurious variables early in rankings.
method Analysis of three sequential procedures: forward stepwise, lasso, and least angle regression.
result The first spurious variable is selected earlier as coefficients become denser.
Develops tools to decompose spurious variations in causal models.
problem Understanding and decomposing spurious variations in causal relationships.
method Formal tools for decomposing spurious effects in Markovian and Semi-Markovian models.
result First results on non-parametric decomposition of spurious effects and sufficient conditions for identification.
LaCIM avoids spurious correlation by modeling latent causal factors.
problem Avoiding spurious correlation in supervised learning.
method Introducing latent variables for causal prediction and optimizing over latent space.
result Improved interpretability, robustness, and prediction power on OOD scenarios.
Framework uses human annotations to make models robust to spurious correlations.
problem Machine learning models fail when unmeasured variables change test distributions.
method Human annotations to augment training examples, UV-DRO objective for robustness.
result Improvements of 5-10% on digit recognition task and 1.5-5% on NYPD Police Stops analysis.
Invariant Causal Set Covering Machines avoid spurious associations.
problem Learning algorithms for rule-based models are vulnerable to spurious associations.
method Building on invariant causal prediction, propose Invariant Causal Set Covering Machines for conjunctions/disjunctions of binary-valued rules.
result The method can identify causal parents of a variable of interest in polynomial time.
NURD improves model performance by distilling representations independent of nuisance variables.
problem Models trained under spurious correlations may fail on data with different nuisance-label relationships.
method Developed Nuisance-Randomized Distillation (NURD) to find representations independent of nuisance variables.
result NURD finds representations that perform better regardless of nuisance-label relationships.
The paper proposes a method to stabilize predictions by identifying causal variables using a seed variable.
problem Stable prediction across unknown test data with potential spurious correlations.
method Conditional independence test based algorithm using a seed variable to separate causal from non-causal variables.
result The algorithm precisely separates causal and non-causal variables for stable prediction across test data.
New findings on identifying latent variables in nonlinear ICA models.
problem Identifying latent variables in nonlinear ICA models is challenging due to spurious solutions.
method Proved that conformal maps are identifiable and provided theoretical results on preventing spurious solutions.
result Conformal maps are identifiable in nonlinear ICA models, preventing spurious solutions.
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
SkewSize detects model biases by analyzing mistakes across subgroups.
problem Benchmarking model performance in the presence of spurious correlations.
method Introducing SkewSize, a metric that captures bias from model mistakes.
result SkewSize highlights biases not captured by other metrics.
Removing spurious features can hurt model accuracy and disproportionately affect different groups.
problem Interference from spurious features in robust model performance across different groups.
method Characterization and analysis of spurious feature removal in noiseless overparameterized linear regression.
result Removal of spurious features can decrease accuracy and disproportionately affect different groups, even in balanced datasets.
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
This research examines rare spurious correlations in neural networks and their impact on accuracy and privacy.
problem Rare spurious correlations in neural networks and their privacy risks.
method Introducing spurious patterns correlated with a fixed class to a few training examples, analyzing ℓ2 regularization and Gaussian noise. result Rare spurious correlations can significantly impact neural network accuracy and privacy, and specific mitigation methods can be effective.
Paper proposes EILLS for invariant linear regression across environments.
problem Estimating true parameter and important variable set in multi-environment settings.
method Environment invariant linear least squares (EILLS) objective function.
result EILLS estimator achieves variable selection consistency and efficient estimation.
New method identifies cause-effect relations in multivariate time series data.
problem Identifying cause-effect relations in multivariate time series data.
method Fictitious vector autoregressive model to identify long-run relations and causality strength.
result High accuracy in identifying true cause-effect relations in simulations and climate change analysis.
Paper proposes mechanism learning to reverse causal inference in ML.
problem Machine learning models learn associational, not causal, relationships.
method Causally weighted Gaussian mixture models (CW-GMMs).
result CW-GMMs can deconfound observational data for reverse causal inference.
Mitigates spurious correlations without bias labels.
problem Spurious correlations bias model performance.
method Introduces a novel training objective and debiasing method DPR.
result DPR achieves state-of-the-art performance.
Complex-valued neural networks avoid spurious local minima.
problem Finding spurious local minima in neural networks.
method Proved no spurious local minima for shallow complex neural networks with quadratic activations.
result Complex-valued weights eliminate spurious local minima in neural networks.
CLIP models robustness to spurious features is re-evaluated using a new dataset.
problem Existing robustness tests of CLIP models may not fully reflect their performance on spurious features.
method Crafted a new dataset (CounterAnimal) to reveal CLIP models' reliance on realistic spurious features.
result CLIP models are robust to spurious features learned from their training data, not ImageNet.
Study identifies and analyzes spurious correlations in data-driven models.
problem Spurious correlations in data-driven models are unreliable and hard to detect.
method Collect and analyze synthetic datasets generated from causal graphs to investigate spurious correlations.
result Patterns connecting spurious correlation hypotheses and model design choices were observed.
Self-training avoids spurious features in domain adaptation.
problem Domain shift with large differences between source and target domains.
method Entropy minimization on unlabeled target data, initialized with a source classifier.
result Entropy minimization avoids using spurious features in large domain shifts.
Algorithm removes spurious concepts from neural network representations without harming task performance.
problem Spurious correlations hinder neural network out-of-distribution generalization.
method Iterative algorithm that identifies two orthogonal subspaces in neural network representation.
result Algorithm outperforms existing methods on computer vision and natural language processing benchmarks.
Nearest neighbor (k-NN) graphs are widely used in machine learning and data mining applications, and our aim is to better understand what they reveal about the cluster structure of the unknown underlying distribution of points. Moreover, is it possible to identify spurious structures that might arise due to sampling va…
Boosts causal discovery by dynamically reweighting samples to learn better DAGs.
problem Overfitting to easier-to-fit samples and violating homogeneity assumptions in causal discovery.
method Adaptive sample reweighting via ReScore function to upweight and downweight samples based on fitting quality.
result Consistent and significant boosts in structure learning performance on synthetic and real-world datasets.
Spurious valleys hinder optimization in neural networks.
problem Understanding why gradient descent succeeds in non-convex neural network optimization.
method Characterizing topological properties of neural network loss landscapes, focusing on spurious valleys.
result Intrinsic dimension determines the presence or absence of spurious valleys in overparametrized neural networks.
This work shows that a simple local search can recover true principal components in non-negative rank-1 RPCA.
problem Recovering true principal components in non-negative rank-1 robust principal component analysis with noisy measurements.
method Using the Burer-Monteiro approach to cast RPCA as a non-convex and non-smooth ℓ1 optimization problem. result The low-dimensional formulation of symmetric and asymmetric positive rank-1 RPCA has a unique global solution and no spurious local solutions.
This paper tackles confounding biases in data augmentation.
problem Mitigating spurious correlations and confounding variables in training data.
method Formal analysis and counterfactual data augmentation.
result Removing confounding biases leads to invariant features and better generalization.
Deep linear networks avoid spurious local minima under certain conditions.
problem Existence of spurious local minima in deep linear networks.
method Reduction to two-layer case, quadratic loss analysis, and perturbation argument to show full rank property.
result Deep linear networks have no spurious local minima under specific conditions.
Current OOD benchmarks overestimate model robustness to spurious correlations.
problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.
method Analyze OOD datasets for spurious correlations and derive conditions for robustness.
result Current OOD benchmarks are misspecified and overestimate model robustness.
AFR simplifies reducing reliance on spurious features, improving model performance.
problem Reducing reliance on spurious features for out-of-distribution generalization.
method Automatic Feature Reweighting (AFR) updates the model with a weighted loss.
result AFR improves model performance on benchmarks with minimal compute.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
problem Understanding the loss landscape of sparse neural networks, especially one-hidden-layer networks.
method Analyzes sparse networks with dense and sparse final layers, focusing on linear and non-linear models.
result Sparse networks can have no spurious valleys under certain conditions, but spurious valleys and minima can exist for wide sparse networks.
Paper shows moderate RIP is insufficient for avoiding spurious local minima in matrix recovery.
problem The need for moderate RIP to avoid spurious local minima in matrix recovery.
method Analyzes the necessity of RIP constants and provides counterexamples.
result Counterexamples show spurious local minima exist even with moderate RIP.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
Paper proves conditions for nonconvex matrix recovery to avoid spurious local minima.
problem Ensuring no spurious local minima in nonconvex matrix recovery.
method Sharp restricted isometry bounds proof technique.
result RIP constant of δ < 1/2 is necessary and sufficient for exact recovery.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
The paper proves deep ReLU networks avoid spurious local minima in NTK regime.
problem The existence of spurious local minima in deep ReLU neural networks.
method Theoretical proof under Neural Tangent Kernel regime.
result Deep ReLU networks do not lie in spurious local minima in NTK regime.
Logit correction improves model performance by correcting spurious correlations.
problem Spurious correlations lead to poor model performance during inference.
method Proposes logit correction (LC) loss to mitigate spurious correlations.
result LC loss outperforms state-of-the-art solutions by 5.5% absolute improvement.
Resolves spurious correlations in causal models via intervention design.
problem Spurious correlations lead to incorrect causal models in reinforcement learning environments.
method Proposes a method to design interventions that improve causal models by incentivizing agents to find errors.
result Experimental results show improved causal models compared to baselines.
Gradient flow in phase retrieval escapes spurious minima with high probability.
problem Understanding gradient-based optimization in high-dimensional non-convex functions.
method Analytical and numerical study of gradient dynamics in phase retrieval.
result Gradient flow avoids spurious minima by drifting along unstable directions.
The paper analyzes how deep models memorize spurious features.
problem Understanding how deep models memorize spurious features in training data.
method Characterizes spurious feature memorization via model stability and feature alignment.
result Memorization of spurious features weakens as generalization capability increases.
Online algorithm detects and removes spurious features for strong generalization.
problem Predictive models learn spurious correlations from incomplete data.
method Online continual learning algorithm that detects and removes spurious features.
result Algorithm converges to solutions with strong generalization.
New method prevents classifiers from relying on spurious correlations.
problem Group invariant learning fails to prevent classifiers from depending on spurious correlations.
method Statistical independence tests to construct groups and reweight samples by group label proportion.
result New method significantly outperforms existing group invariant learning methods in generalizing to spurious correlation shifts.
A model learns causal representations from high-dimensional data.
problem Challenges in learning causal representations from high-dimensional data.
method Formulated a latent variable decoder model, Decoder BCD, for Bayesian causal discovery.
result Shows that using known intervention targets as labels helps in unsupervised Bayesian inference over structure and parameters.
Improved Shapley Value method for better model interpretation.
problem Misunderstanding and incorrect interpretation of Shapley Values in machine learning models.
method Identification of null and active coalitions, coalitional Shapley Value computation.
result Correct computation and inference of important variables using Shapley Values.
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
New research shows spurious samples in deep models are not errors but a feature.
problem Whether spurious samples can be eliminated without losing model coverage.
method Experimental study on deep generative models.
result Spurious samples cannot be completely eliminated without losing model ability to generate some test samples.
The FastICA algorithm is one of the most popular iterative algorithms in the domain of linear independent component analysis. Despite its success, it is observed that FastICA occasionally yields outcomes that do not correspond to any true solutions (known as demixing vectors) of the ICA problem. These outcomes are comm…