Long-range memory in non-equilibrium systems is explained by spurious memory in SDEs, not fBm.
problem Understanding the origin of long-range memory in non-equilibrium systems.
method Analysis of burst and inter-burst duration PDFs in SDE-driven processes.
result Processes described by SDEs exhibit a power-law exponent of 3/2 for burst or inter-burst duration PDFs.
Consensual model explains spurious long-range memory in financial markets.
problem Understanding the origin of long-range memory in financial volatility.
method Non-linear stochastic differential equations.
result Empirical burst and inter-burst duration statistics can be explained by non-linear models.
DMs emerge from DenseAMs, transitioning from memorization to generalization.
problem Hindered memory retrieval in DenseAMs due to spurious states.
method Examined diffusion models through the lens of DenseAMs, focusing on their generative process.
result Identified a critical phase in DMs transitioning from memorization to generalization.
Method approximates first passage times for birth-death processes.
problem Approximating first passage times for birth-death processes.
method General method using birth-death process properties, Keilson's theorem, and Riemann sums.
result Closed-form expressions for first passage times.
Paper proposes a new test to detect spurious seasonality in time series data.
problem Detecting spurious seasonality in time series data.
method Developed a non-parametric test based on ordinal patterns using symbolic dynamics.
result The day-of-the-week effect is partly an artifact of hidden correlation structure.
Proposes a new birth-death process for better modeling of population dynamics.
problem Models of population or opinion dynamics with spurious long-range memory.
method Introduces Bessel-like birth-death process to address the spurious long-range memory.
result Derives equations for the burst and inter-burst duration of the new process.
Removing spurious features can hurt model accuracy and disproportionately affect different groups.
problem Interference from spurious features in robust model performance across different groups.
method Characterization and analysis of spurious feature removal in noiseless overparameterized linear regression.
result Removal of spurious features can decrease accuracy and disproportionately affect different groups, even in balanced datasets.
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
This research examines rare spurious correlations in neural networks and their impact on accuracy and privacy.
problem Rare spurious correlations in neural networks and their privacy risks.
method Introducing spurious patterns correlated with a fixed class to a few training examples, analyzing ℓ2 regularization and Gaussian noise. result Rare spurious correlations can significantly impact neural network accuracy and privacy, and specific mitigation methods can be effective.
Mitigates spurious correlations without bias labels.
problem Spurious correlations bias model performance.
method Introduces a novel training objective and debiasing method DPR.
result DPR achieves state-of-the-art performance.
Complex-valued neural networks avoid spurious local minima.
problem Finding spurious local minima in neural networks.
method Proved no spurious local minima for shallow complex neural networks with quadratic activations.
result Complex-valued weights eliminate spurious local minima in neural networks.
CLIP models robustness to spurious features is re-evaluated using a new dataset.
problem Existing robustness tests of CLIP models may not fully reflect their performance on spurious features.
method Crafted a new dataset (CounterAnimal) to reveal CLIP models' reliance on realistic spurious features.
result CLIP models are robust to spurious features learned from their training data, not ImageNet.
Study nonconvex matrix completion for low-rank approximation without rank assumptions.
problem Low-rank approximation of positive semidefinite matrices from partial entries.
method Nonconvex optimization, local-minimum analysis, no spurious local minima.
result Improved sampling rate for nonconvex matrix completion with no spurious local minima.
Study identifies and analyzes spurious correlations in data-driven models.
problem Spurious correlations in data-driven models are unreliable and hard to detect.
method Collect and analyze synthetic datasets generated from causal graphs to investigate spurious correlations.
result Patterns connecting spurious correlation hypotheses and model design choices were observed.
Self-training avoids spurious features in domain adaptation.
problem Domain shift with large differences between source and target domains.
method Entropy minimization on unlabeled target data, initialized with a source classifier.
result Entropy minimization avoids using spurious features in large domain shifts.
Algorithm removes spurious concepts from neural network representations without harming task performance.
problem Spurious correlations hinder neural network out-of-distribution generalization.
method Iterative algorithm that identifies two orthogonal subspaces in neural network representation.
result Algorithm outperforms existing methods on computer vision and natural language processing benchmarks.
Spurious valleys hinder optimization in neural networks.
problem Understanding why gradient descent succeeds in non-convex neural network optimization.
method Characterizing topological properties of neural network loss landscapes, focusing on spurious valleys.
result Intrinsic dimension determines the presence or absence of spurious valleys in overparametrized neural networks.
Deep linear networks avoid spurious local minima under certain conditions.
problem Existence of spurious local minima in deep linear networks.
method Reduction to two-layer case, quadratic loss analysis, and perturbation argument to show full rank property.
result Deep linear networks have no spurious local minima under specific conditions.
Current OOD benchmarks overestimate model robustness to spurious correlations.
problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.
method Analyze OOD datasets for spurious correlations and derive conditions for robustness.
result Current OOD benchmarks are misspecified and overestimate model robustness.
Mitigates biases in reward models using variational inference.
problem Spurious correlations in reward models that align large language models with human preferences.
method Formulates data-generating process, identifies non-spurious latent variables, and uses variational inference to recover them.
result Effective mitigation of spurious correlation issues, yielding more robust reward models.
AFR simplifies reducing reliance on spurious features, improving model performance.
problem Reducing reliance on spurious features for out-of-distribution generalization.
method Automatic Feature Reweighting (AFR) updates the model with a weighted loss.
result AFR improves model performance on benchmarks with minimal compute.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
problem Understanding the loss landscape of sparse neural networks, especially one-hidden-layer networks.
method Analyzes sparse networks with dense and sparse final layers, focusing on linear and non-linear models.
result Sparse networks can have no spurious valleys under certain conditions, but spurious valleys and minima can exist for wide sparse networks.
Paper shows moderate RIP is insufficient for avoiding spurious local minima in matrix recovery.
problem The need for moderate RIP to avoid spurious local minima in matrix recovery.
method Analyzes the necessity of RIP constants and provides counterexamples.
result Counterexamples show spurious local minima exist even with moderate RIP.
PLUMAGE improves large model training efficiency and stability.
problem Accelerator memory and networking constraints during large model training.
method Probabilistic Low rank Unbiased Minimum Variance Gradient Estimator (PLUMAGE) that resolves bias and variance issues.
result PLUMAGE reduces training loss by 28% on average across the GLUE benchmark.
Riemannian gradient descent escapes some spurious critical points on low-rank matrix manifold.
problem Spurious critical points on the boundary of low-rank matrix manifold.
method Riemannian gradient descent with dynamical low-rank approximation and rescaled gradient flow.
result Riemannian gradient descent escapes some spurious critical points on the boundary of the manifold.
Paper proves conditions for nonconvex matrix recovery to avoid spurious local minima.
problem Ensuring no spurious local minima in nonconvex matrix recovery.
method Sharp restricted isometry bounds proof technique.
result RIP constant of δ < 1/2 is necessary and sufficient for exact recovery.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
The paper proves deep ReLU networks avoid spurious local minima in NTK regime.
problem The existence of spurious local minima in deep ReLU neural networks.
method Theoretical proof under Neural Tangent Kernel regime.
result Deep ReLU networks do not lie in spurious local minima in NTK regime.
Logit correction improves model performance by correcting spurious correlations.
problem Spurious correlations lead to poor model performance during inference.
method Proposes logit correction (LC) loss to mitigate spurious correlations.
result LC loss outperforms state-of-the-art solutions by 5.5% absolute improvement.
Resolves spurious correlations in causal models via intervention design.
problem Spurious correlations lead to incorrect causal models in reinforcement learning environments.
method Proposes a method to design interventions that improve causal models by incentivizing agents to find errors.
result Experimental results show improved causal models compared to baselines.
Develops tools to decompose spurious variations in causal models.
problem Understanding and decomposing spurious variations in causal relationships.
method Formal tools for decomposing spurious effects in Markovian and Semi-Markovian models.
result First results on non-parametric decomposition of spurious effects and sufficient conditions for identification.
Gradient flow in phase retrieval escapes spurious minima with high probability.
problem Understanding gradient-based optimization in high-dimensional non-convex functions.
method Analytical and numerical study of gradient dynamics in phase retrieval.
result Gradient flow avoids spurious minima by drifting along unstable directions.
The paper analyzes how deep models memorize spurious features.
problem Understanding how deep models memorize spurious features in training data.
method Characterizes spurious feature memorization via model stability and feature alignment.
result Memorization of spurious features weakens as generalization capability increases.
Online algorithm detects and removes spurious features for strong generalization.
problem Predictive models learn spurious correlations from incomplete data.
method Online continual learning algorithm that detects and removes spurious features.
result Algorithm converges to solutions with strong generalization.
New method prevents classifiers from relying on spurious correlations.
problem Group invariant learning fails to prevent classifiers from depending on spurious correlations.
method Statistical independence tests to construct groups and reweight samples by group label proportion.
result New method significantly outperforms existing group invariant learning methods in generalizing to spurious correlation shifts.
New research shows spurious samples in deep models are not errors but a feature.
problem Whether spurious samples can be eliminated without losing model coverage.
method Experimental study on deep generative models.
result Spurious samples cannot be completely eliminated without losing model ability to generate some test samples.
The FastICA algorithm is one of the most popular iterative algorithms in the domain of linear independent component analysis. Despite its success, it is observed that FastICA occasionally yields outcomes that do not correspond to any true solutions (known as demixing vectors) of the ICA problem. These outcomes are comm…
Paper trains models to resist spurious patterns by having humans revise documents.
problem Machine learning models often rely on spurious patterns, leading to poor performance.
method Humans revise documents to retain coherence and avoid unnecessary changes, creating a combined dataset.
result Models trained on combined datasets perform better than those trained on original or manipulated data alone.
Paper tackles spurious vanishing problem in approximate vanishing ideals.
problem Capturing nonlinear structure of perturbed data points leads to spurious vanishing problem.
method Proposes a general method integrating coefficient normalization and iterative basis construction.
result Proposed method overcomes spurious vanishing problem, resulting in shorter feature vectors.
Deep models learn spurious features correlated with target, but can still perform well.
problem Spurious correlations in feature learning.
method Empirical risk minimization and specialized group robustness training.
result Feature representations learned by ERM are competitive with specialized methods.
LaCIM avoids spurious correlation by modeling latent causal factors.
problem Avoiding spurious correlation in supervised learning.
method Introducing latent variables for causal prediction and optimizing over latent space.
result Improved interpretability, robustness, and prediction power on OOD scenarios.
Invariant Causal Set Covering Machines avoid spurious associations.
problem Learning algorithms for rule-based models are vulnerable to spurious associations.
method Building on invariant causal prediction, propose Invariant Causal Set Covering Machines for conjunctions/disjunctions of binary-valued rules.
result The method can identify causal parents of a variable of interest in polynomial time.
GRASP removes spurious correlations in fine-tuned models, improving task performance and reducing bias.
problem Fine-tuned models can latch onto spurious correlations, leading to bias and reduced generalization.
method GRASP identifies and removes spurious correlations from model weights without removing latent factors.
result GRASP significantly reduces bias and improves task performance in various fine-tuning tasks.
The study identifies spurious correlations in high-dimensional regression and quantifies their impact.
problem Spurious correlations in high-dimensional regression models.
method Statistical characterization of spurious correlations, quantifying their amount via ridge regularization.
result The value of regularization strength that minimizes test loss is in an interval where spurious correlations increase.
Gradient descent can learn a CNN architecture despite a spurious local minimum.
problem Learning a one-hidden-layer CNN with spurious local minima.
method Gradient descent with weight normalization from random initialization.
result Gradient descent can still recover true parameters with constant probability, even with a spurious local minimum.
Study on W2S generalization with spurious correlations, proposing remedies.
problem Understanding and improving W2S generalization with spurious correlations.
method Theoretical analysis and algorithmic remedies for W2S fine-tuning.
result W2S always happens with sufficient pseudolabels when group fractions match, but may fail otherwise.
New rationalization method avoids spurious correlations.
problem Spurious correlations in rationalization criteria.
method Invariant rationalization using game theory constraints.
result Rationales generalize better and align with human judgments.
Large learning rates enhance model robustness and compressibility.
problem Achieving robustness and resource-efficiency in machine learning models.
method Identifying and utilizing large learning rates as a facilitator for robustness and compressibility.
result Large learning rates produce desirable representation properties and compare favorably to other methods.