New method improves OOD detection without sacrificing generalization.
problem Improving OOD detection models that also generalize well.
method Decouples uncertainty learning from Bayesian perspective.
result Achieves state-of-the-art OOD detection performance.
Paper investigates learnability of OOD detection under various conditions.
problem Learnability of OOD detection under different scenarios.
method Investigates PAC learning theory, proves impossibility theorems, and provides necessary and sufficient conditions.
result Some conditions for learnability of OOD detection may not hold in practical scenarios.
Paper investigates OOD detection learnability under various conditions.
problem Learnability of OOD detection under diverse and unknown test data.
method PAC learning theory applied to OOD detection, proving impossibility theorems and necessary conditions.
result Some conditions for learnability hold in practical scenarios.
Study investigates OOD generalization methods for mechanics problems.
problem Real-world mechanics problems with unknown test environments and data distribution shifts.
method Investigates OOD generalization methods for regression problems in mechanics.
result OOD generalization methods perform better than traditional ML methods on mechanics-specific regression problems.
FeAT improves OOD generalization by learning richer features.
problem Improving feature learning for out-of-distribution (OOD) generalization.
method Feature Augmented Training (FeAT) iteratively augments and retains features from different subsets of training data.
result FeAT effectively learns richer features, boosting OOD performance.
Paper presents a novel framework for OOD learning with human feedback.
problem Multifaceted challenges of OOD generalization and detection.
method Selective human feedback for labeling a small number of informative samples from wild data distribution.
result Enhanced robustness and precision in handling OOD scenarios.
Paper introduces Influence Function to assess OOD generalization stability.
problem Assessing OOD generalization accuracy when target domains are unknown.
method Introduced Influence Function from robust statistics to monitor model stability.
result Accuracy on test domains and Influence Function variance can distinguish OOD algorithms and generalization quality.
Unified framework for OOD detection and generalization using graph theory.
problem Challenges in out-of-distribution (OOD) generalization and detection in real-world machine learning models.
method Graph-theoretic framework to jointly tackle OOD generalization and detection.
result Empirical validation of theoretical underpinnings with competitive performance.
ATOM improves robust OOD detection by mining informative auxiliary examples.
problem Robust OOD detection in open-world settings is challenging due to adversarial inputs.
method ATOM combines adversarial training with outlier mining to improve robustness.
result ATOM achieves state-of-the-art performance in OOD detection, reducing FPR by up to 57.99%.
By design, discriminatively trained neural network classifiers produce reliable predictions only for in-distribution samples. For their real-world deployments, detecting out-of-distribution (OOD) samples is essential. Assuming OOD to be outside the closed boundary of in-distribution, typical neural classifiers do not c…
Geometrically, high-likelihood regions in DGMs are unlikely to generate OOD data.
problem The paradox of high-likelihood OOD detection in deep generative models.
method Local intrinsic dimension estimation to identify high-likelihood regions that do not generate OOD data.
result A method pairing likelihoods and LID estimates for reliable OOD detection.
Study OOD generalization in meta-reinforcement learning using information theory.
problem Understanding how meta-reinforcement learning handles distribution shifts.
method Information-theoretic analysis of Markov Decision Processes and gradient-based algorithms.
result Established fine-grained generalization bounds for meta-reinforcement learning.
Vanilla CNNs, as uncalibrated classifiers, suffer from classifying out-of-distribution (OOD) samples nearly as confidently as in-distribution samples. To tackle this challenge, some recent works have demonstrated the gains of leveraging available OOD sets for training end-to-end calibrated CNNs. However, a critical que…
Gaussian process detects OOD data without needing OOD samples.
problem Overconfident predictions from DNNs on OOD data.
method Gaussian process for uncertainty quantification and decision boundary.
result Outperforms state-of-the-art methods in OOD detection.
PAIR optimizes machine learning models to generalize better to out-of-distribution data.
problem Optimization of machine learning models for out-of-distribution (OOD) generalization often leads to compromises that weaken robustness.
method Introduces a multi-objective optimization (MOO) perspective and a new optimization scheme called PAreto Invariant Risk Minimization (PAIR).
result PAIR improves robustness of OOD objectives by cooperatively optimizing with other objectives, yielding top OOD performances.
The paper exposes common misconceptions about OOD detection and proposes a new framework.
problem Density-based OOD detection fails in deep learning settings.
method Proposes the OOD proxy framework to unify likelihood-ratio-based methods.
result Likelihood ratio is a principled method for OOD detection.
OOD detection methods often misidentify OOD points, leading to ineffective safety improvements.
problem Improving model safety through OOD detection methods often leads to incorrect identification of out-of-distribution points.
method Re-examine popular OOD detection procedures based on predictive uncertainty or features of supervised models trained on in-distribution data.
result Popular OOD detection methods incorrectly conflate high uncertainty and far feature-space distance with being out-of-distribution.
MADOD meta-learns invariant features for OOD detection across unseen domains.
problem Simultaneous covariate and semantic shifts in real-world machine learning applications.
method Meta-learning and G-invariance to learn robust, domain-invariant features.
result Superior performance in semantic OOD detection across unseen domains.
Current OOD benchmarks overestimate model robustness to spurious correlations.
problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.
method Analyze OOD datasets for spurious correlations and derive conditions for robustness.
result Current OOD benchmarks are misspecified and overestimate model robustness.
Enhances OOD detection using latent diffusion for more robust and efficient training.
problem Improving reliability of machine learning models in real-world scenarios.
method Proposes Outlier-Aware Learning (OAL) framework that generates synthetic OOD data in latent space and uses MICL and KD modules.
result Demonstrates superior performance on benchmark datasets.
WOODS benchmarks improve understanding of time series OOD generalization.
problem Limited understanding of OOD generalization in time series.
method Presented eight open-source time series benchmarks and revised OOD algorithms.
result Large room for improvement in OOD generalization algorithms for time series.
Out-of-distribution (OoD) detection is a natural downstream task for deep generative models, due to their ability to learn the input probability distribution. There are mainly two classes of approaches for OoD detection using deep generative models, viz., based on likelihood measure and the reconstruction loss. However…
Neural networks classify OOD images by their nearest neighbor in training data.
problem Understanding out-of-distribution prediction behavior of neural networks.
method Nearest category generalization (NCG) measure to assess OOD prediction accuracy.
result Adversarially robust networks have higher NCG accuracy than natural training, indicating local regularization impacts decision regions.
Deep probabilistic generative models enable modeling the likelihoods of very high dimensional data. An important application of generative modeling should be the ability to detect out-of-distribution (OOD) samples by setting a threshold on the likelihood. However, some recent studies show that probabilistic generative …
NECO detects out-of-distribution data using neural collapse properties.
problem Detecting out-of-distribution data in machine learning models.
method NECO leverages neural collapse geometric properties to identify OOD data.
result NECO achieves state-of-the-art results on OOD detection tasks.
Training on some out-of-distribution data improves generalization error before it deteriorates.
problem Generalization error improves with some out-of-distribution data but deteriorates with more.
method Synthetic datasets and deep networks on computer vision benchmarks.
result Non-monotonic trend in generalization error with OOD samples.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
New method finds unbiased subnetworks in biased models for better OOD performance.
problem How to improve out-of-distribution generalization in deep models.
method Functional modular probing method and Modular Risk Minimization.
result Even in biased models, there are unbiased subnetworks that can achieve better OOD performance.
The paper explores how model complexity affects OOD detection performance.
problem Ensuring reliability and safety of machine learning systems through OOD detection.
method Investigates the relationship between model capacity and OOD detection performance using empirical and theoretical analysis.
result The Double Descent phenomenon is observed in post-hoc OOD detection, indicating that overparameterization can enhance OOD detection.
Framework reduces simplicity bias in NNs, improving OOD generalization and robustness.
problem Simplicity bias in deep learning models leads to biased predictions and poor OOD generalization.
method Proposes a framework that regularizes conditional mutual information to encourage use of diverse features.
result Demonstrates effectiveness in various settings, enhancing OOD generalization and robustness.
FOOD detects out-of-distribution samples quickly without needing OOD data.
problem Detecting out-of-distribution samples efficiently in neural networks.
method Extended DNN classifier with Gaussian layer and log likelihood ratio test.
result FOOD achieves state-of-the-art performance and is fast and applicable.
Over-parameterized models reduce Out-of-Distribution (OOD) generalization loss.
problem Understanding how over-parameterized models handle non-trivial distributional shifts.
method Investigating random feature models and examining non-trivial natural distributional shifts.
result Increasing model parameterization reduces OOD loss.
Bayesian method detects outliers and uncertain points in data.
problem Detecting outliers and uncertain points in data using Bayesian methods.
method Generative model of data curation for aleatoric uncertainty, combining with epistemic uncertainty and outlier exposure.
result Principled Bayesian approach outperforms methods using aleatoric or epistemic uncertainty alone.
BPVAE enhances VAE robustness to OOD inputs.
problem VAEs struggle with OOD detection, assigning higher likelihoods to some OOD samples.
method Combines VAE with two independent priors: training dataset and simple dataset.
result BPVAE outperforms standard VAEs in OOD detection and generalization.
When presented with Out-of-Distribution (OOD) examples, deep neural networks yield confident, incorrect predictions. Detecting OOD examples is challenging, and the potential risks are high. In this paper, we propose to detect OOD examples by identifying inconsistencies between activity patterns and class predicted. We …
Machine learning models encounter Out-of-Distribution (OoD) errors when the data seen at test time are generated from a different stochastic generator than the one used to generate the training data. One proposal to scale OoD detection to high-dimensional data is to learn a tractable likelihood approximation of the tra…
A new OOD detector using an overlap index improves accuracy without high computational costs.
problem Effective OOD detection for machine learning models in open-world scenarios.
method Proposes an overlap index-based confidence score function for OOD detection.
result The proposed method achieves competitive accuracy with lower computational costs compared to state-of-the-art detectors.
SFP prunes ID features to improve OOD generalization without domain data.
problem Improving out-of-distribution (OOD) generalization in biased models.
method Spurious Feature-targeted model Pruning (SFP) framework.
result SFP achieves optimal OOD generalization by pruning ID features.
RIA method improves OoD generalization for covariate shift.
problem Improving out-of-distribution generalization under covariate shift.
method Adversarial label invariant graph data augmentations for OoD generalization.
result RIA method achieves high accuracy compared to OoD baselines.
HOoD detects near-out-of-distribution groups in correlated biomedical assays.
problem Detecting near-out-of-distribution cases in biased or incomplete data.
method Projects correlated measurements through a trained model and uses permutation-based hypothesis tests.
result HOoD reliably identifies OoD groups, outperforming other detectors.
LoD improves model safety by integrating unlabeled wild data, reducing OOD misclassification.
problem Improving model safety and reliability using unlabeled wild data containing both in-distribution and out-of-distribution samples.
method Intentionally label-noisifying unlabeled wild data to enable joint learning of labeled ID and OOD data, distinguishing losses between ID and OOD samples.
result LoD framework achieves superior OOD detection without requiring thresholds, improving model safety.
Normalizing flows fail to detect OOD data due to learning local pixel correlations.
problem Detecting out-of-distribution data in machine learning systems.
method Investigated why normalizing flows fail to distinguish between in- and out-of-distribution data, and modified flow architecture to improve OOD detection.
result Modifying flow architecture can improve OOD detection by biasing the flow towards learning semantic structure of the target data.
Transformers learn to generalize unseen tasks by composing self-attention layers.
problem How Transformers generalize to unseen, out-of-distribution tasks.
method Examined synthetic examples and pretrained LLMs, focusing on induction heads and latent subspace.
result Transformers can learn hidden rules for unseen tasks by composing self-attention layers, achieving OOD generalization.
FROB model improves robustness and reliable confidence for few-shot OoD detection.
problem Challenges in few-shot classification and OoD detection due to limited samples and adversarial attacks.
method FROB model combines support boundary generation and few-shot Outlier Exposure (OE) for improved robustness and reliable confidence.
result FROB achieves generalization to unseen OoD and maintains robustness independent of few-shot number.
Paper proposes contrastive training to improve OOD detection without needing explicit OOD examples.
problem Improving reliable detection of out-of-distribution inputs for machine learning systems.
method Contrastive training approach that doesn't require explicit OOD examples, using CLP score.
result Contrastive training significantly improves OOD detection performance on benchmarks, especially in near OOD classes.
Discriminative neural networks offer little or no performance guarantees when deployed on data not generated by the same process as the training distribution. On such out-of-distribution (OOD) inputs, the prediction may not only be erroneous, but confidently so, limiting the safe deployment of classifiers in real-world…
Unified framework for OOD detection using class ratio estimation.
problem Density-based OOD detection is unreliable for OOD images.
method Unified framework that builds energy-based models and employs differing base distributions, directly estimating the density ratio through class ratio estimation.
result Competitive results on OOD image problems compared to recent work.
New analysis shows entropy term cancels out in likelihood-based OOD detection.
problem Curious likelihood values for out-of-distribution data.
method Decomposed average likelihood into KL divergence and entropy terms.
result Entropy term explains OOD behaviour and cancels out in expectation.