New method improves robustness of OOD detection models.
problem Detecting out-of-distribution inputs is critical for deep learning models.
method Proposes ALOE algorithm for robust training with adversarially crafted examples.
result ALOE substantially improves robustness of OOD detection on CIFAR-10 and CIFAR-100 datasets.
Neural networks classify OOD images by their nearest neighbor in training data.
problem Understanding out-of-distribution prediction behavior of neural networks.
method Nearest category generalization (NCG) measure to assess OOD prediction accuracy.
result Adversarially robust networks have higher NCG accuracy than natural training, indicating local regularization impacts decision regions.
Calibrated ensembles improve both ID and OOD accuracy in distribution shift.
problem Desired balance between in-distribution and out-of-distribution accuracy.
method Ensemble standard and robust models, calibrating on ID data only.
result ID-calibrated ensembles outperform state-of-the-art methods on multiple datasets.
ATOM improves robust OOD detection by mining informative auxiliary examples.
problem Robust OOD detection in open-world settings is challenging due to adversarial inputs.
method ATOM combines adversarial training with outlier mining to improve robustness.
result ATOM achieves state-of-the-art performance in OOD detection, reducing FPR by up to 57.99%.
Medix uses the median to detect outliers from unlabeled data for robust OOD detection.
problem Challenges in using unlabeled data for OOD detection due to mixed InD and OOD samples.
method Introduces Medix, a framework using the median operation to identify outliers from unlabeled data.
result Empirical results show Medix outperforms existing methods in open-world settings.
Current OOD benchmarks overestimate model robustness to spurious correlations.
problem Spurious correlations degrade OOD performance, but benchmarks show the opposite.
method Analyze OOD datasets for spurious correlations and derive conditions for robustness.
result Current OOD benchmarks are misspecified and overestimate model robustness.
MetaPhysiCa tackles robust physics-informed machine learning for OOD tasks.
problem Designing robust PIML methods for OOD forecasting tasks in physics.
method Meta-learning procedure for causal structure discovery including invariant risk minimization.
result Significantly outperforms existing PIML and deep learning methods in OOD tasks.
Proposes DCV-ROOD framework for robust OOD detection evaluation.
problem Ensuring reliable OOD detection methods under diverse conditions.
method Dual Cross-Validation (DCV) adapted for OOD detection evaluation.
result Demonstrates fast convergence to true performance of OOD detection methods.
FROB model improves robustness and reliable confidence for few-shot OoD detection.
problem Challenges in few-shot classification and OoD detection due to limited samples and adversarial attacks.
method FROB model combines support boundary generation and few-shot Outlier Exposure (OE) for improved robustness and reliable confidence.
result FROB achieves generalization to unseen OoD and maintains robustness independent of few-shot number.
AROS uses Lyapunov-stabilized embeddings to improve out-of-distribution detection robustness against adversarial attacks.
problem Robust out-of-distribution (OOD) detection against adversarial attacks.
method Neural Ordinary Differential Equations (NODEs) with Lyapunov stability theory for generating robust embeddings.
result Improves robust detection performance significantly, e.g., from 37.8% to 80.1% on CIFAR-10 vs. CIFAR-100.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
Framework reduces simplicity bias in NNs, improving OOD generalization and robustness.
problem Simplicity bias in deep learning models leads to biased predictions and poor OOD generalization.
method Proposes a framework that regularizes conditional mutual information to encourage use of diverse features.
result Demonstrates effectiveness in various settings, enhancing OOD generalization and robustness.
BPVAE enhances VAE robustness to OOD inputs.
problem VAEs struggle with OOD detection, assigning higher likelihoods to some OOD samples.
method Combines VAE with two independent priors: training dataset and simple dataset.
result BPVAE outperforms standard VAEs in OOD detection and generalization.
Paper introduces Influence Function to assess OOD generalization stability.
problem Assessing OOD generalization accuracy when target domains are unknown.
method Introduced Influence Function from robust statistics to monitor model stability.
result Accuracy on test domains and Influence Function variance can distinguish OOD algorithms and generalization quality.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
PAIR optimizes machine learning models to generalize better to out-of-distribution data.
problem Optimization of machine learning models for out-of-distribution (OOD) generalization often leads to compromises that weaken robustness.
method Introduces a multi-objective optimization (MOO) perspective and a new optimization scheme called PAreto Invariant Risk Minimization (PAIR).
result PAIR improves robustness of OOD objectives by cooperatively optimizing with other objectives, yielding top OOD performances.
Paper aims to ensure reliable detection of out-of-distribution data with certifiable worst-case guarantees.
problem Deep neural networks are overconfident with OOD inputs, posing safety risks.
method Enforces low confidence and bounds in an l∞-ball around OOD points using interval bound propagation (IBP). result Certifiable worst-case guarantees for OOD detection are possible without significant loss in accuracy.
A new algorithm detects out-of-distribution samples by concentrating them in feature space.
problem Building safe AI systems requires effective out-of-distribution detection.
method The paper proposes a novel algorithm based on the observation that OoD samples concentrate in feature space.
result The algorithm achieves state-of-the-art performance on various OoD detection benchmarks.
Enhances OOD detection using latent diffusion for more robust and efficient training.
problem Improving reliability of machine learning models in real-world scenarios.
method Proposes Outlier-Aware Learning (OAL) framework that generates synthetic OOD data in latent space and uses MICL and KD modules.
result Demonstrates superior performance on benchmark datasets.
New method tackles OOD robustness with a single additional variable.
problem Out-of-distribution generalization with unobserved confounders.
method Identifiability assumptions using a single additional variable.
result Superior empirical performance on benchmark tasks.
In-N-Out improves model robustness to out-of-distribution data.
problem Learning robust models with few in-distribution labeled examples.
method Pre-training with auxiliary information and self-training with pseudolabels.
result In-N-Out outperforms auxiliary inputs or outputs alone on both in-distribution and OOD error.
Robust reinforcement learning agents generalize well to out-of-distribution settings using pretrained representations.
problem Achieving sample-efficient reinforcement learning agents that generalize to real-world settings.
method Trained 240 representations and 10,000 RL policies on a simulated robotic setup, evaluating different pretrained VAE-based representations' effects on OOD generalization.
result Many reinforcement learning agents are surprisingly robust to realistic distribution shifts, including sim-to-real cases.
Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.
problem Deep learning models lack out-of-distribution robustness due to reliance on spurious features.
method Theoretical analysis and demonstration of counterfactual data augmentations performed by a context-guessing machine.
result Counterfactual data augmentations by a context-guessing machine do not lead to robust OOD classifiers.
New method improves OOD detection without sacrificing generalization.
problem Improving OOD detection models that also generalize well.
method Decouples uncertainty learning from Bayesian perspective.
result Achieves state-of-the-art OOD detection performance.
Paper presents a novel framework for OOD learning with human feedback.
problem Multifaceted challenges of OOD generalization and detection.
method Selective human feedback for labeling a small number of informative samples from wild data distribution.
result Enhanced robustness and precision in handling OOD scenarios.
ADB framework improves OOD generalization by increasing ID bias during training.
problem Machine learning models degrade on new data distributions.
method ADB framework introduces controlled statistical diversity during training.
result Higher in-distribution bias leads to better out-of-distribution generalization.
TopoGeoScore selects robust checkpoints using only source-domain representations.
problem Selecting robust checkpoints without target-domain labels or samples.
method Constructs class-conditional mutual k-nearest-neighbour graphs and extracts three interpretable signals.
result Source representations contain measurable global-local-topological evidence of robustness.
HOoD detects near-out-of-distribution groups in correlated biomedical assays.
problem Detecting near-out-of-distribution cases in biased or incomplete data.
method Projects correlated measurements through a trained model and uses permutation-based hypothesis tests.
result HOoD reliably identifies OoD groups, outperforming other detectors.
MADOD meta-learns invariant features for OOD detection across unseen domains.
problem Simultaneous covariate and semantic shifts in real-world machine learning applications.
method Meta-learning and G-invariance to learn robust, domain-invariant features.
result Superior performance in semantic OOD detection across unseen domains.
Framework uses human feedback to safely set OOD detection thresholds, reducing false positives.
problem Challenges in setting OOD detection thresholds for safety-critical applications.
method Mathematically grounded framework leveraging expert feedback to dynamically update thresholds.
result Guaranteed to meet FPR constraint while minimizing human feedback, maintaining FPR at most 5%.
Study investigates OOD generalization methods for mechanics problems.
problem Real-world mechanics problems with unknown test environments and data distribution shifts.
method Investigates OOD generalization methods for regression problems in mechanics.
result OOD generalization methods perform better than traditional ML methods on mechanics-specific regression problems.
WOOD detects out-of-distribution samples using Wasserstein distance.
problem Detecting samples from different distributions in neural networks.
method WOOD defines a Wasserstein-distance-based score to evaluate dissimilarity and solves an optimization problem.
result WOOD consistently outperforms other OOD detection methods.
A new OOD detector using an overlap index improves accuracy without high computational costs.
problem Effective OOD detection for machine learning models in open-world scenarios.
method Proposes an overlap index-based confidence score function for OOD detection.
result The proposed method achieves competitive accuracy with lower computational costs compared to state-of-the-art detectors.
This paper combines existing OOD detection methods to improve overall performance.
problem Improving robustness of neural networks in safety-critical applications.
method Integrates four strategies for combining multiple OOD detection scores.
result Enhanced OOD detection through multi-dimensional evaluation metrics.
Machine learning models encounter Out-of-Distribution (OoD) errors when the data seen at test time are generated from a different stochastic generator than the one used to generate the training data. One proposal to scale OoD detection to high-dimensional data is to learn a tractable likelihood approximation of the tra…
Study shows latent space OOD detection isn't a reliable proxy for model performance.
problem Evaluating and interpreting deep learning systems on real-world data.
method Empirical investigation of latent space OOD detection and classification accuracy using SAR datasets.
result OOD detection cannot be used as a proxy measure for model performance.
Paper proposes methods to help autonomous vehicles adapt to unexpected driving scenarios.
problem Autonomous vehicles struggle with unexpected driving conditions.
method Robust imitative planning (RIP) and adaptive robust imitative planning (AdaRIP) methods to detect and adapt to distribution shifts.
result Methods outperform current state-of-the-art approaches in nuScenes prediction challenge.
This paper examines how optimization methods affect the reliability of detecting inputs outside a model's training distribution.
problem The unreliability of deep neural networks on out-of-distribution inputs.
method Analysis of optimization methods' impact on OOD detection approaches.
result Optimization methods significantly influence the robustness of OOD detection approaches.
STOOD-X detects out-of-distribution samples without distributional assumptions and provides explainable visualizations.
problem Challenges in OOD detection, including restrictive assumptions, scalability issues, and lack of interpretability.
method Two-stage methodology combining statistical nonparametric test and explainability enhancements.
result Achieves competitive performance in high-dimensional and complex settings, with explainability framework enabling human oversight.
Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD det…
Normalizing flows fail to detect OOD data due to learning local pixel correlations.
problem Detecting out-of-distribution data in machine learning systems.
method Investigated why normalizing flows fail to distinguish between in- and out-of-distribution data, and modified flow architecture to improve OOD detection.
result Modifying flow architecture can improve OOD detection by biasing the flow towards learning semantic structure of the target data.
DRGO improves graph recommendation by mitigating noisy samples and enhancing out-of-distribution generalization.
problem Noisy samples in training data diminish recommendation systems' out-of-distribution generalization.
method DRGO uses a diffusion paradigm to reduce noisy effects and entropy regularization to avoid extreme weights.
result DRGO outperforms current methods in OOD recommendation across various distribution shifts.
Improved VAE model enhances uncertainty estimation for out-of-distribution samples.
problem VAEs assign higher likelihood to out-of-distribution inputs.
method INCPVAE integrates noise contrastive prior into VAEs for reliable uncertainty estimation.
result INCPVAE outperforms standard VAEs in uncertainty estimation for OOD inputs.
Robustness to out-of-distribution (OOD) data is an important goal in building reliable machine learning systems. Especially in autonomous systems, wrong predictions for OOD inputs can cause safety critical situations. As a first step towards a solution, we consider the problem of detecting such data in a value-based de…
MPT improves CNN and energy-based models' OOD detection and generalization.
problem Challenging out-of-distribution detection in computer vision.
method Applying Maximum Probability Theorem as a regularization scheme in CNN and energy-based models.
result MPT-based regularization strategy stabilizes and improves generalization and robustness of base models.
Universal approach combines OOD detection scores for robustness.
problem Combining diverse OOD detection scores for robustness.
method Quantile normalization to p-values, meta-analysis, probabilistic interpretation.
result Significantly improved robustness and performance across diverse OOD detection scenarios.
This paper improves model robustness to underrepresented groups using ranking metrics and reweighting.
problem Underrepresented groups suffer from low accuracy in models trained via ERM.
method Proposes Discounted Cumulative Gain (DCG) and Discounted Rank Upweighting (DRU) methods.
result Models trained with DRU show superior generalization to unseen groups.
New findings link causal models to strategic classification, improving robustness and alignment.
problem Strategic adaptation by users in classification tasks.
method Causal models to bound worst-case out-of-distribution risk.
result Causal classification optimizes classification error after adaptation under certain noise conditions.