Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a network must determine whether or not an input is outside of the set on which it is expected to safel…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “out-of-distribution examples”
XEnsemble improves DNN robustness against adversarial and out-of-distribution inputs.
problem Protecting DNN models from adversarial and out-of-distribution inputs.
method Diverse input denoising verifiers and disagreement-diversity ensemble learning.
result XEnsemble achieves high defense and detection success rates.
Paper proposes detecting OOD examples using Gram matrices and in-distribution data.
problem Detecting OOD examples with confidence and without OOD data.
method Characterize activity patterns with Gram matrices and identify anomalies in values.
result High OOD detection rates achieved without OOD data.
Open-world evasion attacks using OOD adversarial examples outperform in-distribution attacks.
problem Evasion attacks on deep learning systems in open-world environments.
method Using 11 neural network models trained on 3 datasets, the study demonstrates the effectiveness of OOD adversarial examples against state-of-the-art detectors and defenses.
result OOD adversarial examples can achieve up to 4x higher success rates than in-distribution adversarial examples against known defenses.
A new method approximates pNML for faster out-of-distribution detection.
problem Detecting out-of-distribution examples efficiently.
method Influence functions approximation of pNML for neural networks.
result The approximation effectively detects out-of-distribution examples.
Improves out-of-distribution detection in neural networks.
problem Detecting out-of-distribution examples in neural networks.
method Normalizing flows and residual flow architecture for expressive density modeling.
result Significantly improved true negative rate (77.5%) compared to state-of-the-art (56.7%).
New datasets make AI models fail reliably, highlighting weaknesses.
problem AI models' vulnerabilities to adversarial examples.
method Created challenging datasets with adversarial filtering, tested on various models.
result Existing models perform poorly on new datasets, revealing shared weaknesses.
Deep learning models are known to be overconfident in their predictions on out of distribution inputs. There have been several pieces of work to address this issue, including a number of approaches for building Bayesian neural networks, as well as closely related work on detection of out of distribution samples. Recent…
New method improves robustness of OOD detection models.
problem Detecting out-of-distribution inputs is critical for deep learning models.
method Proposes ALOE algorithm for robust training with adversarially crafted examples.
result ALOE substantially improves robustness of OOD detection on CIFAR-10 and CIFAR-100 datasets.
Improved OOD detection using label smoothing and k-NN density estimates.
problem Detecting out-of-distribution examples in classification models.
method Label smoothing and k-NN density estimate on intermediate activations.
result Label smoothing improves OOD detection performance, both theoretically and empirically.
Combining training and post-training methods improves OOD detection accuracy.
problem Deep networks struggle with OOD detection.
method Divided OOD detection methods into training and post-training, then combined them.
result State-of-the-art results in OOD detection achieved.
Deep Neural Networks are powerful models that attained remarkable results on a variety of tasks. These models are shown to be extremely efficient when training and test data are drawn from the same distribution. However, it is not clear how a network will act when it is fed with an out-of-distribution example. In this …
A new metric predicts model performance on unseen data.
problem Predicting performance on out-of-distribution data without labels.
method Uses model predictions to pseudo-label data, trains a new model, and measures difference from in-distribution models.
result Empirically outperforms existing methods on image and text classification tasks.
Paper proposes contrastive training to improve OOD detection without needing explicit OOD examples.
problem Improving reliable detection of out-of-distribution inputs for machine learning systems.
method Contrastive training approach that doesn't require explicit OOD examples, using CLP score.
result Contrastive training significantly improves OOD detection performance on benchmarks, especially in near OOD classes.
Estimates prediction uncertainty in neural networks using density estimation in representation space.
problem Incorrect predictions with high confidence from models trained on limited data.
method Estimates training data density in representation space and uses it to predict model uncertainty.
result Detects out-of-distribution data without prior exposure, improving model reliability.
Study optimal ridge regularization for out-of-distribution prediction.
problem Optimal ridge regularization for predicting out-of-distribution data.
method Established conditions for optimal regularization under covariate and regression shifts, proving monotonic risk in data aspect ratio.
result Negative regularization can be optimal under shifts, even with isotropic or underparameterized training features.
In-N-Out improves model robustness to out-of-distribution data.
problem Learning robust models with few in-distribution labeled examples.
method Pre-training with auxiliary information and self-training with pseudolabels.
result In-N-Out outperforms auxiliary inputs or outputs alone on both in-distribution and OOD error.
New methods predict language model out-of-distribution behaviors using causal mechanisms.
problem Predicting how language models behave on unseen data.
method Two methods: counterfactual simulation and value probing.
result Both methods achieve high AUC-ROC and outperform causal-agnostic approaches in out-of-distribution settings.
The discriminative approach to classification using deep neural networks has become the de-facto standard in various fields. Complementing recent reservations about safety against adversarial examples, we show that conventional discriminative methods can easily be fooled to provide incorrect labels with very high confi…
With the recently rapid development in deep learning, deep neural networks have been widely adopted in many real-life applications. However, deep neural networks are also known to have very little control over its uncertainty for unseen examples, which potentially causes very harmful and annoying consequences in practi…
Proposes a method to detect out-of-distribution samples without OOD training data.
problem Inability of neural networks to detect novel class distributions.
method Outlier Exposure with Confidence Control (OECC) loss function.
result Superior OOD detection performance on image and text classification tasks.
Novel neural network models quantify uncertainty for deep classifiers.
problem Deep networks' overconfidence and ignorance about uncertainty.
method Variational autoencoders and GANs generate out-of-distribution samples.
result Better uncertainty estimates for in- and out-of-distribution samples.
Generative classifier derived from any discriminative classifier rejects illegal inputs.
problem Detecting and rejecting illegal inputs like adversarial examples and out-of-distribution samples.
method SDIM-logit: learns generative classifier from logits of any discriminative classifier, imposing statistical constraints.
result SDIM-logit inherits performance of base classifier without loss and can reject illegal inputs.
We consider the problem of detecting out-of-distribution images in neural networks. We propose ODIN, a simple and effective method that does not require any change to a pre-trained neural network. Our method is based on the observation that using temperature scaling and adding small perturbations to the input can separ…
Algorithm predicts with optimal loss by abstaining from uncertain test examples.
problem Predicting with training data not matching test data.
method Transductive abstention algorithm using labeled and unlabeled test examples.
result Optimal prediction loss guarantees with additional term for abstaining cost.
This paper improves normalizing flows by combining MLE and sliced-Wasserstein distance for better data fidelity.
problem Normalizing flows struggle with generating realistic data and detecting out-of-distribution data.
method Proposes a hybrid objective function combining MLE and sliced-Wasserstein distance.
result Shows better generative abilities and lower likelihood of out-of-distribution data.
C-Mixup improves generalization in regression tasks by adjusting label similarity.
problem Improving generalization in regression tasks with limited data.
method C-Mixup adjusts the probability of mixing examples based on label similarity.
result C-Mixup achieves better generalization and robustness compared to vanilla mixup.
This paper benchmarks OoDD methods for medical imaging.
problem Medical models trained for one domain may fail on images from a different domain.
method Defined 3 categories of OoD examples and benchmarked methods in 3 medical imaging domains.
result Simple binary classifier on feature representation yields best accuracy and AUPRC.
The study analyzes how deep neural networks treat instances with regular and irregular patterns.
problem Understanding how deep neural networks handle both common and rare patterns in data.
method Characterizes instances using a consistency score based on training data sets of varying sizes.
result The consistency score identifies out-of-distribution and mislabeled examples, distinguishing them from strongly regular examples.
The paper proposes a consensus algorithm to improve deep neural network interpretability and accuracy in mortality prediction.
problem The black-box nature and overgeneralization of deep neural networks in healthcare applications.
method An (, ) consensus algorithm that is insensitive to adversarial examples and can reliably reject out-of-distribution samples.
result The consensus algorithm improves both prediction accuracy and interpretability of deep neural network models in mortality prediction.
Survey examines anomaly detection methods for deep learning.
problem Out-of-distribution and adversarial examples in deep learning.
method Taxonomy of existing anomaly detection techniques.
result Discussion of strengths and weaknesses of techniques.
Survey connects three DNN generalization fields for better inference.
problem Assessing if DNNs generalize correctly at inference time.
method Investigates predictive uncertainty, out-of-distribution, and adversarial example detection.
result Connects three fields for better DNN inference.
A new loss function improves neural networks' out-of-distribution detection without side effects.
problem Neural networks struggle with out-of-distribution detection due to SoftMax loss issues.
method Proposes IsoMax loss replacing SoftMax loss, maintaining high entropy and fast inferences.
result Significantly improves neural networks' out-of-distribution detection performance.
Pre-training improves model robustness and uncertainty.
problem Improving model robustness and uncertainty in machine learning.
method Adversarial pre-training and task-specific methods.
result Approximately a 10% absolute improvement in adversarial robustness.
p-DkNN uses deep representations to detect out-of-distribution data with statistical tests.
problem Lack of reliable confidence estimates in neural networks for safety-critical applications.
method Statistical testing of deep neural network's intermediate hidden representations.
result p-DkNN enables more accurate and reliable predictions by abstaining from incorrect predictions.
Proposes CSG model to separate semantic and variation factors for OOD prediction.
problem Out-of-distribution examples cause conventional models to mix semantic and variation factors, leading to poor performance.
method Causal Semantic Generative model (CSG) based on causal reasoning, using variational Bayes for efficient learning and prediction.
result CSG can identify semantic factor and improve OOD prediction performance.
Understanding normalization in contrastive representation learning and out-of-distribution detectioncs.CV
Method enhances anomaly detection using contrastive learning and out-of-distribution data.
problem Improving anomaly detection in datasets with limited out-of-distribution data.
method Proposes a contrastive learning method that incorporates out-of-distribution data to enhance anomaly detection performance.
result The method significantly improves anomaly detection performance, even with limited out-of-distribution data.
Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic taskscs.LG
Transformers learn to solve modular arithmetic tasks by in-context learning and skill composition.
problem Understanding how large language models generalize to unseen tasks in modular arithmetic.
method Pre-training on a set of modular arithmetic tasks and evaluating out-of-distribution performance.
result Transformers require two transformer blocks for out-of-distribution generalization, and deeper models exhibit transient out-of-distribution performance.
Simple methods combine statistical tests for out-of-distribution detection.
problem Detecting data points not following the training distribution.
method Combining classical parametric tests (Rao's score test) and a typicality test.
result Combining Fisher's method of test statistics improves out-of-distribution detection accuracy.
ATOM improves robust OOD detection by mining informative auxiliary examples.
problem Robust OOD detection in open-world settings is challenging due to adversarial inputs.
method ATOM combines adversarial training with outlier mining to improve robustness.
result ATOM achieves state-of-the-art performance in OOD detection, reducing FPR by up to 57.99%.
DAEDL improves EDL's OOD detection and classification performance by integrating feature space density.
problem Limited OOD detection and classification performance of EDL.
method Integrates feature space density with EDL's output and uses a novel parameterization.
result Demonstrates state-of-the-art performance across uncertainty estimation and classification tasks.
Robust collision prediction for robots using generative uncertainty.
problem Robots need to predict collisions safely in unexpected situations.
method Combines generative modeling and model uncertainty to handle out-of-distribution inputs.
result Improves collision prediction performance, especially for out-of-distribution inputs.
Bayesian Gaussian Processes layer detects out-of-distribution data in medical imaging.
problem Detecting out-of-distribution data in medical imaging tasks.
method Parameter-efficient hierarchical convolutional Gaussian Processes in Wasserstein-2 space.
result Uncertainty estimates enable superior out-of-distribution detection compared to previous methods.
Combining Bayesian deep learning and split conformal prediction affects out-of-distribution coverage.
problem Improving out-of-distribution coverage in multiclass image classification.
method Combining Bayesian deep learning with split conformal prediction methods.
result Combining methods can reduce out-of-distribution coverage in some cases.
A new PLL method uses class activation values to improve robustness.
problem Weakly supervised learning with noisy data and adversarial perturbations.
method Subjective logic with class activation values for uncertainty representation and label weight re-distribution.
result More robust predictions under high noise levels, out-of-distribution examples, and adversarial perturbations.
FRODO method rejects out-of-distribution chest x-ray images with high accuracy.
problem Rejecting out-of-distribution samples in medical image analysis.
method Uses feature activations and Mahalanobis distance to measure distribution mismatch.
result Achieves an AUC score of 0.99 in classifying chest x-ray images as in or out of distribution.
DAB enriches deep networks with uncertainty estimates using a codebook of training inputs.
problem Lack of uncertainty estimation in deep neural networks.
method Distance Aware Bottleneck (DAB) method that learns a codebook of training inputs.
result DAB achieves better OOD detection and misclassification prediction than prior methods.
Popular deep learning uncertainty estimation methods often mislead on out-of-distribution data.
problem Misleading uncertainty estimates on out-of-distribution data.
method Analysis of Gaussian process, Bayesian neural networks, and Monte Carlo dropout methods.
result BNNs and MCDropout do not always provide high uncertainty estimates on out-of-distribution samples.