Enhances adversarial robustness with unlabeled out-of-domain data.
problem Improving robustness of models against adversarial attacks.
method Leveraging unlabeled data from multiple domains to bridge the sample complexity gap in adversarial robustness.
result Better adversarial robustness achieved when unlabeled data comes from a shifted domain.
New framework infers causal shifts in event sequences under out-of-domain interventions.
problem Inferring causal relationships in event sequences without considering out-of-domain interventions.
method Proposes a new causal framework to define ATE, designs an unbiased ATE estimator, and uses a Transformer-based neural network model.
result Demonstrates superior performance in ATE estimation and goodness-of-fit under out-of-domain-augmented point processes.
Researchers use human-in-the-loop to create counterfactually augmented data, improving model performance.
problem Creating ML models less reliant on spurious patterns in NLP datasets.
method A human-in-the-loop process to curate counterfactually augmented data (CAD), prohibiting unnecessary edits.
result Models trained on CAD appear to rely less on semantically irrelevant words and generalize better out of domain.
Framework uses unlabeled out-of-domain data to improve semi-supervised classification.
problem Improving generalization in semi-supervised classification problems.
method Combines Distributionally Robust Optimization (DRO) with self-supervised training.
result Significant improvement in generalization error compared to ERM.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
Valid certifies LLMs' domain adherence, bounding out-of-domain behavior.
problem Adversarial susceptibility of LLMs to generate out-of-domain outputs.
method VALID approach providing adversarial bounds as a certificate.
result Validates LLMs' domain adherence with meaningful certificates.
Out-of-domain (OOD) detection for low-resource text classification is a realistic but understudied task. The goal is to detect the OOD cases with limited in-domain (ID) training data, since we observe that training data is often insufficient in machine learning applications. In this work, we propose an OOD-resistant Pr…
Enhances out-of-domain calibration of neural networks.
problem Improving calibration performance of deep neural networks in out-of-domain settings.
method Consistency-guided temperature scaling (CTS) that considers style and content consistency.
result Significantly enhances out-of-domain calibration performance.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
Transformers can learn new tasks from diverse pretraining data but struggle with out-of-domain tasks.
problem Transformer models' ability to learn new tasks in-context is limited by their pretraining data coverage.
method Investigation of transformer models trained on (x,f(x)) pairs, comparing in-context learning capabilities across different task families. result Transformers can identify and learn within task families in their pretraining data but fail with out-of-domain tasks.
Proposes a measure to predict generalization in non-matching environments.
problem Characterizing and comparing generalization of machine learning models in non-matching environments.
method Neighborhood invariance measure, calculating invariance as the largest fraction of transformed points classified into the same class.
result Strong and robust correlation between neighborhood invariance and actual out-of-domain generalization.
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
fAux tests individual fairness without domain knowledge or out-of-domain samples.
problem Testing for individual fairness in machine learning models.
method Derivative comparison between model predictions and auxiliary model predictions.
result Effectively identifies discrimination on synthetic and real-world datasets.
AFTER technique improves NLP models by preventing overfitting to task-specific domains.
problem Standard fine-tuning degrades pretraining domain representations.
method Complements task-specific loss with adversarial objective.
result AFTER leads to improved performance on various NLP tasks.
The performance of a Part-of-speech (POS) tagger is highly dependent on the domain ofthe processed text, and for many domains there is no or only very little training data available. This work addresses the problem of POS tagging noisy user-generated text using a neural network. We propose an architecture that trains a…
Method combines LD and Fermat Distance for neural network uncertainty.
problem Measuring uncertainty in neural network predictions.
method Statistical Depth (LD) combined with Fermat Distance.
result Effective uncertainty estimation without impacting original model performance.
We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify aco…
The paper tackles extrapolation in extreme regions of regression problems.
problem Extrapolation on the tails of covariates in continuous regression problems.
method Statistical regression on a subsample of furthest observations, focusing on their angular components, using multivariate regular variation theory.
result Quantifies predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a bias-variance decomposition.
TESTED improves multi-domain stance detection with topic-guided sampling and contrastive learning.
problem Challenges in multi-domain stance detection due to domain-specific variations and imbalanced annotations.
method Topic-guided diversity sampling and contrastive learning objective.
result Significant improvement in F1 scores, up to 10.2 points out-of-domain.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
The Intensive Care Unit (ICU) is a hospital department where machine learning has the potential to provide valuable assistance in clinical decision making. Classical machine learning models usually only provide point-estimates and no uncertainty of predictions. In practice, uncertain predictions should be presented to …
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds …
This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.
problem Generalization from easy to hard tasks, especially out-of-domain generalization.
method Formalizes inductive generalization, introduces Inductive Learning, and outlines steps to adapt techniques for learning model successors.
result A new learning paradigm (Inductive Learning) that emphasizes induction and universal properties of learning and computation.
Periodic activation functions improve neural network reliability and interpretability.
problem Neural networks reinforce hidden biases, making them unreliable and hard to interpret.
method Introduce periodic activation functions in Bayesian neural networks to establish a connection with stationary Gaussian process priors.
result Periodic activation functions, including sinusoidal, triangular, and ReLU, improve model performance and sensitivity to perturbations.
Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to m…
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The u…
We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mi…
UBMF tackles fault diagnosis in imbalanced industrial data with enhanced accuracy and adaptability.
problem Fault diagnosis challenges in imbalanced industrial data.
method Integrates four key modules: data perturbation, cross-task feature extraction, uncertainty-based filtering, and Bayesian meta-knowledge integration.
result Achieves an average improvement of 42.22% across ten diagnostic tasks.
OT domain adaptation improves aphasia detection across languages.
problem Detecting aphasia in low-resource languages with limited data.
method Utilized OT domain adaptation to map linguistic features across multiple languages.
result OT domain adaptation significantly improved F1 scores for French and Mandarin aphasia detection.
For many years, i-vector based audio embedding techniques were the dominant approach for speaker verification and speaker diarization applications. However, mirroring the rise of deep learning in various domains, neural network based audio embeddings, also known as d-vectors, have consistently demonstrated superior spe…
Advances in deep neural network (DNN) based molecular property prediction have recently led to the development of models of remarkable accuracy and generalization ability, with graph convolution neural networks (GCNNs) reporting state-of-the-art performance for this task. However, some challenges remain and one of the …
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
The paper presents a framework to quantify the trade-off between synthetic and real data.
problem Improving generalization with synthetic data when real data is scarce.
method Learning-theoretic framework leveraging algorithmic stability to derive generalization error bounds.
result Optimal synthetic-to-real data ratio minimizing expected test error as a function of Wasserstein distance.
NP-PROV separates mean and variance spaces to improve function uncertainty.
problem Neural Processes fail on out-of-domain tasks due to shared latent space uncertainty.
method Separates mean and variance into function-value-related and position-related latent spaces.
result NP-PROV achieves state-of-the-art likelihood with bounded variance in drifts.
Paper presents a method to train NER models without labelled data using weak supervision.
problem Dealing with NER performance drop in new domains without labelled data.
method Weak supervision through automatic annotation and hidden Markov model integration.
result Improvement of about 7 percentage points in entity-level F1 scores. This research improves uncertainty estimation for medical predictions, enhancing model trust and decision support.
problem Improving model uncertainty estimation for rare medical conditions.
method Developed and refined heuristics for selecting uncertainty estimation techniques, distinguishing them by clinical use-case. Also, compared ensembles vs. auto-encoders for detecting out-of-domain examples.
result Auto-encoders outperform ensembles in detecting out-of-domain examples, highlighting their importance for medical tabular data.
CMLE reduces spurious correlations in deep models.
problem Spurious correlations in deep learning models.
method Counterfactual Maximum Likelihood Estimation (CMLE) on interventional distribution.
result CMLE outperforms regular MLE in out-of-domain generalization and spurious correlation reduction.
Learnable token perturbations boost extrapolation in LLMs.
problem Limited flexibility of current discrete perturbations in large language models.
method Learnable continuous latent vector transformations in embedding space, unbiased estimating equations, stochastic gradient descent optimization.
result Significant gains in out-of-domain settings over state-of-the-art methods.
Deep learning has the potential to dramatically impact navigation and tracking state estimation problems critical to autonomous vehicles and robotics. Measurement uncertainties in state estimation systems based on Kalman and other Bayes filters are typically assumed to be a fixed covariance matrix. This assumption is r…
We propose a novel method for closed-form predictive distribution modeling with neural nets. In quantifying prediction uncertainty, we build on Evidential Deep Learning, which has been impactful as being both simple to implement and giving closed-form access to predictive uncertainty. We employ it to model aleatoric un…
The recent surge of text-based online counseling applications enables us to collect and analyze interactions between counselors and clients. A dataset of those interactions can be used to learn to automatically classify the client utterances into categories that help counselors in diagnosing client status and predictin…
New algorithm PRACTISE accelerates networks with tiny sets, reducing latency by 22%.
problem Accelerating networks with tiny training sets to improve performance and privacy.
method Proposes a new algorithm PRACTISE based on finetune convexity hypothesis and feature mimicking.
result PRACTISE outperforms previous methods by reducing latency by 22% on ImageNet-1k.
Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
The paper tackles spurious correlations in machine learning models and introduces counterfactual invariance.
problem Spurious correlations in machine learning models that depend on irrelevant parts of input data.
method The paper uses causal inference to stress test models and introduces counterfactual invariance as a formal requirement.
result Counterfactual invariance is a requirement for models to be robust to irrelevant perturbations in input data.
The paper proposes deep normalization to improve speaker recognition performance.
problem Non-Gaussian and non-homogeneous distributions of deep speaker vectors negatively impact speaker recognition.
method Proposes a deep normalization approach based on a novel discriminative normalization flow (DNF) model.
result DNF-based normalization delivers substantial performance gains and strong generalization capability.
The rising growth of fake news and misleading information through online media outlets demands an automatic method for detecting such news articles. Of the few limited works which differentiate between trusted vs other types of news article (satire, propaganda, hoax), none of them model sentence interactions within a d…
Study on neural scaling laws for solving linear systems in-context.
problem Theoretical guarantees for solving linear systems using a linear transformer architecture.
method Neural scaling laws and task diversity for in-domain and out-of-domain generalization.
result Novel notion of task diversity for necessary and sufficient condition of generalization under task shifts.