New framework infers causal shifts in event sequences under out-of-domain interventions.
problem Inferring causal relationships in event sequences without considering out-of-domain interventions.
method Proposes a new causal framework to define ATE, designs an unbiased ATE estimator, and uses a Transformer-based neural network model.
result Demonstrates superior performance in ATE estimation and goodness-of-fit under out-of-domain-augmented point processes.
Researchers use human-in-the-loop to create counterfactually augmented data, improving model performance.
problem Creating ML models less reliant on spurious patterns in NLP datasets.
method A human-in-the-loop process to curate counterfactually augmented data (CAD), prohibiting unnecessary edits.
result Models trained on CAD appear to rely less on semantically irrelevant words and generalize better out of domain.
Valid certifies LLMs' domain adherence, bounding out-of-domain behavior.
problem Adversarial susceptibility of LLMs to generate out-of-domain outputs.
method VALID approach providing adversarial bounds as a certificate.
result Validates LLMs' domain adherence with meaningful certificates.
Enhances adversarial robustness with unlabeled out-of-domain data.
problem Improving robustness of models against adversarial attacks.
method Leveraging unlabeled data from multiple domains to bridge the sample complexity gap in adversarial robustness.
result Better adversarial robustness achieved when unlabeled data comes from a shifted domain.
Framework uses unlabeled out-of-domain data to improve semi-supervised classification.
problem Improving generalization in semi-supervised classification problems.
method Combines Distributionally Robust Optimization (DRO) with self-supervised training.
result Significant improvement in generalization error compared to ERM.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
Proposes a measure to predict generalization in non-matching environments.
problem Characterizing and comparing generalization of machine learning models in non-matching environments.
method Neighborhood invariance measure, calculating invariance as the largest fraction of transformed points classified into the same class.
result Strong and robust correlation between neighborhood invariance and actual out-of-domain generalization.
Enhances out-of-domain calibration of neural networks.
problem Improving calibration performance of deep neural networks in out-of-domain settings.
method Consistency-guided temperature scaling (CTS) that considers style and content consistency.
result Significantly enhances out-of-domain calibration performance.
Out-of-domain (OOD) detection for low-resource text classification is a realistic but understudied task. The goal is to detect the OOD cases with limited in-domain (ID) training data, since we observe that training data is often insufficient in machine learning applications. In this work, we propose an OOD-resistant Pr…
Transformers can learn new tasks from diverse pretraining data but struggle with out-of-domain tasks.
problem Transformer models' ability to learn new tasks in-context is limited by their pretraining data coverage.
method Investigation of transformer models trained on (x,f(x)) pairs, comparing in-context learning capabilities across different task families. result Transformers can identify and learn within task families in their pretraining data but fail with out-of-domain tasks.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
fAux tests individual fairness without domain knowledge or out-of-domain samples.
problem Testing for individual fairness in machine learning models.
method Derivative comparison between model predictions and auxiliary model predictions.
result Effectively identifies discrimination on synthetic and real-world datasets.
This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.
problem Generalization from easy to hard tasks, especially out-of-domain generalization.
method Formalizes inductive generalization, introduces Inductive Learning, and outlines steps to adapt techniques for learning model successors.
result A new learning paradigm (Inductive Learning) that emphasizes induction and universal properties of learning and computation.
Method combines LD and Fermat Distance for neural network uncertainty.
problem Measuring uncertainty in neural network predictions.
method Statistical Depth (LD) combined with Fermat Distance.
result Effective uncertainty estimation without impacting original model performance.
AFTER technique improves NLP models by preventing overfitting to task-specific domains.
problem Standard fine-tuning degrades pretraining domain representations.
method Complements task-specific loss with adversarial objective.
result AFTER leads to improved performance on various NLP tasks.
The performance of a Part-of-speech (POS) tagger is highly dependent on the domain ofthe processed text, and for many domains there is no or only very little training data available. This work addresses the problem of POS tagging noisy user-generated text using a neural network. We propose an architecture that trains a…
The paper tackles extrapolation in extreme regions of regression problems.
problem Extrapolation on the tails of covariates in continuous regression problems.
method Statistical regression on a subsample of furthest observations, focusing on their angular components, using multivariate regular variation theory.
result Quantifies predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a bias-variance decomposition.
The Intensive Care Unit (ICU) is a hospital department where machine learning has the potential to provide valuable assistance in clinical decision making. Classical machine learning models usually only provide point-estimates and no uncertainty of predictions. In practice, uncertain predictions should be presented to …
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify aco…
This paper evaluates scalable uncertainty estimation methods for DNN-based molecular property prediction.
problem Uncertainty quantification in DNN models for molecular property prediction.
method Quantitative comparison of MC-Dropout, deep ensembles, and bootstrapping on the QM9 dataset.
result Ensembling and bootstrapping consistently outperform MC-Dropout, with different context-specific pros and cons.
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The u…
Study on neural scaling laws for solving linear systems in-context.
problem Theoretical guarantees for solving linear systems using a linear transformer architecture.
method Neural scaling laws and task diversity for in-domain and out-of-domain generalization.
result Novel notion of task diversity for necessary and sufficient condition of generalization under task shifts.
TESTED improves multi-domain stance detection with topic-guided sampling and contrastive learning.
problem Challenges in multi-domain stance detection due to domain-specific variations and imbalanced annotations.
method Topic-guided diversity sampling and contrastive learning objective.
result Significant improvement in F1 scores, up to 10.2 points out-of-domain.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
ConvCNP models translation equivariance in data.
problem Translation equivariance in data.
method Convolutional Conditional Neural Processes (ConvCNP) that embeds data into an infinite-dimensional function space.
result ConvCNP achieves state-of-the-art performance and zero-shot generalization.
The recent surge of text-based online counseling applications enables us to collect and analyze interactions between counselors and clients. A dataset of those interactions can be used to learn to automatically classify the client utterances into categories that help counselors in diagnosing client status and predictin…
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
The paper proposes deep normalization to improve speaker recognition performance.
problem Non-Gaussian and non-homogeneous distributions of deep speaker vectors negatively impact speaker recognition.
method Proposes a deep normalization approach based on a novel discriminative normalization flow (DNF) model.
result DNF-based normalization delivers substantial performance gains and strong generalization capability.
The paper improves model robustness by regularizing posterior differences.
problem Improving model robustness in noisy input scenarios.
method Posterior differential regularization with f-divergence. result Regularizing with f-divergence improves model robustness. Periodic activation functions improve neural network reliability and interpretability.
problem Neural networks reinforce hidden biases, making them unreliable and hard to interpret.
method Introduce periodic activation functions in Bayesian neural networks to establish a connection with stationary Gaussian process priors.
result Periodic activation functions, including sinusoidal, triangular, and ReLU, improve model performance and sensitivity to perturbations.
UBMF tackles fault diagnosis in imbalanced industrial data with enhanced accuracy and adaptability.
problem Fault diagnosis challenges in imbalanced industrial data.
method Integrates four key modules: data perturbation, cross-task feature extraction, uncertainty-based filtering, and Bayesian meta-knowledge integration.
result Achieves an average improvement of 42.22% across ten diagnostic tasks.
We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mi…
Learnable token perturbations boost extrapolation in LLMs.
problem Limited flexibility of current discrete perturbations in large language models.
method Learnable continuous latent vector transformations in embedding space, unbiased estimating equations, stochastic gradient descent optimization.
result Significant gains in out-of-domain settings over state-of-the-art methods.
CMLE reduces spurious correlations in deep models.
problem Spurious correlations in deep learning models.
method Counterfactual Maximum Likelihood Estimation (CMLE) on interventional distribution.
result CMLE outperforms regular MLE in out-of-domain generalization and spurious correlation reduction.
New method improves domain generalization by matching object representations.
problem Existing domain generalization methods fail to generalize to unseen domains.
method Proposes matching-based algorithms to match object representations across domains.
result MatchDG algorithm matches ground-truth object representations and improves out-of-domain accuracy.
We propose a novel method for closed-form predictive distribution modeling with neural nets. In quantifying prediction uncertainty, we build on Evidential Deep Learning, which has been impactful as being both simple to implement and giving closed-form access to predictive uncertainty. We employ it to model aleatoric un…
For many years, i-vector based audio embedding techniques were the dominant approach for speaker verification and speaker diarization applications. However, mirroring the rise of deep learning in various domains, neural network based audio embeddings, also known as d-vectors, have consistently demonstrated superior spe…
Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to m…
The paper presents a framework to quantify the trade-off between synthetic and real data.
problem Improving generalization with synthetic data when real data is scarce.
method Learning-theoretic framework leveraging algorithmic stability to derive generalization error bounds.
result Optimal synthetic-to-real data ratio minimizing expected test error as a function of Wasserstein distance.
Study proposes worst+gap measure for better DG evaluation.
problem Lack of comprehensive exploration of average measure in DG evaluation.
method Introduced worst+gap measure and compared it with average measure.
result Worst+gap measure provides a more accurate approximation of true DG performance.
Causal predictors don't generalize better across domains than non-causal predictors.
problem How well do causal predictors generalize across different domains?
method 16 prediction tasks on tabular datasets, selecting causal features.
result Causal predictors do not outperform non-causal predictors in domain generalization.
NFM improves deep learning by selectively processing hidden states.
problem Processing entire hidden states in each layer limits modularity and reusability.
method Introduces Neural Function Modules (NFM) with attention, sparsity, and feedback.
result Improves results in classification, generalization, generative modeling, and reinforcement learning.
Score-based generative models achieve state-of-the-art classification accuracy on CIFAR-10.
problem Improving classification accuracy on natural image datasets.
method Score-based generative models applied to CIFAR-10.
result Score-based models achieve state-of-the-art classification accuracy on CIFAR-10.
OT domain adaptation improves aphasia detection across languages.
problem Detecting aphasia in low-resource languages with limited data.
method Utilized OT domain adaptation to map linguistic features across multiple languages.
result OT domain adaptation significantly improved F1 scores for French and Mandarin aphasia detection.
NP-PROV separates mean and variance spaces to improve function uncertainty.
problem Neural Processes fail on out-of-domain tasks due to shared latent space uncertainty.
method Separates mean and variance into function-value-related and position-related latent spaces.
result NP-PROV achieves state-of-the-art likelihood with bounded variance in drifts.
Graph model detects fake news by analyzing sentence interactions.
problem Detecting fake news through online media.
method Graph Neural Network-based model for sentence interactions.
result Our model achieves state-of-the-art accuracy on fake news datasets.