Out-of-domain (OOD) detection for low-resource text classification is a realistic but understudied task. The goal is to detect the OOD cases with limited in-domain (ID) training data, since we observe that training data is often insufficient in machine learning applications. In this work, we propose an OOD-resistant Pr…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Out-of-domain tasks”
Valid certifies LLMs' domain adherence, bounding out-of-domain behavior.
problem Adversarial susceptibility of LLMs to generate out-of-domain outputs.
method VALID approach providing adversarial bounds as a certificate.
result Validates LLMs' domain adherence with meaningful certificates.
Transformers can learn new tasks from diverse pretraining data but struggle with out-of-domain tasks.
problem Transformer models' ability to learn new tasks in-context is limited by their pretraining data coverage.
method Investigation of transformer models trained on pairs, comparing in-context learning capabilities across different task families.
result Transformers can identify and learn within task families in their pretraining data but fail with out-of-domain tasks.
SSMBA generates synthetic data to improve robustness in natural language tasks.
problem Improving out-of-domain generalization of models trained on natural language data.
method SSMBA uses corruption and reconstruction functions to generate synthetic data points on the manifold assumption.
result SSMBA consistently outperforms existing methods on robustness benchmarks across multiple tasks and datasets.
New framework infers causal shifts in event sequences under out-of-domain interventions.
problem Inferring causal relationships in event sequences without considering out-of-domain interventions.
method Proposes a new causal framework to define ATE, designs an unbiased ATE estimator, and uses a Transformer-based neural network model.
result Demonstrates superior performance in ATE estimation and goodness-of-fit under out-of-domain-augmented point processes.
AFTER technique improves NLP models by preventing overfitting to task-specific domains.
problem Standard fine-tuning degrades pretraining domain representations.
method Complements task-specific loss with adversarial objective.
result AFTER leads to improved performance on various NLP tasks.
Enhances adversarial robustness with unlabeled out-of-domain data.
problem Improving robustness of models against adversarial attacks.
method Leveraging unlabeled data from multiple domains to bridge the sample complexity gap in adversarial robustness.
result Better adversarial robustness achieved when unlabeled data comes from a shifted domain.
Researchers use human-in-the-loop to create counterfactually augmented data, improving model performance.
problem Creating ML models less reliant on spurious patterns in NLP datasets.
method A human-in-the-loop process to curate counterfactually augmented data (CAD), prohibiting unnecessary edits.
result Models trained on CAD appear to rely less on semantically irrelevant words and generalize better out of domain.
The performance of a Part-of-speech (POS) tagger is highly dependent on the domain ofthe processed text, and for many domains there is no or only very little training data available. This work addresses the problem of POS tagging noisy user-generated text using a neural network. We propose an architecture that trains a…
Learned feature representations and sub-phoneme posteriors from Deep Neural Networks (DNNs) have been used separately to produce significant performance gains for speaker and language recognition tasks. In this work we show how these gains are possible using a single DNN for both speaker and language recognition. The u…
Framework uses unlabeled out-of-domain data to improve semi-supervised classification.
problem Improving generalization in semi-supervised classification problems.
method Combines Distributionally Robust Optimization (DRO) with self-supervised training.
result Significant improvement in generalization error compared to ERM.
Enhances out-of-domain calibration of neural networks.
problem Improving calibration performance of deep neural networks in out-of-domain settings.
method Consistency-guided temperature scaling (CTS) that considers style and content consistency.
result Significantly enhances out-of-domain calibration performance.
TESTED improves multi-domain stance detection with topic-guided sampling and contrastive learning.
problem Challenges in multi-domain stance detection due to domain-specific variations and imbalanced annotations.
method Topic-guided diversity sampling and contrastive learning objective.
result Significant improvement in F1 scores, up to 10.2 points out-of-domain.
This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.
problem Generalization from easy to hard tasks, especially out-of-domain generalization.
method Formalizes inductive generalization, introduces Inductive Learning, and outlines steps to adapt techniques for learning model successors.
result A new learning paradigm (Inductive Learning) that emphasizes induction and universal properties of learning and computation.
In-Context Learning of Linear Systems: Generalization Theory and Applications to Operator Learningcs.LG
Study on neural scaling laws for solving linear systems in-context.
problem Theoretical guarantees for solving linear systems using a linear transformer architecture.
method Neural scaling laws and task diversity for in-domain and out-of-domain generalization.
result Novel notion of task diversity for necessary and sufficient condition of generalization under task shifts.
Gradient boosting can be seen as Gaussian process inference.
problem Improving uncertainty estimates in out-of-domain detection.
method Gradient boosting reformulated as a kernel method converging to Gaussian process inference.
result Gradient boosting can provide better uncertainty estimates through Monte-Carlo estimation of posterior variance.
Proposes a measure to predict generalization in non-matching environments.
problem Characterizing and comparing generalization of machine learning models in non-matching environments.
method Neighborhood invariance measure, calculating invariance as the largest fraction of transformed points classified into the same class.
result Strong and robust correlation between neighborhood invariance and actual out-of-domain generalization.
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
Method combines LD and Fermat Distance for neural network uncertainty.
problem Measuring uncertainty in neural network predictions.
method Statistical Depth (LD) combined with Fermat Distance.
result Effective uncertainty estimation without impacting original model performance.
UBMF tackles fault diagnosis in imbalanced industrial data with enhanced accuracy and adaptability.
problem Fault diagnosis challenges in imbalanced industrial data.
method Integrates four key modules: data perturbation, cross-task feature extraction, uncertainty-based filtering, and Bayesian meta-knowledge integration.
result Achieves an average improvement of 42.22% across ten diagnostic tasks.
This research investigates selectively pruning hyper and hypo neurons to improve neural network generalization.
problem Improving neural network generalization to unseen data.
method Investigates pruning hyper and hypo neurons selectively in fully connected layers of CNNs.
result Selective pruning of hyper and hypo neurons improves model performance on out-of-domain data.
fAux tests individual fairness without domain knowledge or out-of-domain samples.
problem Testing for individual fairness in machine learning models.
method Derivative comparison between model predictions and auxiliary model predictions.
result Effectively identifies discrimination on synthetic and real-world datasets.
Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applicationscs.LG
The Intensive Care Unit (ICU) is a hospital department where machine learning has the potential to provide valuable assistance in clinical decision making. Classical machine learning models usually only provide point-estimates and no uncertainty of predictions. In practice, uncertain predictions should be presented to …
ConvCNP models translation equivariance in data.
problem Translation equivariance in data.
method Convolutional Conditional Neural Processes (ConvCNP) that embeds data into an infinite-dimensional function space.
result ConvCNP achieves state-of-the-art performance and zero-shot generalization.
GCC pre-trains graph neural networks to transfer across diverse datasets.
problem Non-transferable graph models trained for specific datasets.
method Graph Contrastive Coding (GCC) framework using contrastive learning.
result GCC pre-trained models achieve competitive performance across multiple datasets.
This paper evaluates scalable uncertainty estimation methods for DNN-based molecular property prediction.
problem Uncertainty quantification in DNN models for molecular property prediction.
method Quantitative comparison of MC-Dropout, deep ensembles, and bootstrapping on the QM9 dataset.
result Ensembling and bootstrapping consistently outperform MC-Dropout, with different context-specific pros and cons.
The paper proposes deep normalization to improve speaker recognition performance.
problem Non-Gaussian and non-homogeneous distributions of deep speaker vectors negatively impact speaker recognition.
method Proposes a deep normalization approach based on a novel discriminative normalization flow (DNF) model.
result DNF-based normalization delivers substantial performance gains and strong generalization capability.
We compare features for dynamic time warping (DTW) when used to bootstrap keyword spotting (KWS) in an almost zero-resource setting. Such quickly-deployable systems aim to support United Nations (UN) humanitarian relief efforts in parts of Africa with severely under-resourced languages. Our objective is to identify aco…
The paper tackles extrapolation in extreme regions of regression problems.
problem Extrapolation on the tails of covariates in continuous regression problems.
method Statistical regression on a subsample of furthest observations, focusing on their angular components, using multivariate regular variation theory.
result Quantifies predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a bias-variance decomposition.
CMLE reduces spurious correlations in deep models.
problem Spurious correlations in deep learning models.
method Counterfactual Maximum Likelihood Estimation (CMLE) on interventional distribution.
result CMLE outperforms regular MLE in out-of-domain generalization and spurious correlation reduction.
Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
NP-PROV separates mean and variance spaces to improve function uncertainty.
problem Neural Processes fail on out-of-domain tasks due to shared latent space uncertainty.
method Separates mean and variance into function-value-related and position-related latent spaces.
result NP-PROV achieves state-of-the-art likelihood with bounded variance in drifts.
A new metric FRD improves comparing medical images.
problem Comparing medical images for distribution or domain differences.
method Developed a new metric FRD using standardized radiomic features.
result FRD outperforms other metrics in various medical imaging applications.
ORCA calibrates LLMs for efficient, generalizable reasoning.
problem Miscalibration of large language models leading to inefficiencies.
method Online Reasoning Calibration (ORCA) using conformal prediction and test-time training.
result ORCA provides higher efficiency and generalization across different reasoning tasks.
Causal predictors don't generalize better across domains than non-causal predictors.
problem How well do causal predictors generalize across different domains?
method 16 prediction tasks on tabular datasets, selecting causal features.
result Causal predictors do not outperform non-causal predictors in domain generalization.
Develops possibilistic VI using maxitive Donsker-Varadhan formulation.
problem Adapting VI to possibilistic framework to handle uncertainty.
method Maxitive Donsker-Varadhan formulation for possibilistic VI.
result CBOpt optimizers achieve competitive performance in image classification.
The paper improves model robustness by regularizing posterior differences.
problem Improving model robustness in noisy input scenarios.
method Posterior differential regularization with -divergence.
result Regularizing with -divergence improves model robustness.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
NFM improves deep learning by selectively processing hidden states.
problem Processing entire hidden states in each layer limits modularity and reusability.
method Introduces Neural Function Modules (NFM) with attention, sparsity, and feedback.
result Improves results in classification, generalization, generative modeling, and reinforcement learning.
This research improves uncertainty estimation for medical predictions, enhancing model trust and decision support.
problem Improving model uncertainty estimation for rare medical conditions.
method Developed and refined heuristics for selecting uncertainty estimation techniques, distinguishing them by clinical use-case. Also, compared ensembles vs. auto-encoders for detecting out-of-domain examples.
result Auto-encoders outperform ensembles in detecting out-of-domain examples, highlighting their importance for medical tabular data.
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscs.LG
SNGP improves DNNs' uncertainty estimation with minimal changes.
problem Uncertainty estimation in deep learning models for real-time applications.
method Formalizing uncertainty as a minimax problem, SNGP adds weight normalization and replaces the output layer with a Gaussian process.
result SNGP outperforms other single-model approaches in uncertainty estimation across vision and language tasks.
Periodic activation functions improve neural network reliability and interpretability.
problem Neural networks reinforce hidden biases, making them unreliable and hard to interpret.
method Introduce periodic activation functions in Bayesian neural networks to establish a connection with stationary Gaussian process priors.
result Periodic activation functions, including sinusoidal, triangular, and ReLU, improve model performance and sensitivity to perturbations.
Score-based generative models achieve state-of-the-art classification accuracy on CIFAR-10.
problem Improving classification accuracy on natural image datasets.
method Score-based generative models applied to CIFAR-10.
result Score-based models achieve state-of-the-art classification accuracy on CIFAR-10.
We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mi…
Learnable token perturbations boost extrapolation in LLMs.
problem Limited flexibility of current discrete perturbations in large language models.
method Learnable continuous latent vector transformations in embedding space, unbiased estimating equations, stochastic gradient descent optimization.
result Significant gains in out-of-domain settings over state-of-the-art methods.
Self-supervised learning performs better than supervised learning on imbalanced datasets.
problem The performance gap between balanced and imbalanced pre-training with self-supervised learning is smaller than with supervised learning.
method Systematic investigation of self-supervised learning under dataset imbalance, including experiments and theoretical analyses.
result Self-supervised representations are more robust to class imbalance than supervised representations.
WebGUM learns web navigation from multimodal data, outperforming previous methods.
problem Limited generalization from domain-specific models in web navigation.
method Instruction-following multimodal agent trained on vision-language foundation models.
result Significant improvement in web navigation performance on benchmarks.