RAD improves robustness to domain annotation noise without explicit domain annotations.
problem Robustness to domain annotation noise in training data.
method Regularized Annotation of Domains (RAD) for last layer retraining.
result RAD outperforms state-of-the-art methods even with 5% noise in training data.
Framework tackles class imbalance and noisy labels in active learning.
problem Class imbalance and noisy labels in real-world datasets.
method Uses foundation model priors to select informative samples for active learning.
result Substantial annotation savings (over 50%) with preserved performance and robustness.
Proposes a new method to improve target annotation in ATR.
problem Challenges in annotating automatic target recognition due to lack of labeled data.
method Hybrid contrastive learning and cycle-consistency-based transductive transfer learning (C3TTL) framework.
result Significantly lower Fréchet Inception Distance (FID) score and improved performance in annotating civilian and military vehicles, as well as ship targets.
Paper tackles medical image diagnosis with unsupervised domain adaptation.
problem Limited labeled samples and label noise in medical images.
method Collaborative Unsupervised Domain Adaptation (UDA) algorithm.
result Empirical results show superiority of the proposed method.
We consider the problem of Probably Approximate Correct (PAC) learning of a binary classifier from noisy labeled examples acquired from multiple annotators (each characterized by a respective classification noise rate). First, we consider the complete information scenario, where the learner knows the noise rates of all…
Manually labeled corpora are expensive to create and often not available for low-resource languages or domains. Automatic labeling approaches are an alternative way to obtain labeled data in a quicker and cheaper way. However, these labels often contain more errors which can deteriorate a classifier's performance when …
Bayesian method improves deep learning for noisy EEG seizure detection.
problem Label noise in scalp EEG data hinders deep learning performance.
method Integrates domain knowledge into a Bayesian framework to inform deep learning models of label ambiguities.
result BUNDL enhances robustness of seizure detection systems under noisy label conditions.
An automated metric to evaluate dialogue quality is vital for optimizing data driven dialogue management. The common approach of relying on explicit user feedback during a conversation is intrusive and sparse. Current models to estimate user satisfaction use limited feature sets and rely on annotation schemes with low …
spex-LVM infers interpretable latent factors from biomedical data.
problem Inability to learn sparse and interpretable hidden states.
method Factorial latent variable model with sparse priors and domain-relevant annotations.
result Robustly identifies relevant structure in RNA-seq datasets.
Study on noise models for noisy labels in NLP.
problem Quality of noise models from noisy labels.
method Theoretical analysis and synthetic dataset creation.
result Expected error of noise models derived.
Domain shift is unavoidable in real-world applications of object detection. For example, in self-driving cars, the target domain consists of unconstrained road environments which cannot all possibly be observed in training data. Similarly, in surveillance applications sufficiently representative training data may be la…
Study real-world noisy labels from human annotations for better understanding.
problem Understanding and modeling real-world label noise in machine learning.
method Developed two new benchmark datasets (CIFAR-10N, CIFAR-100N) with human-annotated real-world noisy labels.
result Real-world noisy labels exhibit instance-dependent patterns, not class-dependent as previously assumed.
Few-shot cell segmentation from diverse sources to target domain.
problem Efficient cell segmentation from limited annotated images.
method Meta-learning combining cross-domain tasks and invariant representation.
result Promising results from 1-10-shot learning on public databases.
New method pools labels from similar data items to improve learning from small samples.
problem Learning from small, human-annotated samples with potential disagreement among annotators.
method Proposes neighborhood-based pooling for sharing labels across similar data items.
result Improves learning from small, noisy samples by pooling labels from similar items.
ADS filters data points for efficient batch active learning.
problem Efficiently selecting data points for annotation in parallel settings.
method Active Data Shapley (ADS) using the Shapley value of data.
result Significantly increases efficiency of active learning by 6x.
CPATTA uses conformal prediction for efficient test-time adaptation.
problem Low data selection efficiency in existing ATTA methods.
method Conformal Prediction, online weight-update algorithm, domain-shift detector, staged update scheme.
result CPATTA consistently outperforms state-of-the-art methods by 5% in accuracy.
Paper presents a method to train NER models without labelled data using weak supervision.
problem Dealing with NER performance drop in new domains without labelled data.
method Weak supervision through automatic annotation and hidden Markov model integration.
result Improvement of about 7 percentage points in entity-level F1 scores. Majority Vote is optimal for reliable data labeling under certain conditions.
problem Reliable data labeling requires aggregating multiple annotators' labels, but the optimality of Majority Vote is not well understood.
method Characterized conditions under which Majority Vote achieves the optimal label estimation error.
result Majority Vote optimally recovers labels for a given class distribution under tolerable annotation noise limits.
Automated dialogue quality evaluation using user satisfaction estimates across multiple domains.
problem Lack of automated and domain-independent dialogue quality evaluation metrics.
method Created a new Response Quality annotation scheme, introduced five domain-independent feature sets, and experimented with six machine learning models.
result Gradient Boosting Regression model achieved best prediction performance, with a 16% relative improvement in binary satisfaction class prediction accuracy.
Unsupervised learning classifies transient noise in gravitational wave detectors.
problem Transient noise interferes with gravitational wave signals, causing instability.
method Combines variational autoencoder and invariant information clustering.
result Consistent classification with Gravity Spy project labels.
Paper tackles instance-dependent label noise by approximating it with part-dependent noise.
problem Learning with instance-dependent label noise is challenging.
method Approximate instance-dependent label noise with part-dependent noise. Use transition matrices for parts to model noise.
result Method outperforms state-of-the-art approaches for instance-dependent label noise.
We describe a method for selecting relevant new training data for the LSTM-based domain selection component of our personal assistant system. Adding more annotated training data for any ML system typically improves accuracy, but only if it provides examples not already adequately covered in the existing data. However, …
New method uses image-level and pixel-level annotations for brain tumor segmentation.
problem Challenges in obtaining pixel-level annotations for brain tumor segmentation.
method Proposes a learning-based framework that combines both pixel- and image-level annotations.
result Method's performance in segmentation quality is competitive with traditional fully-supervised approach.
This study evaluates zero-shot LLMs in finance, finding ChatGPT performs well but fine-tuned models are better.
problem Evaluating zero-shot LLMs in financial tasks.
method Comparison of ChatGPT and fine-tuned models on annotated data.
result Fine-tuned models generally outperform zero-shot LLMs.
Integrates multiple datasets to solve open set crowdsourcing problems.
problem Crowdsourcing with unknown label space and unfamiliar tasks.
method Integrates multiple crowdsourced datasets, weights them based on category correlation, and uses open set transfer learning.
result Proves OSCrowd solves open set crowdsourcing problems and outperforms related solutions.
This work improves medical image segmentation with limited annotations using contrastive learning.
problem Lack of labeled data for medical image segmentation.
method Contrastive learning framework for semi-supervised segmentation with domain-specific and problem-specific cues.
result Significant improvements in segmentation performance compared to other methods.
Meta-learning method for accurate classifier from noisy annotators' data.
problem Accurate learning from noisy labels provided by multiple annotators.
method Meta-learning neural network to embed examples in latent space and estimate annotators' abilities, then adapt classifiers using EM algorithm.
result Meta-learning method improves classifier performance with minimal labeled data.
Dynamic residual adapters improve performance across multiple latent domains without domain labels.
problem Overfitting to large domains and ignoring smaller ones in multi-domain learning.
method Dynamic residual adapters and augmentation strategies inspired by style transfer.
result Dynamic residual adapters significantly outperform standard models on multiple latent domains.
Paper tackles active learning under human label variation, proposing a new framework.
problem Active learning assumes a single ground truth, ignoring human label variation.
method Survey and propose a conceptual framework for HLV-aware active learning.
result Lay a conceptual foundation for HLV-aware active learning.
Deep learning for supervised learning has achieved astonishing performance in various machine learning applications. However, annotated data is expensive and rare. In practice, only a small portion of data samples are annotated. Pseudo-ensembling-based approaches have achieved state-of-the-art results in computer visio…
Active learning improves inspection systems by using weakly labeled data.
problem Rapidly updating machine vision inspection systems in evolving manufacturing processes.
method Developed a methodology for active learning from weakly labeled data, addressing covariate shift with domain-adversarial training.
result Demonstrated that active learning can accelerate the annotation process and reduce false positives.
In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for wh…
Face recognition system trained with noisy labels.
problem Label noise in training deep learning classifiers.
method Review and apply recent methods to manage noisy annotations.
result Improved performance of face recognition system with noisy labels.
The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground trut…
We propose a general-purpose approach to discovering active learning (AL) strategies from data. These strategies are transferable from one domain to another and can be used in conjunction with many machine learning models. To this end, we formalize the annotation process as a Markov decision process, design universal s…
CORES2 removes noisy labels by sieving out corrupted examples.
problem Instance-dependent label noise degrades DNN performance.
method CORES2 (COnfidence REgularized Sample Sieve) progressively sieves out corrupted examples.
result CORES2 provides theoretical guarantees for filtering out corrupted examples.
Discriminative active learning reduces data annotation costs for domain adaptation.
problem Conditional shift problem hinders domain adaptation between related but different domains.
method Three-stage active adversarial training: invariant feature space learning, uncertainty and diversity criteria, re-training with queried labels.
result Empirical comparisons show the proposed approach is more effective than existing methods.
Proposes a new loss function for robust training of deep neural networks against noisy labels.
problem Training deep neural networks with noisy labels, especially instance-independent noise.
method Introduces a novel information-theoretic loss function, L_DMI, based on Determinant based Mutual Information (DMI).
result L_DMI is the first provably robust loss function to instance-independent label noise, without requiring auxiliary information.
Paper proposes training deep nets on noisy labels without manual annotation.
problem Training deep neural networks on noisy labels without manual annotation.
method Directly train deep neural network on noisy candidate labels, early stopping to avoid overfitting.
result Training on noisy candidate labels yields higher test performance than on clean data.
Aims to eliminate domain bias in authentication without domain labels.
problem Authentication models are biased due to domain differences.
method Discover latent domains and eliminate domain difference alternately, using a meta-learning framework.
result Eliminates domain difference in authentication without domain labels.
Early recognition of abnormal rhythms in ECG signals is crucial for monitoring and diagnosing patients' cardiac conditions, increasing the success rate of the treatment. Classifying abnormal rhythms into exact categories is very challenging due to the broad taxonomy of rhythms, noises and lack of large-scale real-world…
This paper refines human labeling as a measurement process, revealing four sources of variation.
problem Systematic variation in human labeling obscures model learning.
method Introduces a statistical framework to decompose labeling outcomes.
result Empirical evidence for four components of labeling variation.
In the absence of sufficient data variation (e.g., scanner and protocol variability) in annotated data, deep neural networks (DNNs) tend to overfit during training. As a result, their performance is significantly lower on data from unseen sources compared to the performance on data from the same source as the training …
Sentiment analysis is a highly subjective and challenging task. Its complexity further increases when applied to the Arabic language, mainly because of the large variety of dialects that are unstandardized and widely used in the Web, especially in social media. While many datasets have been released to train sentiment …
We propose a method for unsupervised domain adaptation that trains a shared embedding to align the joint distributions of inputs (domain) and outputs (classes), making any classifier agnostic to the domain. Joint alignment ensures that not only the marginal distributions of the domain are aligned, but the labels as wel…
The growing need to analyze large collections of documents has led to great developments in topic modeling. Since documents are frequently associated with other related variables, such as labels or ratings, much interest has been placed on supervised topic models. However, the nature of most annotation tasks, prone to …
Supervised Deep Learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real world problems, manual annotation is …
When learning a hidden Markov model (HMM), sequen- tial observations can often be complemented by real-valued summary response variables generated from the path of hid- den states. Such settings arise in numerous domains, includ- ing many applications in biology, like motif discovery and genome annotation. In this pape…