ECN framework improves training on noisy structured labels.
problem Structured errors in fine-grained annotations lead to biased models.
method Error-Correcting Networks (ECN) framework.
result ECN improves fine-grained annotation prediction.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Noisy Annotations”
The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the "truth" under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground trut…
Meta-learning method for accurate classifier from noisy annotators' data.
problem Accurate learning from noisy labels provided by multiple annotators.
method Meta-learning neural network to embed examples in latent space and estimate annotators' abilities, then adapt classifiers using EM algorithm.
result Meta-learning method improves classifier performance with minimal labeled data.
One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations a…
Framework tackles class imbalance and noisy labels in active learning.
problem Class imbalance and noisy labels in real-world datasets.
method Uses foundation model priors to select informative samples for active learning.
result Substantial annotation savings (over 50%) with preserved performance and robustness.
In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for wh…
We consider the problem of Probably Approximate Correct (PAC) learning of a binary classifier from noisy labeled examples acquired from multiple annotators (each characterized by a respective classification noise rate). First, we consider the complete information scenario, where the learner knows the noise rates of all…
A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…
Study real-world noisy labels from human annotations for better understanding.
problem Understanding and modeling real-world label noise in machine learning.
method Developed two new benchmark datasets (CIFAR-10N, CIFAR-100N) with human-annotated real-world noisy labels.
result Real-world noisy labels exhibit instance-dependent patterns, not class-dependent as previously assumed.
Face recognition system trained with noisy labels.
problem Label noise in training deep learning classifiers.
method Review and apply recent methods to manage noisy annotations.
result Improved performance of face recognition system with noisy labels.
We present a new dataset for form understanding in noisy scanned documents (FUNSD) that aims at extracting and structuring the textual content of forms. The dataset comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging t…
TCR improves DNN robustness to noisy labels with minimal overhead.
problem Training on noisy labeled datasets degrades DNN generalization.
method TCR combines original labels and previous epoch predictions for regularization.
result TCR consistently enhances DNN robustness to label noise.
Paper proposes an alternative to anchor points for learning with noisy labels.
problem Learning with noisy labels is challenging due to inaccurate labels.
method Estimates transition matrix using clusterability condition and noisy labels.
result Estimation of transition matrix is more accurate and efficient than anchor points.
Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model trai…
While label fusion from multiple noisy annotations is a well understood concept in data wrangling (tackled for example by the Dawid-Skene (DS) model), we consider the extended problem of carrying out learning when the labels themselves are not consistently annotated with the same schema. We show that even if annotators…
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
problem Inaccurate and insufficient confusion matrices for annotators in multi-class classification tasks.
method PTBCC (ProtoType learning-driven Bayesian Classifier Combination) uses prototype confusion matrices to capture annotator expertise.
result PTBCC achieves up to 15% accuracy improvement and 3% higher average accuracy compared to existing methods.
Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collec…
Framework prevents deep learning models from memorizing noisy labels.
problem Deep learning models memorize noisy labels during early learning phase.
method Develops a technique that exploits early learning phase via regularization.
result Framework achieves robustness to noisy annotations on benchmarks and real-world datasets.
Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexacs.LG
This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expert…
This work improves deep learning from noisy crowdsourced labels.
problem Learning label correction and neural classifier from noisy crowdsourced data.
method Coupled Cross-Entropy Minimization (CCEM) with identifiability and regularization.
result The CCEM criterion correctly identifies annotators' confusion and neural classifier under realistic conditions.
INN: A Method Identifying Clean-annotated Samples via Consistency Effect in Deep Neural Networkscs.LG
INN method refines clean labeled data from noisy labels.
problem Handling noisy labels in deep neural networks.
method INN method based on memorization effect at neighbor regions.
result INN method resolves memorization effect shortcomings.
Deep learning from crowdsstat.ML
Over the last few years, deep learning has revolutionized the field of machine learning by dramatically improving the state-of-the-art in various domains. However, as the size of supervised artificial neural networks grows, typically so does the need for larger labeled datasets. Recently, crowdsourcing has established …
We study the problem of training an accurate linear regression model by procuring labels from multiple noisy crowd annotators, under a budget constraint. We propose a Bayesian model for linear regression in crowdsourcing and use variational inference for parameter estimation. To minimize the number of labels crowdsourc…
The Impact of Annotation Guidelines and Annotated Data on Extracting App Features from App Reviewscs.IR
Annotation guidelines used to guide the annotation of training and evaluation datasets can have a considerable impact on the quality of machine learning models. In this study, we explore the effects of annotation guidelines on the quality of app feature extraction models. As a main result, we propose several changes to…
Bayesian method improves deep learning for noisy EEG seizure detection.
problem Label noise in scalp EEG data hinders deep learning performance.
method Integrates domain knowledge into a Bayesian framework to inform deep learning models of label ambiguities.
result BUNDL enhances robustness of seizure detection systems under noisy label conditions.
Crowdsourcing has been proven to be an effective and efficient tool to annotate large datasets. User annotations are often noisy, so methods to combine the annotations to produce reliable estimates of the ground truth are necessary. We claim that considering the existence of clusters of users in this combination step c…
Bayesian method improves segmentation accuracy with noisy labels.
problem Annotation errors in semantic segmentation due to mislabeling and spatial correlations.
method Approximate Bayesian estimation with spatially correlated discrete distributions and variational inference.
result The method achieves performance comparable to clean labels under moderate noise levels.
A two-stage optimization framework reduces label noise in federated learning.
problem Label noise from noisy clients degrades federated learning model performance.
method MaskedOptim framework: detects noisy clients, corrects labels, and aggregates models robustly.
result Our framework improves model robustness and data quality in federated learning.
Manually labeled corpora are expensive to create and often not available for low-resource languages or domains. Automatic labeling approaches are an alternative way to obtain labeled data in a quicker and cheaper way. However, these labels often contain more errors which can deteriorate a classifier's performance when …
Crowdsourcing is a relatively economic and efficient solution to collect annotations from the crowd through online platforms. Answers collected from workers with different expertise may be noisy and unreliable, and the quality of annotated data needs to be further maintained. Various solutions have been attempted to ob…
Despite the success of deep neural networks (DNNs) in image classification tasks, the human-level performance relies on massive training data with high-quality manual annotations, which are expensive and time-consuming to collect. There exist many inexpensive data sources on the web, but they tend to contain inaccurate…
Interactive learning is a process in which a machine learning algorithm is provided with meaningful, well-chosen examples as opposed to randomly chosen examples typical in standard supervised learning. In this paper, we propose a new method for interactive learning from multiple noisy labels where we exploit the disagr…
ANTIDOTE reduces noisy labels influence during learning.
problem Learning with noisy labels.
method Information-divergence neighborhood relaxation and adversarial training.
result ANTIDOTE outperforms standard cross-entropy loss in noisy label settings.
CbMLC improves multi-label classification with noisy labels.
problem Evaluating multi-label classifiers with noisy labels.
method Context-Based Multi-Label Classifier (CbMLC) that handles noisy labels without additional supervision.
result CbMLC yields substantial improvements over previous methods in noisy label settings.
Proposes a new method to improve target annotation in ATR.
problem Challenges in annotating automatic target recognition due to lack of labeled data.
method Hybrid contrastive learning and cycle-consistency-based transductive transfer learning (C3TTL) framework.
result Significantly lower Fréchet Inception Distance (FID) score and improved performance in annotating civilian and military vehicles, as well as ship targets.
We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. More importantly, we incorporate model uncertainty into the sampling probability to com…
A framework learns dynamic soft labels to improve model generalization and accuracy.
problem Models trained on one-hot labels overfit and are sensitive to noisy annotations.
method Proposes a framework where labels are treated as learnable parameters, adapting dynamically during optimization.
result Consistent gains across different datasets and architectures, improving ResNet18 by 2.1% on CIFAR100.
CrossFilter tackles noisy labels in audio tagging.
problem Noisy labels in large audio datasets.
method CrossFilter framework using multiple representations and multi-task learning.
result Improves audio tagging performance on FSDKaggle2018 and FSDKaggle2019 datasets.
Proposes a new model for noisy labels considering multiple labelers and adversarial attacks.
problem Real-world noisy label models with multiple labelers and adversarial attacks.
method Labeler-dependent noise model with adversarial attack vectors.
result State-of-the-art approaches for learning from noisy labels are defeated by adversarial label attacks.
SoQal uses selective oracle questioning to improve active learning of cardiac signals.
problem Active learning of cardiac signals is challenging due to limited labelled data and high annotation costs.
method Proposes a framework combining selective oracle questioning and Bayesian active learning by consistency.
result SoQal outperforms baseline methods in active learning of cardiac signals, even with noisy oracles.
Collecting large training datasets, annotated with high-quality labels, is costly and time-consuming. This paper proposes a novel framework for training deep convolutional neural networks from noisy labeled datasets that can be obtained cheaply. The problem is formulated using an undirected graphical model that represe…
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
Study on noise models for noisy labels in NLP.
problem Quality of noise models from noisy labels.
method Theoretical analysis and synthetic dataset creation.
result Expected error of noise models derived.
Image classification problems are typically addressed by first collecting examples with candidate labels, second cleaning the candidate labels manually, and third training a deep neural network on the clean examples. The manual labeling step is often the most expensive one as it requires workers to label millions of im…
CORES2 removes noisy labels by sieving out corrupted examples.
problem Instance-dependent label noise degrades DNN performance.
method CORES2 (COnfidence REgularized Sample Sieve) progressively sieves out corrupted examples.
result CORES2 provides theoretical guarantees for filtering out corrupted examples.
Deep neural networks (DNNs) have achieved tremendous success in a variety of applications across many disciplines. Yet, their superior performance comes with the expensive cost of requiring correctly annotated large-scale datasets. Moreover, due to DNNs' rich capacity, errors in training labels can hamper performance. …
Leveraging weak or noisy supervision for building effective machine learning models has long been an important research problem. Its importance has further increased recently due to the growing need for large-scale datasets to train deep learning models. Weak or noisy supervision could originate from multiple sources i…
New method improves robustness of deep learning with noisy labels.
problem Robust deep learning on corrupted labels with noisy samples.
method Meta-transition adaptation through clean meta data guidance.
result More accurate estimation of noise transition matrix and classifier parameters.