arXiv research

Study improves app feature extraction models with new annotation guidelines and data.

Pipeline detects pulmonary embolisms from sparse CT images.

West Frisian lemmatizer, POS tagger, and parser created.

This paper evaluates AI ethics guidelines and their implementation.

This paper tackles the curse of dimensionality in semi-supervised learning using Laplacian regularization.

Researchers analyze tagging patterns on Stack Exchange communities.

This work simplifies choosing reinforcement-learning algorithms.

Study shows annotation instrument design affects model performance in hate speech detection.

New method to estimate doctors' effort in annotating medical images.

Active learning selects both observations and annotation precision for Gaussian Processes.

One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations a…

Generative model combines multi-dimensional annotations for more accurate ground truth estimation.

Survey on AL strategies for cost-effective annotation in classification.

RAD improves robustness to domain annotation noise without explicit domain annotations.

The study challenges the notion that partial data annotation is inferior, suggesting it can sometimes outperform complete annotation.

Framework for applying GPs to real-world data with scalability guidelines.

A new method uses triplet embeddings to improve human annotation for hidden constructs.

PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.

ActiveLab improves classifier accuracy with fewer annotations by re-labeling.

Bayesian methods improve text annotation quality.

Paper proposes an efficient method for bounding box annotation in object detection.

Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.

A new method for faster data annotation using click-supervision and hierarchical object detection.

Improves off-policy evaluation with imperfect annotations.

A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…

Crowdlab uses classifiers to estimate consensus labels and annotator quality.

Method learns true labels from noisy annotators using regularization.

Cluster annotation reduces active learning human interactions.

This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expert…

The paper proposes incentivizing human annotators with 'golden questions' to improve data quality.

Paper proposes active learning for sound event detection with reduced annotation effort.

The splendid success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, in medical imaging, it is challenging to create such large annotated datasets, as annotating medical images is not only tedious…

Training 3D object detectors for autonomous driving has been limited to small datasets due to the effort required to generate annotations. Reducing both task complexity and the amount of task switching done by annotators is key to reducing the effort and time required to generate 3D bounding box annotations. This paper…

Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collec…

Interactive IL beats BC by state-wise annotation cost.

End-to-end framework learns from imperfect annotations directly.

Learning algorithms normally assume that there is at most one annotation or label per data point. However, in some scenarios, such as medical diagnosis and on-line collaboration,multiple annotations may be available. In either case, obtaining labels for data points can be expensive and time-consuming (in some circumsta…

Bayesian model improves truth inference from highly redundant crowd annotations.

Meta-learning method for accurate classifier from noisy annotators' data.

New method uses image-level and pixel-level annotations for brain tumor segmentation.

Sketched SVD improves SVD runtime for large datasets.

Proposes guidelines for developing medical AI products.

Annotating temporal relations (TempRel) between events described in natural language is known to be labor intensive, partly because the total number of TempRels is quadratic in the number of events. As a result, only a small number of documents are typically annotated, limiting the coverage of various lexical/semantic …

To be effective, state of the art machine learning technology needs large amounts of annotated data. There are numerous compelling applications in healthcare that can benefit from high performance automated decision support systems provided by deep learning technology, but they lack the comprehensive data resources req…

This paper proposes an approach for rapid bounding box annotation for object detection datasets. The procedure consists of two stages: The first step is to annotate a part of the dataset manually, and the second step proposes annotations for the remaining samples using a model trained with the first stage annotations. …

ECN framework improves training on noisy structured labels.

Guidelines for using explainable ML to avoid misuse.

Semi-automatic data annotation helps experts label unlabeled samples based on feature space projection.