arXiv research

Model for detecting rare labels in imbalanced crowdsourcing data.

End-to-end framework learns from imperfect annotations directly.

Study real-world noisy labels from human annotations for better understanding.

New method improves LLM judge accuracy by accounting for dependencies in aggregated binary labels.

Paper tackles robust classification under class-dependent domain shift.

Study reveals class-dependent effects in perturbation-based feature attribution metrics for time series classification.

AVDA transfers knowledge from source to target domains using embeddings.

MCAL reduces labeling costs by 6x for auto-labeling data sets.

Aims to eliminate domain bias in authentication without domain labels.

Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.

Proposes a game-theoretic approach for class-dependent rationalization.

Adversarial training is a useful approach to promote the learning of transferable representations across the source and target domains, which has been widely applied for domain adaptation (DA) tasks based on deep neural networks. Until very recently, existing adversarial domain adaptation (ADA) methods ignore the usefu…

Develops NPSVC++ to improve NPSVC performance through representation learning.

EmTract extracts emotions from financial social media text.

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and mult…

Machine learning approaches hold great potential for the automated detection of lung nodules in chest radiographs, but training the algorithms requires vary large amounts of manually annotated images, which are difficult to obtain. Weak labels indicating whether a radiograph is likely to contain pulmonary nodules are t…

Study shows annotation instrument design affects model performance in hate speech detection.

Tautological classes, or generalised Miller-Morita-Mumford classes, are basic characteristic classes of smooth fibre bundles, and have recently been used to describe the rational cohomology of classifying spaces of diffeomorphism groups for several types of manifolds. We show that rationally tautological classes depend…

Active learning selects both observations and annotation precision for Gaussian Processes.

One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations a…

Generative model combines multi-dimensional annotations for more accurate ground truth estimation.

Survey on AL strategies for cost-effective annotation in classification.

RAD improves robustness to domain annotation noise without explicit domain annotations.

The paper addresses over-fitting in deep learning models trained on imbalanced data.

Accurate annotation of medical image is the crucial step for image AI clinical application. However, annotating medical image will incur a great deal of annotation effort and expense due to its high complexity and needing experienced doctors. To alleviate annotation cost, some active learning methods are proposed. But …

The study challenges the notion that partial data annotation is inferior, suggesting it can sometimes outperform complete annotation.

PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.

ActiveLab improves classifier accuracy with fewer annotations by re-labeling.

Bayesian methods improve text annotation quality.

Paper proposes an efficient method for bounding box annotation in object detection.

Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model trai…

Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.

Annotation guidelines used to guide the annotation of training and evaluation datasets can have a considerable impact on the quality of machine learning models. In this study, we explore the effects of annotation guidelines on the quality of app feature extraction models. As a main result, we propose several changes to…

The paper extends spacetime topology results using codimension 2 null cut locus properties.

Improves off-policy evaluation with imperfect annotations.

A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…

Crowdlab uses classifiers to estimate consensus labels and annotator quality.

The increase in data collection has made data annotation an interesting and valuable task in the contemporary world. This paper presents a new methodology for quickly annotating data using click-supervision and hierarchical object detection. The proposed work is semi-automatic in nature where the task of annotations is…

This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expert…

The paper proposes incentivizing human annotators with 'golden questions' to improve data quality.

Paper proposes active learning for sound event detection with reduced annotation effort.

The splendid success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, in medical imaging, it is challenging to create such large annotated datasets, as annotating medical images is not only tedious…

Training 3D object detectors for autonomous driving has been limited to small datasets due to the effort required to generate annotations. Reducing both task complexity and the amount of task switching done by annotators is key to reducing the effort and time required to generate 3D bounding box annotations. This paper…

Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collec…

Mathematical approach assesses human resource competences accurately.

Interactive IL beats BC by state-wise annotation cost.

We construct a family of hyperbolic link complements by gluing tangles along totally geodesic four-punctured spheres, then investigate the commensurability relation among its members. Those with different volume are incommensurable, distinguished by their scissors congruence classes. Mutation produces arbitrarily large…

This paper considers the existence of conformally compact Einstein metrics on 4-manifolds. A reasonably complete understanding is obtained for the existence of such metrics with prescribed conformal infinity, when the conformal infinity is of positive scalar curvature. We find in particular that general solvability in …