arXiv research

PCBMs turn any neural network into interpretable models without dense annotations.

Self-supervised learning helps train deep features without needing lots of labeled data.

During active learning, an effective stopping method allows users to limit the number of annotations, which is cost effective. In this paper, a new stopping method called Predicted Change of F Measure will be introduced that attempts to provide the users an estimate of how much performance of the model is changing at e…

Active learning improves text classification performance by forecasting earlier.

LFD method improves text classification by making features clearer and less label-leaking.

Method makes non-interpretable models more intervenable.

Lack of labeled training data is a major bottleneck for neural network based aspect and opinion term extraction on product reviews. To alleviate this problem, we first propose an algorithm to automatically mine extraction rules from existing training examples based on dependency parsing results. The mined rules are the…

New models can't beat existing ones, so debiasing methods only slightly reduce needed labels.

New method improves adversarial learning with unlabeled data.

The success of deep learning has been due, in no small part, to the availability of large annotated datasets. Thus, a major bottleneck in current learning pipelines is the time-consuming human annotation of data. In scenarios where such input-output pairs cannot be collected, simulation is often used instead, leading t…

Annotating large unlabeled datasets can be a major bottleneck for machine learning applications. We introduce a scheme for inferring labels of unlabeled data at a fraction of the cost of labeling the entire dataset. Our scheme, bounded expectation of label assignment (BELA), greedily queries an oracle (or human labeler…

Personal assistant AI systems such as Siri, Cortana, and Alexa have become widely used as a means to accomplish tasks through natural language commands. However, components in these systems generally rely on supervised machine learning algorithms that require large amounts of hand-annotated training data, which is expe…

Model identifies urgent radiology reports with high accuracy.

TBAL reduces manual annotation but requires validated data.

Interactive weak supervision learns useful heuristics from user feedback.

Annotation of training data is the major bottleneck in the creation of text classification systems. Active learning is a commonly used technique to reduce the amount of training data one needs to label. A crucial aspect of active learning is determining when to stop labeling data. Three potential sources for informing …

Study shows annotation instrument design affects model performance in hate speech detection.

This paper focuses on a new task, i.e., transplanting a category-and-task-specific neural network to a generic, modular network without strong supervision. We design an functionally interpretable structure for the generic network. Like building LEGO blocks, we teach the generic network a new category by directly transp…

Extends V-IP framework to use LLMs for generating task-relevant concepts, improving interpretability and performance.

This paper focuses on a new task, i.e., transplanting a category-and-task-specific neural network to a generic, modular network without strong supervision. We design a functionally interpretable structure for the generic network. Like building LEGO blocks, we teach the generic network a new category by directly transpl…

Active learning selects both observations and annotation precision for Gaussian Processes.

Improves data labeling efficiency in machine learning.

One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations a…

Generative model combines multi-dimensional annotations for more accurate ground truth estimation.

Survey on AL strategies for cost-effective annotation in classification.

RAD improves robustness to domain annotation noise without explicit domain annotations.

Accurate annotation of medical image is the crucial step for image AI clinical application. However, annotating medical image will incur a great deal of annotation effort and expense due to its high complexity and needing experienced doctors. To alleviate annotation cost, some active learning methods are proposed. But …

BERT models can classify multilingual party manifestos across different dimensions.

New research shows non-bottlenecked autoencoders can outperform bottlenecked ones for anomaly detection.

PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.

ActiveLab improves classifier accuracy with fewer annotations by re-labeling.

Bayesian methods improve text annotation quality.

Paper proposes an efficient method for bounding box annotation in object detection.

Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model trai…

Framework learns interpretable concepts from data without interventions.

SCBMs model causal effects using low-dimensional bottlenecks.

Wide neural networks with narrow bottlenecks behave like deep Gaussian processes.

Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.

Annotation guidelines used to guide the annotation of training and evaluation datasets can have a considerable impact on the quality of machine learning models. In this study, we explore the effects of annotation guidelines on the quality of app feature extraction models. As a main result, we propose several changes to…

Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning. However, a recent study revealed that softmax can be a bottleneck of representational capacity of neural networks in language modeling (the softmax bottleneck). In this paper, we propose an…

Improves off-policy evaluation with imperfect annotations.

A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…

Crowdlab uses classifiers to estimate consensus labels and annotator quality.

The increase in data collection has made data annotation an interesting and valuable task in the contemporary world. This paper presents a new methodology for quickly annotating data using click-supervision and hierarchical object detection. The proposed work is semi-automatic in nature where the task of annotations is…

This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expert…

The paper proposes incentivizing human annotators with 'golden questions' to improve data quality.

Paper proposes active learning for sound event detection with reduced annotation effort.

The splendid success of convolutional neural networks (CNNs) in computer vision is largely attributable to the availability of massive annotated datasets, such as ImageNet and Places. However, in medical imaging, it is challenging to create such large annotated datasets, as annotating medical images is not only tedious…