PLRM synthesizes labels from mismatched sources for better training sets.
problem Creating labeled training sets is a major challenge in machine learning.
method PLRM uses probabilistic modeling to synthesize labels from indirect supervision sources with different output spaces.
result PLRM outperforms baselines by 2%-9% on various tasks.
Different types of training data have led to numerous schemes for supervised classification. Current learning techniques are tailored to one specific scheme and cannot handle general ensembles of training data. This paper presents a unifying framework for supervised classification with general ensembles of training dat…
DoubleMatch combines pseudo-labeling with self-supervision for SSL.
problem Lack of effective use of unlabeled data in SSL.
method Combines pseudo-labeling with self-supervised loss.
result Achieves state-of-the-art accuracies on multiple datasets.
FROST speeds up and stabilizes one-shot semi-supervised learning.
problem Slow training and sensitivity to labeled data choices in semi-supervised learning.
method Combines semi-supervised learning with a one-stage, single network self-training approach.
result FROST trains up to 10x faster and is more robust to labeled data choices.
Improved voice conversion with semi-supervised learning.
problem Voice conversion with limited parallel data.
method Amortized variational inference with parallel and non-parallel utterances.
result Semi-supervised training improves voice conversion performance.
Survey on self-supervised pre-training for neural networks using unlabeled data.
problem Improving model performance using unlabeled data.
method Pre-training on unlabeled data followed by task-specific adaptation.
result Enhanced model performance through self-supervised pre-training.
Supervised object detection and semantic segmentation require object or even pixel level annotations. When there exist image level labels only, it is challenging for weakly supervised algorithms to achieve accurate predictions. The accuracy achieved by top weakly supervised algorithms is still significantly lower than …
SuNCEt accelerates contrastive learning with minimal labeled data.
problem Efficiently learning visual representations with limited labeled data.
method Noise-contrastive estimation and neighbourhood component analysis-based semi-supervised loss.
result SuNCEt achieves semi-supervised learning accuracy with less than half the labeled data.
Self-supervision improves GCNs' generalizability and robustness.
problem Improving graph convolutional networks' performance.
method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.
result Self-supervision enhances GCNs' robustness and generalizability.
Study investigates one-shot semi-supervised learning for image classification.
problem Training deep networks requires many labeled samples, limiting adoption.
method Empirical investigation of FixMatch method for one-shot semi-supervised learning.
result Uneven class accuracy is a barrier to high performance in one-shot semi-supervised learning.
Novel approach trains ASR models with less supervision using bilevel optimization.
problem Training acoustic models for ASR with minimal supervision.
method Bilevel optimization with unsupervised and supervised losses.
result Achieves superior performance compared to existing methods.
Adversarial training provides a means of regularizing supervised learning algorithms while virtual adversarial training is able to extend supervised learning algorithms to the semi-supervised setting. However, both methods require making small perturbations to numerous entries of the input vector, which is inappropriat…
Self-supervised learning improves few-shot classification and segmentation on point clouds.
problem Efficiently learn from limited labeled data in point cloud applications.
method Hierarchical cover-tree partitioning for self-supervised pre-training; restricted to support set for few-shot learning.
result Self-supervised learning significantly improves downstream classification and segmentation accuracy.
Paper tackles domain invariant sentiment classification using weak supervision.
problem Learning a sentiment classification model that adapts to any target domain.
method Two-stage training procedure with weakly supervised datasets.
result Transfer learning with weak supervision achieves performance close to supervised training.
Study compares memorization of SimCLR to supervised and random labels training.
problem Understanding memorization in contrastive learning.
method Investigated SimCLR's memorization properties compared to supervised and random labels training.
result SimCLR's memorization is similar to random labels training in terms of training object complexity distribution.
Improved ASR for English-isiZulu code-switched speech with semi-supervised training.
problem Improving ASR for code-switched speech between English and isiZulu.
method Semi-supervised training using automatic transcription of multilingual speech data.
result Semi-supervised training achieved significant WER reduction in ASR performance.
Improved self-supervised learning for document images.
problem Performance of self-supervised pre-training on document images is poor.
method Proposed context-aware alternatives and a novel multi-modal method.
result Novel method outperforms other self-supervised methods on document image classification.
We consider the task of training classifiers without labels. We propose a weakly supervised method---adversarial label learning---that trains classifiers to perform well against an adversary that chooses labels for training data. The weak supervision constrains what labels the adversary can choose. The method therefore…
BOSS learns from one labeled sample per class to match fully supervised performance.
problem Achieving fully supervised performance with minimal labeled data.
method Combines class prototype refining, class balancing, and self-training.
result BOSS achieves comparable test accuracies to fully supervised learning.
S4 learns new self-supervision automatically, improving accuracy with less human effort.
problem Lack of direct supervision in machine learning.
method Combines deep learning and probabilistic logic to automatically generate and verify new self-supervision.
result S4 can automatically propose accurate self-supervision, matching supervised methods with less human effort.
We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in …
Graph Convolutional Networks(GCNs) play a crucial role in graph learning tasks, however, learning graph embedding with few supervised signals is still a difficult problem. In this paper, we propose a novel training algorithm for Graph Convolutional Network, called Multi-Stage Self-Supervised(M3S) Training Algorithm, co…
A method for collecting human supervision that combines rules and instance labels.
problem Lack of labeled data and inefficient human supervision.
method Rule-exemplar method with training algorithm for joint denoising and model training.
result Our algorithm is more accurate than existing methods and effectively denoises rules.
Study compares different levels of supervision for training graph embeddings in wireless networks.
problem Improving power control in wireless interference networks.
method Training graph neural networks (GNNs) with different levels of supervision (supervised, unsupervised, self-supervised).
result Different levels of supervision impact system-level throughput, convergence, and generalization.
Self-supervised skip-tree training improves mathematical reasoning in language models.
problem Improving logical reasoning in language models for formal mathematics.
method Self-supervised language modeling on mathematical formulas, skip-tree task.
result Models trained on skip-tree task outperform standard models in mathematical reasoning tasks.
Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Previous algorithms based on consistency regularization can harness the abundance of unlabeled data to pr…
We combine supervised learning with unsupervised learning in deep neural networks. The proposed model is trained to simultaneously minimize the sum of supervised and unsupervised cost functions by backpropagation, avoiding the need for layer-wise pre-training. Our work builds on the Ladder network proposed by Valpola (…
A new method for weakly supervised learning that improves model accuracy.
problem Training machine learning models with precise labels is expensive; weak supervision provides a low-cost alternative.
method Data consistent weak supervision algorithm that searches over classifiers to find plausible labelings, considering features of the training data and estimating labels for low/no coverage data.
result Empirically, the method significantly outperforms state-of-the-art weak supervision methods on text and image classification tasks.
We describe an approach to Grammatical Error Correction (GEC) that is effective at making use of models trained on large amounts of weakly supervised bitext. We train the Transformer sequence-to-sequence model on 4B tokens of Wikipedia revisions and employ an iterative decoding strategy that is tailored to the loosely-…
Deep neural networks are gaining increasing popularity for the classic text classification task, due to their strong expressive power and less requirement for feature engineering. Despite such attractiveness, neural text classification models suffer from the lack of training data in many real-world applications. Althou…
Interactive machine learning with weak supervision and pre-trained embeddings.
problem Training machine learning models with limited labeled data.
method Use pre-trained embeddings to define a distance function and extend source votes to nearby points.
result Significantly outperforms traditional weakly-supervised and fully-supervised methods.
Paper compares semi-supervised training for differentiable particle filters.
problem Lack of labelled data in real-world applications.
method Compares two semi-supervised training objectives.
result Improved performance in environments with scarce labelled data.
ProbKT uses probabilistic logical reasoning to train object detection models with weak supervision.
problem Training object detection models requires instance-level annotations, which are often unavailable.
method ProbKT, a framework based on probabilistic logical reasoning, uses arbitrary types of weak supervision.
result ProbKT leads to significant improvement and better generalization compared to existing baselines.
A new method for training generative models with sparse supervision.
problem Training deep generative models with sparse and varying supervision.
method Caffeinated Wake-Sleep (CWS) method, combining reweighted wake-sleep and teacher-forcing.
result The CWS method is robust to variable length supervision and performs well on various datasets.
ASTRA uses unlabeled data and weak rules to train deep models effectively.
problem Learning with weak supervision rules is challenging due to their heuristic and noisy nature.
method ASTRA framework that considers contextualized representations and pseudo-labels for unlabeled data, and a rule attention network to aggregate labels.
result Significant improvements over state-of-the-art baselines on text classification benchmarks.
Doubly robust self-training improves semi-supervised learning by balancing labeled and pseudo-labeled data.
problem Improving semi-supervised learning performance with limited labeled data.
method Introduces doubly robust self-training, a method that combines labeled and pseudo-labeled data to balance between labeled-only and pseudo-labeled-only training.
result Demonstrates superior performance of doubly robust self-training on ImageNet and nuScenes datasets.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
RIO uses rotation-equivariance to train robust inertial odometry models.
problem Training robust inertial odometry models with limited labeled data.
method Rotation-equivariance as self-supervisor, adaptive Test-Time Training (TTT).
result RIO-trained models achieve on-par performance with full-labeled data models.
The paper improves semi-supervised learning using f-divergences and α-Rényi divergences.
problem Improving semi-supervised learning with noisy pseudo-labels.
method Inspired by f-divergences and α-Rényi divergences, the paper develops new empirical risk functions and regularization techniques. result The new methods show better performance than traditional self-training methods, especially in noisy pseudo-label scenarios.
Self-training improves model accuracy by refining pseudo-labels.
problem Improving semi-supervised learning with self-training.
method Theoretical insights into self-training algorithm with a focus on linear classifiers.
result Self-training iterations can improve model accuracy even if stuck in sub-optimal fixed points.
GANs involve training two networks in an adversarial game, where each network's task depends on its adversary. Recently, several works have framed GAN training as an online or continual learning problem. We focus on the discriminator, which must perform classification under an (adversarially) shifting data distribution…
In many machine learning scenarios, supervision by gold labels is not available and consequently neural models cannot be trained directly by maximum likelihood estimation (MLE). In a weak supervision scenario, metric-augmented objectives can be employed to assign feedback to model outputs, which can be used to extract …
As machine learning models continue to increase in complexity, collecting large hand-labeled training sets has become one of the biggest roadblocks in practice. Instead, weaker forms of supervision that provide noisier but cheaper labels are often used. However, these weak supervision sources have diverse and unknown a…
We investigate the effects of the unsupervised pre-training method under the perspective of information theory. If the input distribution displays multiple views of the supervision, then unsupervised pre-training allows to learn hierarchical representation which communicates these views across layers, while disentangli…
Improves understanding of PWS by calculating influence of sources and data.
problem Understanding the influence of each component in PWS.
method Proposes source-aware Influence Function (IF) to decompose and calculate influence.
result Improves end model's generalization performance and identifies mislabeling.
Recent works demonstrated the usefulness of temporal coherence to regularize supervised training or to learn invariant features with deep architectures. In particular, enforcing smooth output changes while presenting temporally-closed frames from video sequences, proved to be an effective strategy. In this paper we pro…
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
Building a large image dataset with high-quality object masks for semantic segmentation is costly and time consuming. In this paper, we introduce a principled semi-supervised framework that only uses a small set of fully supervised images (having semantic segmentation labels and box labels) and a set of images with onl…