The paper improves self-training in semi-supervised learning by selecting more robust pseudo-labeled data.
problem Improving the reliability of pseudo-labeled data selection in self-training for semi-supervised learning.
method Proposes a multi-objective utility function to select pseudo-labeled data that maximizes reliability, considering model selection, accumulation of errors, and covariate shift uncertainties.
result Robustness towards model choice can lead to substantial accuracy gains in self-training.
Pseudo-label selection affects semi-supervised learning performance.
problem Selection of pseudo-labeled data impacts semi-supervised learning's generalization performance.
method Embedding pseudo-label selection into decision theory, deriving a novel selection criterion based on posterior predictive.
result BPLS (Bayesian pseudo-label selection) outperforms traditional methods in overfitting-prone data.
MTL method uses unlabeled data with pseudo labels to improve classification with disjoint datasets.
problem Improving classification performance with disjoint labeled datasets using unlabeled data.
method Proposes MTL-SA method to select and augment unlabeled data with confident pseudo labels and close distribution to labeled data.
result Extensive experiments show the effectiveness of MTL-SA method in improving classification performance.
Bayesian Pseudo Label Selection reduces overfitting in semi-supervised learning.
problem Overfitting in pseudo-label selection for semi-supervised learning.
method BPLS, a Bayesian framework that approximates the posterior predictive of pseudo-samples.
result BPLS outperforms traditional PLS methods, especially in high-dimensional data.
A method for selecting pseudo-labeled data in semi-supervised learning using generalized Bayes and soft revision.
problem Selecting pseudo-labeled data for semi-supervised learning with robustness to uncertainty.
method Using credal sets and the Gamma-Maximin method with soft revision to update priors and select pseudo-labeled data.
result The Gamma-Maximin method with soft revision can achieve promising results, especially in scenarios with low labeled data proportions.
A single pre-trained agent guides feature selection using knockoffs.
problem Feature selection challenges in AI-readiness of data.
method Generates knockoff features and uses reinforcement learning.
result Optimal feature subset identified with reduced dependency on target variable.
Dash selects dynamic pseudo labels from unlabeled data for semi-supervised learning.
problem Efficiently using unlabeled data in semi-supervised learning while avoiding incorrect pseudo labels.
method Dynamic thresholding to select a subset of unlabeled examples for training.
result Dash achieves theoretical convergence and outperforms state-of-the-art methods empirically.
Often domain adaptation is performed using a discriminator (domain classifier) to learn domain-invariant feature representations so that a classifier trained on labeled source data will generalize well to unlabeled target data. A line of research stemming from semi-supervised learning uses pseudo labeling to directly g…
Framework for domain adaptation using pseudo-labels from unlabeled data.
problem Improving prediction accuracy in target domain with covariate shift.
method Kernel GLMs with labeled and pseudo-labeled data, using imputation model for target data.
result Non-asymptotic excess-risk bounds for effective labeled sample size.
Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain su…
New method removes pseudo-label bias for unsupervised domain adaptation.
problem Class imbalance and distribution shift between domains.
method Implicit class-conditioned domain alignment without explicit pseudo-label optimization.
result Effective in handling within-domain class imbalance and between-domain class distribution shift.
In this paper, we propose a new wrapper feature selection approach with partially labeled training examples where unlabeled observations are pseudo-labeled using the predictions of an initial classifier trained on the labeled training set. The wrapper is composed of a genetic algorithm for proposing new feature subsets…
Develops a method for kernel ridge regression under covariate shift using pseudo-labels.
problem Learning a regression function with small mean squared error over a target distribution with labeled data from a different feature distribution.
method Split labeled data into two subsets, conduct kernel ridge regression on each, use imputation model to fill missing labels, and select the best candidate model.
result Non-asymptotic excess risk bounds demonstrate effective adaptation to target distribution and covariate shift.
A new SSL method uses instance-dependent thresholds to improve accuracy.
problem Improving semi-supervised learning by better selecting confident unlabeled instances.
method Proposes instance-dependent thresholds that vary based on the ambiguity and error rates of pseudo-labels for each unlabeled instance.
result Demonstrates that instance-dependent thresholds provide a probabilistic guarantee for correct pseudo-labels.
Mutual teaching improves graph models with less labeled data.
problem Training graph models with limited labeled data.
method Dual model training with mutual teaching strategy.
result Significant performance improvement with less labeled data.
A new confidence measure improves self-training in biased data.
problem Improving self-training in biased data.
method Proposes a new confidence measure, T-similarity, based on ensemble diversity of linear classifiers.
result Empirically shows the benefit of T-similarity for pseudo-labeling policies on various datasets.
PAS method improves UDA by progressively refining subspaces for reliable pseudo-labels.
problem Mitigating mode collapse in unsupervised domain adaptation.
method Progressive Adaptation of Subspaces (PAS) approach.
result PAS effectively mitigates mode collapse and improves performance in UDA and PDA.
Meta Pseudo Labels boosts image classification accuracy to 90.2%.
problem Improving semi-supervised learning for image classification.
method Adapts a teacher network to generate better pseudo labels through student feedback.
result Achieves a new state-of-the-art top-1 accuracy of 90.2% on ImageNet.
MTSSL optimizes threshold τ for better semi-supervised learning performance.
problem Optimizing the threshold τ for effective semi-supervised learning.
method Meta-Thresholding approach to optimize τ during training.
result Optimal values of τ are not necessary for achieving similar performance.
Proposes PUUPL for PUL in imbalanced datasets, boosting minority class signals.
problem Imbalanced datasets and model calibration in PUL.
method Uncertainty-aware pseudo-labeling procedure (PUUPL).
result Substantial performance gains in highly imbalanced settings.
Doubly robust self-training improves semi-supervised learning by balancing labeled and pseudo-labeled data.
problem Improving semi-supervised learning performance with limited labeled data.
method Introduces doubly robust self-training, a method that combines labeled and pseudo-labeled data to balance between labeled-only and pseudo-labeled-only training.
result Demonstrates superior performance of doubly robust self-training on ImageNet and nuScenes datasets.
Most recent semi-supervised deep learning (deep SSL) methods used a similar paradigm: use network predictions to update pseudo-labels and use pseudo-labels to update network parameters iteratively. However, they lack theoretical support and cannot explain why predictions are good candidates for pseudo-labels. In this p…
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
Solves biased pseudo-labels in imbalanced SSL by refining them.
problem Imbalanced class distributions in semi-supervised learning lead to biased pseudo-labels.
method Formulates a convex optimization problem to refine pseudo-labels and develops an efficient algorithm, DARP.
result Demonstrates the effectiveness of DARP in various imbalanced semi-supervised scenarios.
This paper introduces a method to select and weight pretext tasks for better self-supervised speech representation learning.
problem Combining pretext tasks for better performance in self-supervised speech representation learning.
method Estimating calibrated weights for partial losses corresponding to pretext tasks during self-supervised training.
result Groups of selected and weighted pretext tasks perform better than classic baselines in automatic speech recognition and speaker/emotion recognition.
Paper improves short text clustering by integrating semantic relationships into Optimal Transport.
problem Erroneous pseudo-labels caused by neglecting semantic consistency in existing OT methods.
method Designs an instance-level attention mechanism to capture semantic relationships and integrates them into the OT formulation.
result Generates reliable pseudo-labels that improve clustering accuracy.
In this paper we revisit the idea of pseudo-labeling in the context of semi-supervised learning where a learning algorithm has access to a small set of labeled samples and a large set of unlabeled samples. Pseudo-labeling works by applying pseudo-labels to samples in the unlabeled set by using a model trained on the co…
CAT improves domain adaptation for fault diagnosis by calibrating teacher network predictions.
problem Performance drops in deep learning models when applied to different data distributions.
method CAT uses post-hoc calibration techniques to calibrate predictions of the teacher network during self-training.
result CAT achieves state-of-the-art performance on most transfer tasks in domain-adaptive IFD.
Contrastive regularization improves semi-supervised learning by better propagating confident pseudo-labels.
problem Consistency regularization's limitation in high performance and efficiency.
method Proposes contrastive regularization to update model features, pushing confident labels into unlabeled samples.
result Improves semi-supervised learning tasks with fewer training iterations and robust performance.
Improves SSL with doubly robust estimation of unlabeled class distribution.
problem Limited labeled data and long-tailed class distributions in unlabeled data.
method Explicitly estimate unlabeled class distribution using doubly robust estimator.
result Improves performance of SSL methods on unlabeled data.
POTA improves short text clustering by generating reliable pseudo-labels.
problem Limited discriminative representations in short texts.
method POTA uses instance-level attention and optimal transport for semantic consistency and cluster structure.
result POTA outperforms state-of-the-art methods in short text clustering.
GOLFS selects features for clustering by combining global and local information.
problem Feature selection for high-dimensional clustering without labels.
method Combines global and local information via manifold learning and regularized self-representation.
result Improves feature selection and clustering accuracy.
Proposes ConstraintMatch for semi-supervised clustering with unconstrained data.
problem Leveraging unconstrained data alongside constraints for clustering models.
method Semi-supervised context with pseudo-constraining and pseudo-labeling mechanisms.
result Demonstrates effectiveness of ConstraintMatch over baselines.
RAEUFS selects features from data without labels, improving robustness to outliers.
problem Feature selection in high-dimensional data, especially in the presence of outliers.
method RAEUFS uses a deep autoencoder to learn nonlinear feature representations, improving robustness to outliers.
result RAEUFS outperforms state-of-the-art UFS methods in both clean and outlier-contaminated data settings.
Mitigates confirmation bias in SSL by adjusting pseudo labels dynamically.
problem Confirmation bias in semi-supervised learning leads to errors in pseudo labels.
method TaMatch framework adjusts scaling ratio to debias pseudo labels and dynamically adjusts target distribution.
result TaMatch significantly outperforms existing methods in SSL tasks.
The paper improves semi-supervised learning using f-divergences and α-Rényi divergences.
problem Improving semi-supervised learning with noisy pseudo-labels.
method Inspired by f-divergences and α-Rényi divergences, the paper develops new empirical risk functions and regularization techniques. result The new methods show better performance than traditional self-training methods, especially in noisy pseudo-label scenarios.
DCEM algorithm reduces bias in machine learning models trained on selective labels.
problem Bias in machine learning models trained on selective labels.
method Disparate Censorship Expectation-Maximization (DCEM) algorithm.
result DCEM improves bias mitigation without sacrificing discriminative performance.
New PL method improves ASR accuracy without pseudo-labels.
problem Improving ASR accuracy with limited labeled data.
method End-to-end continuous pseudo-labeling with soft-labels.
result Soft-labels can lead to model collapse, but regularization can mitigate this.
Self-training in linear models shows a U-shaped test-risk curve due to signal forgetting and denoising.
problem Understanding the dynamics of iterative self-training in high-dimensional linear regression.
method Derivation of deterministic-equivalent recursions for prediction risk and effective noise, analysis of signal forgetting and denoising effects.
result An optimal early-stopping time is determined, and a U-shaped test-risk curve is observed.
DoubleMatch combines pseudo-labeling with self-supervision for SSL.
problem Lack of effective use of unlabeled data in SSL.
method Combines pseudo-labeling with self-supervised loss.
result Achieves state-of-the-art accuracies on multiple datasets.
GUST framework improves self-training by estimating node uncertainty and generating pseudo-labels.
problem Over-confidence in pseudo-labels during self-training.
method Graph-based uncertainty-aware self-training with stochastic node labeling.
result GUST achieves state-of-the-art performance, especially in sparse labeled data settings.
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels usin…
Proposes a new contrastive loss for semi-supervised medical image segmentation.
problem Lack of labeled data for medical image segmentation.
method Uses pseudo-labels and a local contrastive loss to learn good local representations.
result Achieved high segmentation performance on public cardiac and prostate datasets.
Self-training improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.
problem Understanding how self-training improves generalization in high-dimensional Gaussian mixtures.
method Analyzing iterative self-training on binary Gaussian mixtures in the asymptotic limit.
result ST improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.
Self-training algorithm improves classifier performance with labeled and unlabeled data.
problem Improving classifier performance with limited labeled data.
method Iterative learning of halfspaces, exploration and pruning phases.
result Misclassification error is bounded and never degrades compared to initial labeled set.
A new metric predicts model performance on unseen data.
problem Predicting performance on out-of-distribution data without labels.
method Uses model predictions to pseudo-label data, trains a new model, and measures difference from in-distribution models.
result Empirically outperforms existing methods on image and text classification tasks.
New method estimates covariance in deep heteroscedastic regression without labels.
problem Estimating covariance in deep heteroscedastic models is challenging due to sample-dependent covariance and lack of ground truth.
method Proposes a self-supervised approach using KL Divergence and 2-Wasserstein distance for covariance estimation and a neighborhood-based heuristic for pseudo labels.
result Demonstrates effective pseudo labels and a computationally cheaper yet accurate deep heteroscedastic regression.
Self-training improves model accuracy by refining pseudo-labels.
problem Improving semi-supervised learning with self-training.
method Theoretical insights into self-training algorithm with a focus on linear classifiers.
result Self-training iterations can improve model accuracy even if stuck in sub-optimal fixed points.