SupSiam and SupBYOL improve supervised representation learning with ANCL.
problem Improving supervised representation learning with ANCL.
method Proposed supervised ANCL framework leveraging labels to avoid collapse.
result Supervised ANCL improves representation learning across various datasets and tasks.
S4 learns new self-supervision automatically, improving accuracy with less human effort.
problem Lack of direct supervision in machine learning.
method Combines deep learning and probabilistic logic to automatically generate and verify new self-supervision.
result S4 can automatically propose accurate self-supervision, matching supervised methods with less human effort.
Self-supervised and supervised methods learn similar intermediate visual representations but diverge in final layers.
problem Comparing self-supervised and supervised methods for visual learning.
method Comparison of contrastive self-supervised and supervised methods on simple image data.
result Contrastive and supervised methods learn similar intermediate representations but diverge in final layers.
Survey of self-supervised learning methods in computer vision, NLP, and graph learning.
problem Manual labeling and vulnerability to attacks in supervised learning.
method Generative, contrastive, and generative-contrastive approaches.
result Self-supervised learning achieves high performance in representation learning.
Proposes a framework for semi-supervised continual learning from sequentially arriving data.
problem Learning from data with changing task distribution over time, especially in domains with a mix of labeled and unlabeled data.
method Meta-Consolidation for Continual Semi-Supervised Learning (MCSSL) framework with a hypernetwork and semi-supervised auxiliary classifier.
result Significant improvements in continual semi-supervised learning setting.
Improved VAE learns disentangled representations with less supervision.
problem Learning disentangled representations is challenging.
method Semi-supervised disentanglement learning with label replacement.
result Significant improvement in disentanglement with minimal supervision.
Improves label propagation for weakly supervised learning.
problem Reducing the need for labeled data in machine learning.
method Label Propagation with Weak Supervision (LPA) analysis.
result Demonstrated improvements over existing methods on weakly supervised classification tasks.
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
Paper proposes an algorithm to recover full supervision from weakly labeled data.
problem Machine learning requires expensive data annotation, motivating the use of weak supervision.
method The paper introduces a disambiguation principle and an empirical disambiguation algorithm for partial labelling.
result The algorithm achieves exponential convergence rates under learnability assumptions.
BOSS learns from one labeled sample per class to match fully supervised performance.
problem Achieving fully supervised performance with minimal labeled data.
method Combines class prototype refining, class balancing, and self-training.
result BOSS achieves comparable test accuracies to fully supervised learning.
Generative Adversarial Networks (GAN) have shown promising results on a wide variety of complex tasks. Recent experiments show adversarial training provides useful gradients to the generator that helps attain better performance. In this paper, we intend to theoretically analyze whether supervised learning with adversar…
Improved self-supervised learning on ImageNet achieves top-1 accuracy of 77.1%.
problem Self-supervised ResNets underperform supervised learning on ImageNet.
method ReLICv2 combines explicit invariance loss with contrastive objective over varied data views.
result ReLICv2 achieves 77.1% top-1 accuracy on ImageNet, improving over previous state-of-the-art by 1.5%.
This paper analyzes self-supervised learning from a multi-view perspective.
problem Understanding and optimizing self-supervised learning from multi-view data.
method Information-theoretical framework to understand and design self-supervised learning objectives.
result Self-supervised representations can extract task-relevant information and discard task-irrelevant information.
Unified framework for semi-supervised learning reduces annotation needs.
problem Sparse annotations and large amounts of unlabeled data in computational pathology.
method S5CL integrates fully-supervised, self-supervised, and semi-supervised learning through hierarchical contrastive losses.
result S5CL improves accuracy and F1-score in histopathological datasets with sparse labels.
We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in …
Paper tackles supervision bottleneck in machine learning.
problem Difficulty in generating supervision signals for learning models.
method Describes several learning paradigms to alleviate the supervision bottleneck.
result Illustrates the benefit of these paradigms in inducing semantic representations from text.
This paper provides an overview of deep semi-supervised learning methods.
problem Reducing the need for large annotated datasets in deep learning.
method Summarizes dominant semi-supervised approaches in deep learning.
result Provides a comprehensive overview of deep semi-supervised learning.
Generative model explains self-supervised learning across various tasks.
problem Lack of theoretical understanding of self-supervised learning methods.
method Generative latent variable model for self-supervised learning.
result Improves representation learning performance and narrows the gap between generative and discriminative methods.
Study compares semi-supervised learning methods for anomaly detection in hydraulic systems.
problem Challenges in anomaly detection for hydraulic systems due to scarce labeled data.
method Compared various semi-supervised learning methods including traditional models and deep neural networks.
result Customized extreme learning machine (HELM) model achieved state-of-the-art performance.
Self-supervised learning excels in topic modeling by being less model-specific.
problem How self-supervised learning discovers useful representations in topic models.
method Applying self-supervised learning objectives to topic model-generated data.
result Self-supervised learning objectives can recover useful posterior information for topic models, outperforming misspecified models.
New approach combines semi-supervised learning and bandits for better predictions.
problem Online semi-supervised learning with bandit feedback for applications like clinical trials and ad recommendations.
method Adjusted Graph Convolutional Network (GCN) for contextual bandits, with semi-supervised missing rewards imputation.
result Developed multi-GCN embedded contextual bandit algorithms verified on real-world datasets.
Unified framework for policy learning using weak supervision.
problem High-quality supervision is often infeasible or expensive in practice.
method Treat weak supervision as imperfect peer information and evaluate policies based on correlated agreement.
result Substantial performance improvements, especially in complex or noisy environments.
Poisson learning doesn't solve graph semi-supervised learning issues.
problem Global information loss in graph-based semi-supervised learning.
method Poisson learning is Laplace regularization with thresholding.
result Poisson learning cannot overcome the global information loss problem.
In this study, importance of user inputs is studied in the context of personalizing human activity recognition models using incremental learning. Inertial sensor data from three body positions are used, and the classification is based on Learn++ ensemble method. Three different approaches to update models are compared:…
Self-supervised learning performs better than supervised learning on imbalanced datasets.
problem The performance gap between balanced and imbalanced pre-training with self-supervised learning is smaller than with supervised learning.
method Systematic investigation of self-supervised learning under dataset imbalance, including experiments and theoretical analyses.
result Self-supervised representations are more robust to class imbalance than supervised representations.
Proposes supervised method for whole DAG causal structure learning.
problem Learning causal directions from data, especially for whole DAG structure.
method Supervised learning approach using permutation equivariant models.
result Promising results compared to previous approaches on synthetic and real data.
Develops a framework for fair semi-supervised learning.
problem Balancing fairness and accuracy in semi-supervised learning.
method Formulates a framework as an optimization problem, incorporating classifier loss, label propagation loss, and fairness constraints.
result Achieves fair semi-supervised learning with better accuracy-fairness trade-off than fair supervised learning.
Study shows semi-supervised learning can be more robust with fewer labeled examples.
problem Learning robust predictors in semi-supervised PAC model with minimal labeled data.
method Characterizes the minimal labeled and unlabeled data required for robust learning.
result Proves nearly matching upper and lower bounds on labeled sample complexity.
Labeled data used for training activity recognition classifiers are usually limited in terms of size and diversity. Thus, the learned model may not generalize well when used in real-world use cases. Semi-supervised learning augments labeled examples with unlabeled examples, often resulting in improved performance. Howe…
Is all of machine learning supervised to some degree? The field of machine learning has traditionally been categorized pedagogically into supervised vs unsupervised learning; where supervised learning has typically referred to learning from labeled data, while unsupervised learning has typically referred to learning …
New framework analyzes why more negative samples improve self-supervised learning performance.
problem Inconsistency between theoretical degradation and empirical improvement of downstream supervised tasks with more negative samples.
method Coupon collector's problem framework to analyze self-supervised representation learning with more negative samples.
result Bound can implicitly incorporate supervised loss in self-supervised loss by increasing negative samples.
Self-supervised learning improves ECG classification performance.
problem Label scarcity in clinical 12-lead ECG data.
method Adapted self-supervised methods to ECG domain, focusing on contrastive representations and latent forecasting.
result Contrastive predictive coding adaptation yields linear evaluation performance only 0.5% below supervised performance.
Study compares different levels of supervision for training graph embeddings in wireless networks.
problem Improving power control in wireless interference networks.
method Training graph neural networks (GNNs) with different levels of supervision (supervised, unsupervised, self-supervised).
result Different levels of supervision impact system-level throughput, convergence, and generalization.
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
Unified framework for learning with indirect supervision signals.
problem Learning from indirect supervision signals when gold labels are missing or costly.
method Developed a unified theoretical framework for multi-class classification with variable supervision.
result Introduced the concept of separation to characterize learnability and generalization bounds.
Study investigates one-shot semi-supervised learning for image classification.
problem Training deep networks requires many labeled samples, limiting adoption.
method Empirical investigation of FixMatch method for one-shot semi-supervised learning.
result Uneven class accuracy is a barrier to high performance in one-shot semi-supervised learning.
This work tackles continual learning with semi-supervised data, showing that even with minimal labeled data, performance can match full-supervised methods.
problem Training deep networks on a stream of tasks without forgetting, especially when labeled data is scarce.
method Designing a novel CSSL method that leverages metric learning and consistency regularization to learn from both labeled and unlabeled data.
result Our method outperforms state-of-the-art methods trained with full supervision, achieving comparable performance with only 25% labeled data.
Paper resolves the debate on process vs. outcome supervision in reinforcement learning.
problem Distinguishing between process and outcome supervision in reinforcement learning.
method Developed a technical tool (Change of Trajectory Measure Lemma) to show equivalence between outcome and process supervision under standard data coverage assumptions.
result Reinforcement learning through outcome supervision is statistically equivalent to process supervision, up to polynomial factors in horizon.
New approach for semi-supervised learning under covariate shifts.
problem Semi-supervised learning under covariate shifts where labeled and unlabeled data distributions differ.
method Information-theoretical approach, addressing covariate shifts.
result Improved performance compared to previous methods.
Self-supervised learning improves few-shot classification and segmentation on point clouds.
problem Efficiently learn from limited labeled data in point cloud applications.
method Hierarchical cover-tree partitioning for self-supervised pre-training; restricted to support set for few-shot learning.
result Self-supervised learning significantly improves downstream classification and segmentation accuracy.
New research shows unlabeled data is equally valuable as labeled data in certain semi-supervised learning scenarios.
problem Improving learning performance with limited labeled data.
method Statistical models with continuous parameters, showing equal utility of unlabeled data under specific conditions.
result The learning rate of semi-supervised learning scales similarly to supervised learning when unlabeled data is abundant.
Self-supervision improves GCNs' generalizability and robustness.
problem Improving graph convolutional networks' performance.
method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.
result Self-supervision enhances GCNs' robustness and generalizability.
Adapts self-supervised learning using probabilistic sets with validity guarantees.
problem Lack of validity guarantees in pseudo-labels from self-supervised learning.
method Uses conformal prediction to provide validity guarantees for probabilistic labels.
result Valid probabilistic labels improve calibration and performance.
Optimal and safe semi-supervised learning estimator for high-dimensional data.
problem Improving regression parameter estimation with unlabeled data in high-dimensional settings.
method Established minimax lower bound, proposed optimal and safe semi-supervised estimators.
result Optimal semi-supervised estimator achieves the minimax lower bound.
Novel approach to contextual bandits using self-supervised learning.
problem Exploiting rich data representations in computer vision without explicit labels.
method Combining contextual bandit objective with self-supervision objective.
result Substantial gains in cumulative reward on computer vision datasets.
A method for collecting human supervision that combines rules and instance labels.
problem Lack of labeled data and inefficient human supervision.
method Rule-exemplar method with training algorithm for joint denoising and model training.
result Our algorithm is more accurate than existing methods and effectively denoises rules.
Survey on self-supervised pre-training for neural networks using unlabeled data.
problem Improving model performance using unlabeled data.
method Pre-training on unlabeled data followed by task-specific adaptation.
result Enhanced model performance through self-supervised pre-training.
Improved self-supervised learning for document images.
problem Performance of self-supervised pre-training on document images is poor.
method Proposed context-aware alternatives and a novel multi-modal method.
result Novel method outperforms other self-supervised methods on document image classification.