The authors discuss reproducibility in pattern recognition research.
problem Making pattern recognition research fully reproducible.
method Defined reproducibility, used tools, set up analysis, showed alternative analyses.
result Discussion on the trade-offs and benefits of reproducible research.
PixelHop uses SSL for image classification, outperforming CNN.
problem Image classification accuracy improvement.
method PixelHop combines SSL with supervised and unsupervised dimension reduction.
result PixelHop outperforms CNN on MNIST, Fashion MNIST, and CIFAR-10 datasets.
We explain SSL objectives as log-likelihoods in a data curation model.
problem Lack of understanding of SSL objectives as log-likelihoods.
method Formulate SSL objectives as a log-likelihood in a generative model of data curation.
result SSL methods can be understood as lower-bounds on a principled log-likelihood.
Paper analyzes how unlabeled data improves SSL and adversarial robustness.
problem Understanding how unlabeled data impacts SSL and adversarial robustness.
method Minimax analysis and adversarial training.
result Reconstruction-based SSL algorithm is rate-optimal under various models and enhances adversarial robustness.
Semi-supervised learning benefits the rich more than the poor, affecting fairness.
problem Disparate impact of semi-supervised learning on different sub-populations.
method Theoretical and empirical analysis of a broad family of SSL algorithms using pseudo-labels.
result Semi-supervised learning benefits the rich more than the poor, potentially violating fairness.
Unified evaluation of SSL methods reveals real-world performance issues.
problem Real-world SSL performance issues not captured by benchmarks.
method Unified reimplementation and testing of SSL techniques.
result Simple baselines often outperform SSL methods, and performance degrades with out-of-class examples.
Researchers propose a new SSL risk decomposition method to evaluate and improve self-supervised learning models.
problem Self-supervised learning evaluation is limited to a single metric, providing little insight into model performance and improvement.
method Proposes an SSL risk decomposition that considers four error components: approximation, representation usability, probe generalization, and encoder generalization.
result Analysis of 169 SSL vision models reveals the main sources of error and provides insights for improving SSL models in specific settings.
New theory explains how self-supervised learning converges, advancing AI research.
problem Lack of precise theoretical explanation for self-supervised learning convergence.
method Synthesized Identifiability Theory with empirical evidence to propose Singular Identifiability Theory (SITh).
result SITh provides deeper insights into SSL's implicit data assumptions and advances representation learning.
RealMix improves SSL performance with real-world data challenges.
problem Poor SSL performance when labeled and unlabeled data distributions differ.
method Develops RealMix, a new SSL algorithm.
result Achieves state-of-the-art results on standard benchmark datasets.
A new SSL method improves medical image classification using global latent mixing.
problem Costly annotation of large-scale medical image data sets.
method Linear mixing of labeled and unlabeled data in both input and latent space.
result Improved performance in semi-supervised classification of thoracic disease and skin lesion.
Maximizes image representation dependence for self-supervised learning.
problem Learning meaningful image representations from unlabeled data.
method Maximizes Hilbert-Schmidt Independence Criterion (HSIC) between image transformations and identity.
result Matches state-of-the-art performance on ImageNet and other vision tasks.
Generative model creates synthetic unlabeled data for SSL.
problem Training SSL models without real unlabeled datasets.
method Meta-optimized synthetic samples generated from generative models.
result Synthetic samples improve SSL performance more efficiently than real unlabeled data.
DoubleMatch combines pseudo-labeling with self-supervision for SSL.
problem Lack of effective use of unlabeled data in SSL.
method Combines pseudo-labeling with self-supervised loss.
result Achieves state-of-the-art accuracies on multiple datasets.
Semi-supervised learning debiased for better performance.
problem Bias in SSL methods impairs theoretical guarantees and empirical performance.
method Simple debiasing approach to remove risk estimate bias.
result Debiased SSL methods provide better calibrated models.
A new framework decouples SSL tasks into VDA and VLC, revealing VDA's importance.
problem Designing effective self-supervised learning tasks without manual annotation.
method Borrowing a multi-view perspective, the paper decouples popular pretext tasks into VDA and VLC, focusing on VDA's role in feature learning.
result VDA tasks dominate SSL performance, and integrating predictions from augmented views improves overall performance.
Solves biased pseudo-labels in imbalanced SSL by refining them.
problem Imbalanced class distributions in semi-supervised learning lead to biased pseudo-labels.
method Formulates a convex optimization problem to refine pseudo-labels and develops an efficient algorithm, DARP.
result Demonstrates the effectiveness of DARP in various imbalanced semi-supervised scenarios.
Flaky performance found in GNN SSL on RDBs, leading to worse linear evaluation.
problem Downstream task performances of GNN SSL on RDBs are poor.
method Proposed InfoNode to maximize mutual information between initial and final node representations.
result InfoNode improves GNN SSL performance on RDBs, supporting conjecture of conflict between SSL and GNN message passing.
MTSSL optimizes threshold τ for better semi-supervised learning performance.
problem Optimizing the threshold τ for effective semi-supervised learning.
method Meta-Thresholding approach to optimize τ during training.
result Optimal values of τ are not necessary for achieving similar performance.
This work defines idealized SSL representations and improves existing methods.
problem Unclear characteristics of SSL representations leading to high downstream accuracies.
method Characterized ideal properties and derived necessary and sufficient conditions.
result Improved SSL methods and derived new objectives for contrastive and non-contrastive learning.
MET learns tabular data representations without data augmentations.
problem Lack of effective self-supervised learning methods for tabular data.
method Reconstruction-based approach using masked encoding, with separate representations for each coordinate and adversarial reconstruction loss.
result MET achieves state-of-the-art performance on five diverse tabular datasets, improving up to 9% over current methods.
SSL theory improves representation learning from raw data.
problem Challenges in SSL, including instability and collapse.
method Precise analysis of generalization performance with a theory-friendly setup.
result Insights for SSL practitioners on data augmentation, network architecture, and training algorithm.
Computer vision SSL methods show effectiveness on time series data.
problem Evaluate if computer vision SSL frameworks are effective on time series data.
method Evaluated on UCR and UEA archives, proposed a new method improving VICReg.
result Computer vision SSL frameworks can be effective on time series data.
Semi-supervised learning improves classification in high dimensions.
problem Combining labeled and unlabeled data for high-dimensional classification.
method Information theoretic and computational lower bounds analysis for feature selection.
result Semi-supervised learning is advantageous for classification in high dimensions.
This paper explores SSL for graph neural networks, improving performance on real-world datasets.
problem Leveraging unlabeled data for graph neural networks to improve deep learning performance.
method Empirical study of various SSL pretext tasks on graphs and proposing a new approach called SelfTask.
result Proposes SelfTask, achieving state-of-the-art performance on real-world datasets.
A new probabilistic model for semi-supervised learning unifies various methods.
problem Combining different aspects of data distribution for semi-supervised learning.
method A probabilistic model that interprets and improves upon existing SSL methods.
result The model unifies various SSL methods and extends to neuro-symbolic learning.
Paper proves supermodularity of AG-SSL objective and proposes a greedy sampling algorithm.
problem Improving semi-supervised learning with limited labeled data.
method Proves supermodularity of AG-SSL objective under Stieltjes regularization and proposes a greedy sampling algorithm.
result Proposed method achieves superior classification accuracy compared to state-of-the-art methods.
The study analyzes consistency-based SSL methods and proposes a new framework.
problem Improving performance in semi-supervised learning with minimal labeled data.
method Analysis of consistency-based SSL methods, including the Π-model, and proposing a new framework. result A simple extension of the Hidden Manifold Model that incorporates data-augmentation schemes.
From a machine learning perspective, the human ability localize sounds can be modeled as a non-parametric and non-linear regression problem between binaural spectral features of sound received at the ears (input) and their sound-source directions (output). The input features can be summarized in terms of the individual…
Declarative entropy constraints improve semi-supervised learning.
problem Improving semi-supervised learning performance.
method Declarative specification of entropy constraints for semi-supervised learning.
result Consistent improvements on SSL benchmarks, including a new state-of-the-art result.
Meta-Semi learns to optimize SSL with minimal hyper-parameter tuning.
problem Limited labeled data in SSL makes it impractical to tune many hyper-parameters.
method Meta-learning approach that dynamically reweights unlabeled data loss.
result Meta-Semi achieves competitive performance on various SSL tasks.
SSL framework identifies non-linear systems without labeled data.
problem System identification in non-linear environments without labeled data.
method Dynamics contrastive learning framework.
result SSL can identify non-linear dynamics in latent space.
Paper tackles class-imbalanced semi-supervised learning, improving performance.
problem Class imbalance in semi-supervised learning datasets.
method Proposes Suppressed Consistency Loss (SCL) to handle class imbalance.
result Method shows better performance in class-imbalanced semi-supervised learning environments.
GAN-based semi-supervised learning improves classifier generalization.
problem Improving classifier performance with limited labeled data.
method Theoretical analysis of GAN-SSL, proving equivalence of discriminator optimization and supervised learning, and exploring conditions for perfect discriminator.
result GAN-SSL theoretically outputs a perfect discriminator on both labeled and unlabeled data.
Expanding on SSL, this study considers predicting outcomes from both causes and effects.
problem Understanding the limits and possibilities of semi-supervised learning.
method Explores the conditional distribution of effect features given causal features for semi-supervised learning.
result Shows that semi-supervised learning can be effective when predicting outcomes from both causes and effects.
Unified framework interprets SSL models, revealing biases.
problem Interpreting black-box SSL models for transparency.
method KREPES framework using Representer Landmarks and Nyström approximation.
result Direct audit of SSL latent space without supervision.
Self-supervised learning performs better than supervised learning on imbalanced datasets.
problem The performance gap between balanced and imbalanced pre-training with self-supervised learning is smaller than with supervised learning.
method Systematic investigation of self-supervised learning under dataset imbalance, including experiments and theoretical analyses.
result Self-supervised representations are more robust to class imbalance than supervised representations.
Enhances SSL methods with depth cues for better image understanding.
problem Lack of depth cues in 2D image pixel maps limits SSL performance.
method Integrates depth signals from a pretrained monocular RGB-to-depth model into contrastive learning frameworks.
result Improves SSL methods' robustness and generalization with depth signals.
MaxMatch improves SSL with worst-case consistency for better generalization.
problem Efficiently supervised learning with unlabeled data.
method Worst-case consistency regularization for SSL, providing a bound and an algorithm.
result The proposed method converges to a stationary point and improves generalization.
RegMixMatch optimizes Mixup for semi-supervised learning by integrating high- and low-confidence samples.
problem Mixup degrades SSL performance by compromising artificial labels purity.
method RegMixMatch integrates high- and low-confidence samples, uses class-aware Mixup, and mitigates confirmation bias.
result RegMixMatch achieves state-of-the-art performance in SSL benchmarks.
New theory explains how equivariant self-supervised learning improves feature extraction.
problem Contrastive learning sacrifices useful features due to invariance to data augmentations.
method Information-theoretic perspective to understand E-SSL's generalization ability.
result Equivariant self-supervised learning creates synergy between equivariant and classification tasks.
Paper analyzes high-dimensional GMM for SSL, showing RMLE near-optimal performance.
problem Understanding when and why SSL works well with unlabeled data.
method Message-passing and state evolution methods for high-dimensional settings.
result RMLE achieves near-optimal performance in estimation and prediction with appropriate regularization.
Proposes SSL method for non-randomly sampled data.
problem Evaluation of prediction rules under non-random sampling.
method Two-step procedure with imputation and augmentation.
result Proposed method outperforms supervised methods in efficiency.
This work improves SSL by leveraging disentangled latent space for better self-ensembling.
problem Improving semi-supervised learning performance with limited labeled data.
method Stacked SSL model using unsupervised disentangled representation learning for stochastic embedding.
result Improved performance and interpretability of disentangled representations over related SSL models.
Survey of techniques for learning with small samples in big data.
problem Learning from limited data in big data environments.
method Divided into concept learning and experience learning.
result Clarifies the rationality of SSL and its relationship with human learning.
A new SSL method using DRO with unlabeled data to improve generalization.
problem Improving SSL performance with limited labeled data.
method Distributionally Robust Optimization (DRO) with optimal transport metrics and stochastic gradient descent.
result Our method significantly reduces generalization error compared to supervised and SSL methods.
A new method MixGDA combines mixup and gradient-based data augmentation for SSL.
problem Improving semi-supervised learning performance with limited labeled data.
method Gradient-based Data Augmentation (GDA) combined with mixup methods.
result MixGDA achieves state-of-the-art performance in various SSL benchmarks.
Paper improves deep learning models using contrastive predictive coding for semi-supervised learning.
problem Limited labeled data in semi-supervised learning.
method Contrastive predictive coding technique to improve deep learning models with unlabeled data.
result Proposed cpc-SSL and ccpc-SSL models effectively use unlabeled data, scaling well to large datasets.
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.