This paper introduces a method to select and weight pretext tasks for better self-supervised speech representation learning.
problem Combining pretext tasks for better performance in self-supervised speech representation learning.
method Estimating calibrated weights for partial losses corresponding to pretext tasks during self-supervised training.
result Groups of selected and weighted pretext tasks perform better than classic baselines in automatic speech recognition and speaker/emotion recognition.
Enhanced rotation prediction improves SSL models by capturing both shape and texture information.
problem Rotation prediction misses texture information, limiting model performance.
method Introduces image enhanced rotation prediction (IE-Rot) that combines rotation and image enhancement tasks.
result IE-Rot models outperform Rotation on various benchmarks.
Self-supervised learning improves by predicting known information, reducing labeled data needs.
problem Efficiently learn useful semantic representations without labeled data.
method Develops a mechanism exploiting statistical connections between pretext tasks to learn representations that solve downstream tasks.
result Proves linear layer yields small approximation error and drastically reduces labeled sample complexity.
A new framework decouples SSL tasks into VDA and VLC, revealing VDA's importance.
problem Designing effective self-supervised learning tasks without manual annotation.
method Borrowing a multi-view perspective, the paper decouples popular pretext tasks into VDA and VLC, focusing on VDA's role in feature learning.
result VDA tasks dominate SSL performance, and integrating predictions from augmented views improves overall performance.
This work improves adaptive conformal prediction using self-supervised learning.
problem Improving the adaptability of conformal prediction intervals.
method Train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores.
result Empirically demonstrates the benefit of additional information in improving the efficiency (width), deficit, and excess of conformal prediction intervals.
This paper explores SSL for graph neural networks, improving performance on real-world datasets.
problem Leveraging unlabeled data for graph neural networks to improve deep learning performance.
method Empirical study of various SSL pretext tasks on graphs and proposing a new approach called SelfTask.
result Proposes SelfTask, achieving state-of-the-art performance on real-world datasets.
Self-supervised ECG learning improves emotion recognition.
problem Improving emotion recognition from ECG signals.
method Multi-task deep learning framework with signal transformations as pretext tasks.
result Significant performance improvement in emotion classification.
SuNCEt accelerates contrastive learning with minimal labeled data.
problem Efficiently learning visual representations with limited labeled data.
method Noise-contrastive estimation and neighbourhood component analysis-based semi-supervised loss.
result SuNCEt achieves semi-supervised learning accuracy with less than half the labeled data.
This paper presents a novel yet intuitive approach to unsupervised feature learning. Inspired by the human visual system, we explore whether low-level motion-based grouping cues can be used to learn an effective visual representation. Specifically, we use unsupervised motion-based segmentation on videos to obtain segme…
Expanding self-supervised learning to diverse domains reveals Rotation's semantic superiority.
problem Limited self-supervised learning experiments on diverse domains.
method Experimented on various domains (satellite, textural, biological) using popular self-supervised methods.
result Rotation task is semantically most meaningful, with other tasks relying on distribution rather than semantic understanding.
Theoretical analysis shows pretext-based self-supervised learning can be boosted by downstream data under certain conditions.
problem Theoretical analysis of pretext-based self-supervised learning and downstream data refinement.
method Theoretical analysis and experiments on synthetic and real-world datasets.
result Theoretical lower bounds and experiments show that downstream data refinement can boost or hurt performance depending on conditions.
Novel method reduces radiomic data annotation needs.
problem Insufficient labeled radiomic data for disease diagnosis.
method Collaborative self-supervised learning with two pretext tasks.
result Outperforms other self-supervised methods on radiomic data.
Perception technologies in Autonomous Driving are experiencing their golden age due to the advances in Deep Learning. Yet, most of these systems rely on the semantically rich information of RGB images. Deep Learning solutions applied to the data of other sensors typically mounted on autonomous cars (e.g. lidars or rada…
Unified framework for semi-supervised learning reduces annotation needs.
problem Sparse annotations and large amounts of unlabeled data in computational pathology.
method S5CL integrates fully-supervised, self-supervised, and semi-supervised learning through hierarchical contrastive losses.
result S5CL improves accuracy and F1-score in histopathological datasets with sparse labels.
EQ-Net combines LLR estimation and quantization using deep learning.
problem Unified solution for LLR estimation and quantization.
method Two-stage algorithm using LLR compression as a pretext task.
result Achieves state-of-the-art results with gains in efficiency and latency.
Paper tackles singularity detection in PDEs using data-driven self-supervised learning.
problem Detecting singularities in PDE solutions for efficient numerical methods.
method Data-driven self-supervised learning framework with filtering tasks.
result Proposes filtering methods for raw unlabeled data to improve singularity detection.
CRATE-MAE learns structured representations from unlabeled data.
problem Learning structured representations from unlabeled data.
method Structured Diffusion with White-Box Transformers.
result CRATE-MAE achieves highly promising performance on large-scale imagery datasets.
This study predicts parking availability using multi-source data and a self-supervised learning enhanced transformer.
problem Accurate parking availability prediction to support urban planning and management.
method Proposes SST-iTransformer, a self-supervised learning enhanced spatio-temporal inverted transformer, integrating multi-source data.
result SST-iTransformer achieves state-of-the-art performance in parking availability prediction.
This work uses self-supervised learning to generate better labels for financial time-series data.
problem Lack of reliable labels for financial time-series data due to noise and non-stationarity.
method Inspired by image classification, applies computer vision techniques to financial time-series data to generate denoised labels.
result Generated denoised labels improve the performance of downstream learning algorithms.