Improved self-supervised denoising for Poisson-Gaussian noise.
problem Handling Poisson-Gaussian noise in self-supervised denoising.
method Extended blindspot model, improved training scheme without hyperparameters.
result Improved denoising performance on microscope image benchmarks.
Paper analyzes self-supervised image denoising with denatured data.
problem Understanding the performance of self-supervised image denoising with denatured data.
method Theoretical analysis and numerical experiments on a denoising algorithm.
result Theoretical analysis shows the algorithm finds desired solutions to the optimization problem.
Self-supervised method predicts clean signal and noise distribution from noisy images.
problem Blind denoising and noise estimation in biomedical images with limited clean data.
method Two neural networks jointly predict clean signal and noise distribution from noisy observations.
result Significantly outperforms state-of-the-art algorithms on six biomedical image datasets.
New method denoises images without clean reference using Tweedie distributions.
problem Image denoising without clean reference images.
method Combining Tweedie distributions, Noise2Score, and saddle point approximation.
result General closed-form denoising formula for various noise distributions.
DiffDenoise preserves fine structures in medical images using conditional diffusion models.
problem Medical image denoising often results in loss of fine structures.
method Conditional diffusion model with stabilized reverse sampling and supervised training.
result DiffDenoise outperforms state-of-the-art methods in medical image denoising.
L2R learns to denoise images without needing noise distribution knowledge.
problem Traditional denoising methods require noise distribution knowledge, limiting their applicability.
method L2R uses a learnable monotonic neural network to learn recorruption without distribution knowledge.
result L2R achieves state-of-the-art performance across various noise distributions.
Paper proposes a self-supervised method to denoise autoregressive signals with heavy-tailed noise.
problem Denoising autoregressive signals corrupted by heavy-tailed noise.
method Self-supervised learning approach without requiring full noise distribution knowledge.
result Strong denoising performance compared to baseline methods, especially for impulsive noise.
SidAE combines autoencoders and Siamese networks for self-supervised feature extraction.
problem Efficiently extract meaningful features from unlabeled data.
method Proposes SidAE, a combination of autoencoders and Siamese networks for self-supervised image classification.
result SidAE outperforms self-supervised baselines across multiple datasets and scenarios, especially with limited labeled data.
This work uses self-supervised learning to generate better labels for financial time-series data.
problem Lack of reliable labels for financial time-series data due to noise and non-stationarity.
method Inspired by image classification, applies computer vision techniques to financial time-series data to generate denoised labels.
result Generated denoised labels improve the performance of downstream learning algorithms.
Transformer model removes noise from light curves efficiently.
problem Challenges in processing astrophysical light curves due to noise.
method Denoising Time Series Transformer (DTST) model trained with masked objective.
result DTST model excels at removing noise and outliers in time series datasets.
New method improves image denoising with fewer parameters and less data.
problem Image denoising requires large datasets and supervised settings, limiting practical applications.
method Self-supervised framework using Tucker low-rank tensor approximation.
result Improves model generalizability and reduces data acquisition costs.
Deep GNNs and self-supervision boost graph learning at scale.
problem Efficiently deploying GNNs at large scale remains challenging.
method Two large-scale GNNs: a deep transductive node classifier and a very deep inductive graph regressor.
result Award-level performance on MAG240M and PCQM4M benchmarks.
We propose a general framework for denoising high-dimensional measurements which requires no prior on the signal, no estimate of the noise, and no clean training data. The only assumption is that the noise exhibits statistical independence across different dimensions of the measurement, while the true signal exhibits s…
Recovering a high-quality image from noisy indirect measurements is an important problem with many applications. For such inverse problems, supervised deep convolutional neural network (CNN)-based denoising methods have shown strong results, but the success of these supervised methods critically depends on the availabi…
We describe a novel method for training high-quality image denoising models based on unorganized collections of corrupted images. The training does not need access to clean reference images, or explicit pairs of corrupted images, and can thus be applied in situations where such data is unacceptably expensive or impossi…
Self-supervised method enhances ultrasound images without needing clean targets.
problem Multiplicative speckle, acquisition blur, and scanner artifacts hamper ultrasound interpretation.
method Physics-guided degradation model trained on rotated/cropped patches with synthesized inputs.
result Achieves highest PSNR/SSIM across Gaussian and speckle noise levels, with significant improvements in heavy noise conditions.
G-SimCLR improves unsupervised learning by clustering images into pseudo labels.
problem Improving unsupervised learning for image recognition.
method Proposes a method to cluster images into pseudo labels to batch images of the same category.
result Comparable performance enhancements on CIFAR10 and ImageNet datasets.
Enhances neural networks' robustness against adversarial samples without sacrificing clean sample generalization.
problem Limited generalization and time complexity of adversarial training.
method Feature Pyramid Decoder (FPD) framework that integrates denoising and image restoration modules into CNNs and constrains the Lipschitz constant.
result FPD-enhanced CNNs achieve sufficient robustness against general adversarial samples on various datasets.
DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.
problem Efficiently decomposing tensors with deep learning to capture nonlinear structures.
method Low-rank tensor decomposition using deep generative networks trained to minimize approximation error.
result DeepTensor outperforms classical methods like SVD and PCA in various applications, including image denoising and 3D MRI.
S4 learns new self-supervision automatically, improving accuracy with less human effort.
problem Lack of direct supervision in machine learning.
method Combines deep learning and probabilistic logic to automatically generate and verify new self-supervision.
result S4 can automatically propose accurate self-supervision, matching supervised methods with less human effort.
Self-supervision improves GCNs' generalizability and robustness.
problem Improving graph convolutional networks' performance.
method Three mechanisms of self-supervision, multi-task learning, and graph adversarial training.
result Self-supervision enhances GCNs' robustness and generalizability.
Self-supervised learning excels in topic modeling by being less model-specific.
problem How self-supervised learning discovers useful representations in topic models.
method Applying self-supervised learning objectives to topic model-generated data.
result Self-supervised learning objectives can recover useful posterior information for topic models, outperforming misspecified models.
We consider the effect of structure-agnostic and structure-dependent masking schemes when training a universal marginaliser (arXiv:1711.00695) in order to learn conditional distributions of the form P(xi∣xb), where xi is a given random variable and xb is some arbitrary sub…
Improved self-supervised learning for document images.
problem Performance of self-supervised pre-training on document images is poor.
method Proposed context-aware alternatives and a novel multi-modal method.
result Novel method outperforms other self-supervised methods on document image classification.
Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating or reducing the need for annotations. We find that self-supervision can benefit robustness in a varie…
Survey of self-supervised learning methods in computer vision, NLP, and graph learning.
problem Manual labeling and vulnerability to attacks in supervised learning.
method Generative, contrastive, and generative-contrastive approaches.
result Self-supervised learning achieves high performance in representation learning.
This work improves adaptive conformal prediction using self-supervised learning.
problem Improving the adaptability of conformal prediction intervals.
method Train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores.
result Empirically demonstrates the benefit of additional information in improving the efficiency (width), deficit, and excess of conformal prediction intervals.
This paper analyzes self-supervised learning from a multi-view perspective.
problem Understanding and optimizing self-supervised learning from multi-view data.
method Information-theoretical framework to understand and design self-supervised learning objectives.
result Self-supervised representations can extract task-relevant information and discard task-irrelevant information.
Generative model explains self-supervised learning across various tasks.
problem Lack of theoretical understanding of self-supervised learning methods.
method Generative latent variable model for self-supervised learning.
result Improves representation learning performance and narrows the gap between generative and discriminative methods.
Novel approach to contextual bandits using self-supervised learning.
problem Exploiting rich data representations in computer vision without explicit labels.
method Combining contextual bandit objective with self-supervision objective.
result Substantial gains in cumulative reward on computer vision datasets.
Paper tackles self-supervised learning for non-homophilous graphs.
problem Existing self-supervised learning methods assume homophilous graphs, but real-world graphs often lack this assumption.
method Develops a decoupled self-supervised learning (DSSL) framework that decouples different semantics between neighborhoods.
result DSSL framework achieves better performance on various graph benchmarks compared to competitive baselines.
We propose the first qualitative hypothesis characterizing the behavior of visual transformation based self-supervision, called the VTSS hypothesis. Given a dataset upon which a self-supervised task is performed while predicting instantiations of a transformation, the hypothesis states that if the predicted instantiati…
Deep nearest neighbors outperform self-supervised methods in anomaly detection.
problem Anomaly detection using self-supervised deep methods.
method Simple nearest-neighbor approach on Imagenet pretrained features.
result Nearest-neighbor method outperforms self-supervised methods in accuracy, few shot generalization, training time, and noise robustness.
We present a technique to improve the transferability of deep representations learned on small labeled datasets by introducing self-supervised tasks as auxiliary loss functions. While recent approaches for self-supervised learning have shown the benefits of training on large unlabeled datasets, we find improvements in …
We describe a novel weakly labeled Audio Event Classification approach based on a self-supervised attention model. The weakly labeled framework is used to eliminate the need for expensive data labeling procedure and self-supervised attention is deployed to help a model distinguish between relevant and irrelevant parts …
SASSL improves self-supervised learning by preserving image structure.
problem Distorted augmented samples in self-supervised learning.
method Neural Style Transfer to decouple semantic and stylistic attributes.
result Boosts ImageNet top-1 accuracy by up to 2 percentage points.
DM improves self-supervised transfer learning by matching target distributions.
problem Improving self-supervised transfer learning performance.
method Distribution Matching (DM) method that drives representation distribution towards a predefined reference distribution.
result DM outperforms existing methods on target classification tasks.
Improved self-supervised learning on ImageNet achieves top-1 accuracy of 77.1%.
problem Self-supervised ResNets underperform supervised learning on ImageNet.
method ReLICv2 combines explicit invariance loss with contrastive objective over varied data views.
result ReLICv2 achieves 77.1% top-1 accuracy on ImageNet, improving over previous state-of-the-art by 1.5%.
Unified framework improves deep multi-view clustering by addressing self-supervision and contrastive alignment issues.
problem Variations in self-supervision-based methods for deep multi-view clustering.
method Unified DeepMVC framework that includes recent methods and leverages self-supervision and contrastive alignment.
result Contrastive alignment negatively impacts cluster separability, especially with many views.
Paper compares optimal denoising methods for generative models, finding different results based on data regularity.
problem Optimizing denoising in score-based generative models for various data types.
method Comparison of full-denoising and half-denoising approaches, analyzing performance in terms of distribution distances.
result Different denoising methods perform better under different data regularity conditions.
Plan2Explore learns new tasks efficiently through self-supervised planning.
problem Challenges in reinforcement learning, especially task-specific learning and sample efficiency.
method Self-supervised exploration and fast adaptation to new tasks through efficient planning.
result Plan2Explore outperforms prior methods in learning new tasks without supervision.
Survey on self-supervised pre-training for neural networks using unlabeled data.
problem Improving model performance using unlabeled data.
method Pre-training on unlabeled data followed by task-specific adaptation.
result Enhanced model performance through self-supervised pre-training.
This paper reviews self-supervised learning methods for time series anomaly detection.
problem Challenges in traditional unsupervised methods for time series anomaly detection.
method Self-supervised learning techniques for time series anomaly detection.
result Enhanced performance of anomaly detectors through self-supervised learning.
SSNAS finds neural architectures without labeled data.
problem Limited labeled data for NAS.
method Self-supervised learning for NAS.
result Comparable results to supervised NAS with labeled data.
Self-supervised learning improves ECG classification performance.
problem Label scarcity in clinical 12-lead ECG data.
method Adapted self-supervised methods to ECG domain, focusing on contrastive representations and latent forecasting.
result Contrastive predictive coding adaptation yields linear evaluation performance only 0.5% below supervised performance.
New measure quantifies contrastive self-supervised learning's generalization ability.
problem Limited theoretical understanding of contrastive self-supervised learning's generalization.
method Defined (σ,δ)-measure to mathematically quantify data augmentation and provide an upper bound for downstream classification error. result Generalization ability is related to alignment of positive samples, divergence of class centers, and concentration of augmented data.
New framework analyzes why more negative samples improve self-supervised learning performance.
problem Inconsistency between theoretical degradation and empirical improvement of downstream supervised tasks with more negative samples.
method Coupon collector's problem framework to analyze self-supervised representation learning with more negative samples.
result Bound can implicitly incorporate supervised loss in self-supervised loss by increasing negative samples.
Self-supervised pretraining for heterogeneous hypergraphs improves graph-based tasks.
problem Capturing higher-order relations in heterogeneous hypergraphs.
method SPHH framework for self-supervised pretraining of heterogeneous HyperGNNs.
result SPHH consistently outperforms state-of-the-art baselines in various downstream tasks.