The paper proposes a model to learn disentangled representations using mutual information.
problem Learning disentangled representations from shared and exclusive attributes.
method Mutual information maximization for shared attributes and minimization for disentanglement.
result The proposed model outperforms state-of-the-art models in representation disentanglement.
A method to prevent image representation collapse through data-dependent augmentation.
problem Representation collapse due to image augmentations that damage information.
method Formalizing a stochastic encoding process with a tug-of-war between corruption and preserved information, using infoMax objective.
result Learning a data-dependent distribution of augmentations to avoid representation collapse.
SketchEmbedNet learns image representations from sketches, useful for few-shot learning.
problem Learning image representations from sketches for few-shot learning.
method Training a model to produce sketches of images, focusing on informative embeddings.
result Model produces informative embeddings of novel images, classes, and datasets.
Survey of self-supervised image representation learning methods.
problem Learning meaningful image representations without labeled data.
method Comprehensive review and taxonomy of self-supervised methods.
result Quality of self-supervised representations is comparable to supervised learning.
Adversarially trained generative models (GANs) have recently achieved compelling image synthesis results. But despite early successes in using GANs for unsupervised representation learning, they have since been superseded by approaches based on self-supervision. In this work we show that progress in image generation qu…
Reduces GAN image priors' representation error using a Deep Decoder.
problem Representation error in GAN priors for in-distribution and out-of-distribution images.
method Hybrid model combining GAN prior and Deep Decoder.
result Consistently higher PSNRs on in-distribution and out-of-distribution images.
CLIP learns joint image-text representations for zero-shot learning.
problem Understanding and improving zero-shot transfer performance in CLIP.
method Formal study of transferrable representation learning and analysis of zero-shot transfer performance.
result Proposes a new CLIP-type approach that outperforms existing methods.
Deep learning method improves gene ontology classification of neural images.
problem Classifying gene expression in neural in situ hybridization images.
method End-to-end deep learning using convolutional denoising autoencoders (CDAE).
result Significant improvement in classification accuracy (96% reduction in error rate).
New text-to-image diffusion models improve scene understanding for AI agents.
problem Fine-grained scene understanding for AI agents from text and images.
method Pre-trained text-to-image diffusion models optimized for generating images from text prompts.
result Policies learned with Stable Control Representations outperform state-of-the-art approaches on various control tasks.
A new model decouples global and local image representations without supervision.
problem Learning decoupled global and local image representations without supervision.
method Variational auto-encoding framework with invertible generative flow.
result The model effectively learns decoupled representations of images.
This paper presents a novel deep learning-based method for learning a functional representation of mammalian neural images. The method uses a deep convolutional denoising autoencoder (CDAE) for generating an invariant, compact representation of in situ hybridization (ISH) images. While most existing methods for bio-ima…
A novel feature representation method for non-image based features.
problem Inability of Convolutional Neural Networks for non-image based features or features without spatial correlations.
method REFINED: Representation of Features as Images with Neighborhood Dependencies.
result Higher prediction accuracy compared to existing methodologies.
Study reveals differences in medical image models' hidden representation refinement.
problem Understanding how intrinsic dimensionality changes in neural network hidden representations across different domains.
method Analysis of 11 natural and medical image datasets using 6 network architectures.
result Medical image models refine hidden representations earlier, suggesting differences in feature abstraction.
A new method, REC, compresses images by encoding their latent representations efficiently.
problem Efficiently compressing single images with latent representations.
method Relative Entropy Coding (REC) that directly encodes latent representations with codelength close to relative entropy.
result REC is more efficient for single image compression compared to previous methods and is competitive for lossy compression.
Improved VAE representations lead to better image classification.
problem VAE representations are inferior to non-latent models for image classification.
method Used a decoder that prefers local features, improving global feature capture in latent variables.
result Significant improvement in downstream semantic classification tasks.
Pix2Shape learns 3D scene representations from single images without supervision.
problem Learning 3D scene information from a single image without supervision.
method Pix2Shape uses an encoder, decoder, and critic network to generate 2.5D surfel-based reconstructions.
result Pix2Shape can generate complex 3D scenes from a single image, scaling with on-screen resolution.
Plan2Vec learns image representations without labels, improving control tasks.
problem Learning image representations without labeled data.
method Constructs a weighted graph using near-neighbor distances and extrapolates to global embedding.
result Plan2Vec achieves accurate long-term value estimates in control tasks with reduced computational and memory costs.
Sparse learning has been shown to be effective in solving many real-world problems. Finding sparse representations is a fundamentally important topic in many fields of science including signal processing, computer vision, genome study and medical imaging. One important issue in applying sparse representation is to find…
A simple method flags images as out-of-distribution based on their distance to nearest neighbors.
problem Detecting images not aligned with a trained model's in-distribution data.
method Flag images as OOD if their average distance to K nearest neighbors is large in the classifier's representation space.
result Simple methods can outperform more complex ones when considering learned representations.
This work proposes a model to disentangle image factors effectively and control their manipulation.
problem Controlling disentanglement during image editing while preserving object identity.
method Encoder-decoder architecture with decorrelation regularization and soft target representations.
result The model successfully disentangles image factors and manipulates them effectively.
Proposes a new contrastive loss for semi-supervised medical image segmentation.
problem Lack of labeled data for medical image segmentation.
method Uses pseudo-labels and a local contrastive loss to learn good local representations.
result Achieved high segmentation performance on public cardiac and prostate datasets.
A parallel algorithm learns efficient Kronecker product dictionaries.
problem Sparse representation of 2D signals like images and hyperspectral data.
method Highly parallelizable algorithm for learning separable dictionaries.
result Competitive sparse representations at lower computational cost.
BYOL learns useful representations without batch statistics.
problem BYOL's reliance on batch statistics for representation learning.
method Training BYOL without batch normalization and using alternative normalization schemes.
result BYOL performance comparable to vanilla BYOL without batch normalization.
Maximizes image representation dependence for self-supervised learning.
problem Learning meaningful image representations from unlabeled data.
method Maximizes Hilbert-Schmidt Independence Criterion (HSIC) between image transformations and identity.
result Matches state-of-the-art performance on ImageNet and other vision tasks.
CDPL-Net integrates CNN and DL for better image representation.
problem Improving image representation learning by combining CNN and DL.
method CDPL-Net architecture combining CNN and DPL layers, using l1-norm for sparse representation, and efficient stochastic gradient descent.
result Enhanced performance compared to state-of-the-art methods.
Study finds CLIP's caption-based learning outperforms image-only methods under certain conditions.
problem Comparing CLIP's performance with traditional image-only methods in learning transferable representations.
method Controlled comparison of CLIP and image-only methods using a dataset with descriptive captions and specific criteria.
result CLIP's caption-based learning outperforms image-only methods when certain conditions are met, but can be detrimental in others.
Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from …
Improved visual representation learning with conditional negative sampling.
problem Learning strong unsupervised visual representations using contrastive learning.
method Introduce a family of mutual information estimators that sample negatives conditionally.
result Improves accuracy by 2-5% points on four standard image datasets.
New method learns useful disentangled representations from weakly labeled data.
problem Learning useful representations from weakly labeled data.
method Model pairs of non-i.i.d. images, learn disentangled representations without requiring annotation.
result Learn disentangled representations reliably from pairs of images without requiring group, individual factor, or number of changed factors annotation.
Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale coherence. To address this, we propose two methods to learn discrete representations of images which abstract away local detail. We show th…
In this paper we propose a strategy for semi-supervised image classification that leverages unsupervised representation learning and co-training. The strategy, that is called CURL from Co-trained Unsupervised Representation Learning, iteratively builds two classifiers on two different views of the data. The two views c…
BYOL learns image representations without negative pairs.
problem Self-supervised image representation learning.
method Two neural networks interact and learn from each other, updating the target network with a slow-moving average of the online network.
result Achieves state-of-the-art performance on ImageNet.
Unified principle LZN unifies generative modeling, representation learning, and classification.
problem Disjoint solutions for generative modeling, representation learning, and classification.
method LZN creates a shared latent space for all tasks, using encoders and decoders for each data type.
result LZN improves FID on CIFAR10 and outperforms existing methods in representation learning and joint generation/classification.
Learning representations with diversified information remains as an open problem. Towards learning diversified representations, a new approach, termed Information Competing Process (ICP), is proposed in this paper. Aiming to enrich the information carried by feature representations, ICP separates a representation into …
One of the biggest challenges for deep learning algorithms in medical image analysis is the indiscriminate mixing of image properties, e.g. artifacts and anatomy. These entangled image properties lead to a semantically redundant feature encoding for the relevant task and thus lead to poor generalization of deep learnin…
Local differential privacy method protects image privacy in machine learning.
problem Protecting image privacy in machine learning models from data owners.
method Locally perturbs image representations before releasing them to the data user.
result Proposes DCAConv, an image feature extractor that maintains high data utility while preserving privacy.
Enhances visual localization using graph smoothing.
problem Inferring camera pose from a single image.
method Constructs a graph based on GPS coordinates and temporal information, then smooths feature representations.
result Significantly improves localization accuracy on large datasets.
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoid…
Generative Adversarial Networks (GAN) have demonstrated impressive results in modeling the distribution of natural images, learning latent representations that capture semantic variations in an unsupervised basis. Beyond the generation of novel samples, it is of special interest to exploit the ability of the GAN genera…
Proposes a new metric learning method for image recognition.
problem Improving image recognition performance using learned distance representations.
method Introduces a Generalized Hybrid Metric Loss (GHM-Loss) to learn hybrid proximity features combining geometric and probabilistic spaces.
result Demonstrates superior performance compared to existing methods on public datasets.
Image generating neural networks are mostly viewed as black boxes, where any change in the input can have a number of globally effective changes on the output. In this work, we propose a method for learning disentangled representations to allow for localized image manipulations. We use face images as our example of cho…
A new image interpolation model using sparse representation and nonlocal linear regression.
problem Image interpolation without blurring and noise.
method Sparse representation, nonlocal self-similarity, nonlocal linear regression, adaptive sub-dictionary learning, weighted encoding.
result Our method outperforms state-of-the-art methods in quantitative measures and visual quality.
Sparse representations using overcomplete dictionaries have proved to be a powerful tool in many signal processing applications such as denoising, super-resolution, inpainting, compression or classification. The sparsity of the representation very much depends on how well the dictionary is adapted to the data at hand. …
Enhances image captioning with novel context combination methods.
problem Improving machine learning for image captioning with structured learning and meaningful interpretation.
method Combines Feature Distribution Composition (FDC), Multiple Role Representation Crossover (MRRC) attention layers, and language decoder.
result Significantly improved image captioning performance (35.3%) and established new standards.
This study investigates how much knowledge from natural images can be transferred to pathology images.
problem Quantifying how much knowledge from natural images can be transferred to pathology images.
method Proposes a framework to quantify knowledge gain by a particular layer, conducts empirical investigation in pathology image centered transfer learning.
result Early layers of deep models can transfer knowledge to pathology image classification tasks.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
Improved RL for grasping in cluttered scenes using state representation learning.
problem Poor performance of RL methods in grasping diverse objects from raw images.
method Employed state representation learning (SRL) with disentanglement of raw input images.
result Deep RL can learn grasping skills from varied visual inputs.
The paper introduces multimodal generative models to improve data marginal likelihood.
problem Improving data marginal likelihood in multimodal settings.
method Derives variational bounds on the evidence for multimodal deep generative models, generalizes objectives for different model types, and benchmarks across various datasets.
result Multimodal VAEs excel in image, label, and text datasets with and without weak supervision.