SemGANs generate pixel-level accurate semantic images.
problem Generating semantic images with pixel-level accuracy.
method Semantic Generative Adversarial Networks (SemGANs).
result SemGANs outperform standard GANs in semantic image generation tasks.
New diffusion models improve counterfactual image generation with semantic control.
problem Challenges in preserving identity, maintaining quality, and ensuring causal model faithfulness in counterfactual image generation.
method Integrates semantic representations into diffusion models through Pearlian causality, introducing spatial, semantic, and dynamic abduction.
result Demonstrates high-level semantic identity preservation and principled trade-offs between faithful causal control and identity preservation.
Semantic inpainting is the task of inferring missing pixels in an image given surrounding pixels and high level image semantics. Most semantic inpainting algorithms are deterministic: given an image with missing regions, a single inpainted image is generated. However, there are often several plausible inpaintings for a…
This work provides uncertainty intervals for semantic latent variables in disentangled latent spaces.
problem Challenges in providing meaningful uncertainty quantification for semantic information in disentangled latent spaces.
method Uses quantile regression to output heuristic uncertainty intervals, calibrates these intervals to contain true latent values, and propagates them through the generator.
result Reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in image super-resolution and image completion.
New method evaluates text-to-image synthesis for realism, variety, and semantic accuracy.
problem Lack of metrics revealing semantic accuracy in text-to-image synthesis.
method Uses Inception network representations and t-SNE visualization for semantic evaluation.
result Classification accuracy of generated images to real images' visual concepts correlates with semantic accuracy.
ScoreAG generates unrestricted adversarial images maintaining semantic integrity.
problem Limited robustness evaluations due to ℓp-norm constraints. method Score-Based Adversarial Generation (ScoreAG) using score-based generative models.
result ScoreAG improves robustness assessments across multiple benchmarks.
This paper proposes synthetic augmentation for nuclei image segmentation in medical pathology.
problem Rare and time-consuming labeling of tumor nuclei images for semantic segmentation.
method Label-to-image translation to generate synthetic images.
result Synthetic augmentation improves segmentation accuracy.
Paper develops a framework for generating coherent image captions using visual features and hierarchical topics.
problem Generating semantically coherent paragraphs to describe image content.
method Plug-and-play hierarchical-topic-guided image paragraph generation framework integrating visual extractor and deep topic model.
result Proposed models can distill interpretable multi-layer semantic topics and generate diverse and coherent captions.
Generative model creates realistic scenes from pixel-wise labels.
problem Creating photo-realistic scenes from pixel-wise labels.
method Semantic bottleneck GAN model combining conditional and unconditional generation networks.
result Model outperforms state-of-the-art models in unsupervised image synthesis.
Paper proposes structured semantic perturbations to improve adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Manipulates semantic attributes via disentangled latent codes.
result Demonstrates the effectiveness of structured semantic perturbations.
The paper uses differentiable rendering to generate semantic counterexamples for improving neural network robustness.
problem Neural networks' brittleness to semantic transformations.
method Differentiable rendering for generating realistic images that model semantic changes, combined with adversarial machine learning attacks.
result Semantic counterexamples improve generalization, robustness, and transferability of neural networks.
We present a framework for translating unlabeled images from one domain into analog images in another domain. We employ a progressively growing skip-connected encoder-generator structure and train it with a GAN loss for realistic output, a cycle consistency loss for maintaining same-domain translation identity, and a s…
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to tr…
Paper proposes a method to improve semantic segmentation for fisheye urban driving images.
problem Semantic segmentation for fisheye urban driving images is challenging due to distortion and lack of large datasets.
method A seven degrees of freedom augmentation method is proposed to transform rectilinear images into fisheye images.
result Training with seven-DoF augmentation improves model accuracy and robustness against distorted fisheye data.
Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance on a variety of computer vision tasks, particularly visual classification problems, where new algorithms reported to achieve or even surpass the human performance. In this paper, we examine whether CNNs are capable of learning the semantics…
Enhances image classification by integrating semantic hierarchy into CNN models.
problem Limited use of external guidance in image classification.
method Integrates label-hierarchy knowledge into CNN-based classifiers and uses order-preserving embeddings.
result Boosts image classification performance through semantic hierarchy integration.
Image captioning has demonstrated models that are capable of generating plausible text given input images or videos. Further, recent work in image generation has shown significant improvements in image quality when text is used as a prior. Our work ties these concepts together by creating an architecture that can enabl…
We develop a method for user-controllable semantic image inpainting: Given an arbitrary set of observed pixels, the unobserved pixels can be imputed in a user-controllable range of possibilities, each of which is semantically coherent and locally consistent with the observed pixels. We achieve this using a deep generat…
Self-supervised learning of visual semantics in image games.
problem Learning visual semantics in referential emergent language games.
method Investigating the impact of feature extractor weights and tasks on visual semantics, using various image augmentations and additional tasks.
result Communication systems can learn visual semantics in a self-supervised manner by playing the right types of games.
SASSL improves self-supervised learning by preserving image structure.
problem Distorted augmented samples in self-supervised learning.
method Neural Style Transfer to decouple semantic and stylistic attributes.
result Boosts ImageNet top-1 accuracy by up to 2 percentage points.
New method uses coupled SDEs to edit images with high fidelity and consistency.
problem Challenges in editing image content with text-to-image models.
method Using coupled stochastic differential equations to guide generative model sampling.
result Achieves high prompt fidelity and near-pixel-level consistency.
A new ZSL algorithm uses shared sparse representations for unseen classes.
problem Classifying images from unseen classes using only semantic information.
method Coupled dictionary learning to represent visual and semantic features in an intermediate space.
result The proposed method outperforms state-of-the-art ZSL algorithms on benchmark datasets.
The thesis introduces methods to use semantic hierarchy in image classification.
problem Limited work in training image classifiers with non-conventional external guidance.
method Injects label hierarchy knowledge into arbitrary classifiers and uses order-preserving embeddings for image classification.
result Both embedding-based models and CNN-classifiers with hierarchical information outperform a hierarchy-agnostic model.
This survey analyzes deep learning methods for real-time semantic image segmentation.
problem Efficient and accurate semantic image segmentation for real-time applications.
method Comprehensive analysis of deep learning architectures and techniques for fast inference.
result Best-performing architectures and techniques for real-time semantic image segmentation.
Advocates Tversky's model for image similarity learning.
problem Learning Tversky similarity measures from image data.
method Computational approach using Tversky's ratio model.
result Performs well compared to existing methods on image datasets.
Method synthesizes 4D CMR images from XCAT model using GAN and SPADE.
problem Synthesizing realistic 4D CMR images with annotations and adaptable styles.
method Hybrid GAN approach with XCAT anatomical ground truth and SPADE for semantic preservation.
result Synthesized images with modality-specific features learned from real CMR data.
New research shows semantic data matching can degrade SSDL performance.
problem The limits of semantic data set matching in semi-supervised learning.
method Demonstrated through simulations and a new dissimilarity measure.
result Semantic data matching can degrade SSDL performance under non-IID data.
Novel framework for whole-slide blood cell segmentation.
problem Semantic segmentation of whole-slide blood cell images.
method Convolutional encoder-decoder framework with VGG-16 feature extraction.
result Outstanding accuracy (97.18% global accuracy) in classwise segmentation.
RCAV quantifies model sensitivity to semantic concepts, improving interpretability methods.
problem Lack of semantic interpretability in image classification models.
method RCAV calculates concept gradients and ascent steps to assess model sensitivity to semantic concepts.
result RCAV yields more accurate and robust interpretations of model behavior.
Unsupervised segmentation learns features without labels, improving accuracy.
problem Discover and localize semantically meaningful categories in images without annotations.
method Separates feature learning from cluster compactification; distills unsupervised features into discrete semantic labels using a contrastive loss function.
result Significant improvement over prior state of the art on semantic segmentation challenges.
CAggNet improves medical image segmentation by fusing coarse and fine features.
problem Medical image segmentation accuracy and efficiency.
method Crossing Aggregation Network with nested skip connections and weighted aggregation.
result CAggNet achieves more accurate and efficient segmentation compared to existing methods.
Method interprets GAN latent space via latent variable correlation analysis.
problem Understanding the inner workings of GANs.
method Analyzing correlation between latent variables and semantic contents in generated images.
result A method for controllable semantic content generation in GANs.
Building a large image dataset with high-quality object masks for semantic segmentation is costly and time consuming. In this paper, we introduce a principled semi-supervised framework that only uses a small set of fully supervised images (having semantic segmentation labels and box labels) and a set of images with onl…
A new method for image translation using disentangled style and content preservation.
problem Difficulty in maintaining original content during reverse diffusion in diffusion-based image translation.
method Disentangled style and content representation using intermediate keys from ViT model, CLIP loss, semantic divergence loss, and resampling strategy.
result Outperforms state-of-the-art models in text-guided and image-guided translation tasks.
Paper tackles zero-shot learning for semantic image interpretation.
problem Extracting structured semantic descriptions from images requires complete training sets, which are often unavailable.
method Uses Logic Tensor Networks to leverage logical constraints and similarities among relationships in the training set.
result Background knowledge can alleviate the incompleteness of training sets, improving zero-shot learning performance.
ZegOT uses optimal transport to zero-shot segment images with text prompts.
problem Zero-shot semantic segmentation with limited image-text alignment knowledge.
method ZegOT uses optimal transport to match multiple text prompts with frozen image embeddings.
result ZegOT achieves state-of-the-art performance in zero-shot semantic segmentation.
New method corrects complex distortions in single view images.
problem Complex distortions in images, especially those caused by refractive surfaces.
method Differentiable image sampling and semantic information augmentation.
result Model can estimate and correct highly complex distortions.
In this paper we study image captioning as a conditional GAN training, proposing both a context-aware LSTM captioner and co-attentive discriminator, which enforces semantic alignment between images and captions. We empirically focus on the viability of two training methods: Self-critical Sequence Training (SCST) and Gu…
Method explains anomaly detection by generating normal modifications.
problem Complexity of deep learning methods in anomaly detection.
method Generates multiple alternative modifications for anomalies.
result High-quality semantic explanations provided for anomaly detection.
Generative model extracts road networks from images.
problem Extracting road networks from image data.
method Developed Generative Graph Transformer (GGT) for image-conditioned graph generation.
result Demonstrated effective road network generation from semantic segmentation data.
Semantify-NN verifies neural network robustness against semantic perturbations.
problem Verifying robustness of neural networks against semantic adversarial attacks.
method Inserting semantic perturbation layers (SP-layers) into neural networks to verify robustness.
result Semantify-NN significantly improves robustness verification performance over ℓp-norm-based methods. StylEx trains a GAN to explain classifier decisions in StyleSpace.
problem Creating meaningful image-specific explanations for classifier decisions.
method Training a StyleGAN to learn a classifier-specific StyleSpace, incorporating the classifier model.
result StylEx finds attributes that align with semantic ones and generates human-interpretable explanations.
Superpixel-mix enhances reliability in semantic segmentation.
problem Improving reliability in real-world semantic segmentation.
method Superpixel-mix, a new data augmentation method with teacher-student consistency training.
result Superpixel-mix achieves state-of-the-art results in semi-supervised semantic segmentation.
For bidirectional joint image-text modeling, we develop variational hetero-encoder (VHE) randomized generative adversarial network (GAN), a versatile deep generative model that integrates a probabilistic text decoder, probabilistic image encoder, and GAN into a coherent end-to-end multi-modality learning framework. VHE…
NICE framework explains CNN predictions and compresses images.
problem Interpreting deep neural networks and efficient image compression.
method End-to-end Neural Image Compression and Explanation (NICE) framework.
result NICE achieves high compression rates while maintaining classification accuracy.
Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.
problem Creating annotated datasets for semantic segmentation of lung diseases is laborious and time-consuming.
method Proposes a weak supervision technique that utilizes partially annotated datasets to improve CNN semantic segmentation accuracy.
result Significantly improved segmentation accuracy using partially annotated datasets.
Generates coherent storybooks from plain text using diffusion models.
problem Ensuring coherency in a sequence of images for storytelling applications.
method Combines pre-trained LLM and text-guided Latent Diffusion Model for zero-shot generation.
result Outperforms state-of-the-art image editing baselines in generating coherent storybooks.
Synthetic image translation has significant potentials in autonomous transportation systems. That is due to the expense of data collection and annotation as well as the unmanageable diversity of real-words situations. The main issue with unpaired image-to-image translation is the ill-posed nature of the problem. In thi…