PG-IM uses neural-symbolic programs to manipulate images.
problem Creating holistic image representations and manipulations.
method PG-IM detects patterns, induces symbolic programs, and manipulates images using a neural network.
result PG-IM learns from a single image and achieves superior performance.
Many problems in image processing and computer vision (e.g. colorization, style transfer) can be posed as 'manipulating' an input image into a corresponding output image given a user-specified guiding signal. A holy-grail solution towards generic image manipulation should be able to efficiently alter an input image wit…
Model manipulates facial expressions without affecting other attributes.
problem Manipulating specific visual attributes in real scenes without altering others.
method Trains model on nonphotorealistic 3D renders to manipulate facial expressions, preserving other attributes.
result Model can manipulate facial expressions without affecting other attributes like head orientation.
FaceSigns embeds a secret watermark in images to authenticate and detect deepfakes.
problem Realistic image and video manipulation threats, especially deepfakes.
method Semi-fragile watermarking using neural networks, robust to face-swapping but fragile to deepfake manipulations.
result FaceSigns can reliably detect deepfake content with high accuracy.
Improved autoencoder for realistic face manipulation.
problem Face identity conservation and disentanglement in high-resolution images.
method Modified progressively growing autoencoder (PIONEER) with new normalization schemes.
result Significantly improved visual and quantitative results for face identity conservation.
Paper proposes detecting video manipulation using stream descriptors.
problem Misuse of manipulated video content.
method Binary classifiers on multimedia stream descriptors.
result Scalable approach can detect high-quality manipulations.
P3I learns holistic scene representations from a single image.
problem Inferring camera poses, object locations, and global scene structures from a single image.
method Combines search-based and gradient-based algorithms.
result P3I outperforms baselines on various image manipulation tasks.
Proposes a new method to generate unrestricted adversarial examples.
problem Generating unrestricted adversarial examples without norm constraints.
method Leveraging state-of-the-art generative models to manipulate image fine-grained aspects.
result Our adversarial images look indistinguishable from natural images and can bypass certified defenses.
The method learns disentangled representations for localized image manipulations.
problem Image generating neural networks are viewed as black boxes with global effects.
method Localized ResNet Autoencoder with multiple loss functions.
result The network can transfer specific facial attributes like shape and color of eyes, hair, mouth, etc. between persons.
Paper explores vulnerabilities in image authenticity detection methods, especially printing and scanning attacks.
problem Vulnerability of image authenticity detection models to printing and scanning attacks.
method Demonstrates and proposes a new machine learning model to counter these attacks.
result Proposed model outperforms state-of-the-art models when trained on images from a single printer.
Paper discovers and manipulates artistic styles in paintings without supervision.
problem Automatic discovery and manipulation of artistic styles in large art collections.
method Unsupervised learning using archetypal analysis on deep image representations.
result Learned dictionary of archetypal styles can be used for style interpretation and manipulation.
Paper proposes structured semantic perturbations to improve adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Manipulates semantic attributes via disentangled latent codes.
result Demonstrates the effectiveness of structured semantic perturbations.
Diffusion models enhance robotic manipulation through probabilistic multi-modal learning.
problem Enhancing robotic manipulation through robust and multi-modal learning.
method Probabilistic diffusion models integrating imitation and reinforcement learning.
result Diffusion models improve grasp learning, trajectory planning, and data augmentation in robotics.
New method learns various data manipulation schemes for model training.
problem Improving model training with data manipulation.
method Adapts RL reward learning algorithm for data manipulation learning.
result Significant improvement in classification performance.
Unified CLIP space manipulations improve GAN adaptation with a single target image.
problem Overfitting or underfitting in fine-tuning a pre-trained generator with a single target image.
method Two-step training strategy: latent optimization in CLIP space followed by generator fine-tuning with CLIP space consistency loss.
result Our model generates diverse outputs with the target texture and outperforms baseline models.
Robotic clothing manipulation improved with fashion image analysis techniques.
problem Automated identification of clothing categories and landmarks for robotic tasks.
method Training data augmentation methods and rotation invariant convolutions.
result Our approach outperforms state-of-the-art models on unseen datasets.
Optimizes synthetic image augmentation for sim2real policy transfer in robotics.
problem Difficulty in transferring learned policies from simulated to real environments.
method Optimizes random transformations to augment synthetic images, enabling policy learning without real data.
result Significant improvement in policy accuracy on real robots for three manipulation tasks.
This work proposes a model to disentangle image factors effectively and control their manipulation.
problem Controlling disentanglement during image editing while preserving object identity.
method Encoder-decoder architecture with decorrelation regularization and soft target representations.
result The model successfully disentangles image factors and manipulates them effectively.
Robotic grasping system learns to target objects from a single image.
problem Robotic grasping in unstructured environments.
method Distributed reinforcement learning, active vision, synchronous SGD.
result System learns to grasp unseen objects without retraining.
Deep learning improves microscale object segmentation.
problem Accurate estimation of crowded microscale objects.
method Deep learning model for semantic segmentation.
result High mean Intersection Over Union score of 0.91.
Model removes objects from general scenes using weak supervision.
problem Automatic object removal from general scene images with weak supervision.
method Two-stage editor architecture with mask generator and image in-painter; novel GAN prior for mask generator.
result Effectively removes a wide variety of objects from general scenes using weak supervision.
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
Glow uses invertible 1x1 convolutions to improve image generation and manipulation.
problem Efficient and realistic image generation and manipulation.
method Invertible 1x1 convolutions in generative flows.
result Significant improvement in log-likelihood and realistic image synthesis.
A new method for disentangled latent spaces in VAEs that can manipulate attributes.
problem Disentangled representation of attributes in latent spaces of VAEs.
method Attribute-based regularization loss to enforce monotonic relationships between attributes and latent codes.
result Manipulation of attributes in latent spaces post-training.
Method interprets GAN latent space via latent variable correlation analysis.
problem Understanding the inner workings of GANs.
method Analyzing correlation between latent variables and semantic contents in generated images.
result A method for controllable semantic content generation in GANs.
InvGAN combines generative and inference models for photo-realistic image manipulation.
problem GANs lack an inference model for image editing and downstream tasks.
method Train inference and generative models together to adapt and converge.
result InvGAN embeds real images into a high-quality generative model's latent space.
Simpler method for separating and manipulating latent attributes in autoencoders.
problem Separating and manipulating latent attributes in autoencoders.
method Matrix subspace projection
result Our method allows for changing selected attributes while preserving other information.
RG-Flow combines RG and sparse priors for hierarchical image disentanglement.
problem Disentangling and manipulating image representations at different scales.
method Hierarchical flow model using RG and sparse prior distributions.
result RG-Flow enables semantic manipulation and style mixing at different image scales.
Weak supervision enables learning causal representations from unstructured data.
problem Learning high-level causal representations from unstructured data like images.
method Weakly supervised setting with paired samples before and after interventions. Implicit latent causal models using variational autoencoders.
result Models can reliably identify causal structure and disentangle causal variables.
Scanning Transmission Electron Microscopy (STEM) has become the main stay for materials characterization on atomic level, with applications ranging from visualization of localized and extended defects to mapping order parameter fields. In the last several years, attention was attracted by potential of STEM to explore b…
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed …
Defense against adversarial attacks by manipulating feature thickness.
problem Vulnerability of machine learning models to adversarial attacks.
method Feature Manipulation (FM)-Defense using a combo-variational autoencoder.
result Detection and purification of adversarial examples with high accuracy.
This work introduces a framework to detect unintended bias in facial analysis models.
problem Detecting unintended biases in facial analysis models used in critical applications.
method Image counterfactual sensitivity analysis using generative adversarial networks.
result Identifies factors affecting facial classifier predictions, revealing unintended biases.
Method estimates fingertip forces, torques, and curvatures from fingernail images.
problem Estimating fingertip forces and curvatures in various contact scenarios.
method Deformation and color distribution analysis of fingernail images using neural networks.
result High accuracy in predicting fingertip forces, torques, and curvatures.
Robot learns tool use from effects, detecting features of tools, objects, and actions.
problem Teaching robots to understand and manipulate objects using tools.
method Deep learning model trained on sensory-motor data from a robot performing a tool-use task.
result Robot can detect features of tools, objects, and actions from effects of object manipulation.
Proposes a new framework for image generation using classification latent space representations.
problem Combining discriminative and dense representations for image generation and reconstruction.
method Discriminative modeling framework using manipulated supervised latent representations.
result Higher classification accuracy and visually realistic image generation compared to existing models.
A new image representation method using hypernetworks.
problem Representing images in a way that allows for continuous manipulation and analysis.
method Constructing a hypernetwork that maps pixel positions to colors, allowing for continuous image manipulation.
result Comparable image super-resolution results to existing methods using a single model.
Deep neural networks have achieved impressive experimental results in image classification, but can surprisingly be unstable with respect to adversarial perturbations, that is, minimal changes to the input image that cause the network to misclassify it. With potential applications including perception modules and end-t…
Agent learns third-person manipulation tasks from a single video.
problem Learning from third-person videos to perform novel tasks.
method Decoupling high-level task generation from low-level action prediction.
result Agent successfully learns and performs tasks in unseen scenarios.
New method uses counterfactuals to reveal modular structure in deep generative models.
problem Challenges in manipulating deep generative models' latent representations without supervision.
method Proposes a non-statistical framework based on counterfactual manipulations.
result Modules of disentangled latent variables can be used for targeted interventions.
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …
Transflow Learning transforms pre-trained models without retraining.
problem Transforming pre-trained models without retraining.
method Bayesian inference to warp latent vector probability distribution.
result Transforms model outputs to resemble new data without training.
The paper addresses model vulnerability to image transformations.
problem Vulnerability of computer vision models to distributional shifts.
method Formulates a combinatorial optimization problem and uses search algorithms to evaluate vulnerability regions. Embeds this idea in a training procedure to define new data augmentation rules.
result Trains more robust models against distributional shifts and image manipulations.
Camera stickers can fool deep learning systems by manipulating the lens, achieving 49.6% misclassification rate.
problem The vulnerability of deep learning systems to physical adversarial attacks.
method Iterative procedure to update attack perturbation and threat model for physical realizability.
result Achieved 49.6% misclassification rate for targeted attacks on ImageNet classifiers.
In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier. We train a masking model to manipulate the scores of the classifier by masking salient parts of the input image. Our model generalises well to unseen images and requires a single forward pass to perform …
Localized adversarial training improves image classifiers' robustness.
problem State-of-the-art image classifiers fail on carefully manipulated adversarial images.
method Developed a localized adversarial attack and used it to train a robust classifier.
result Localized adversarial training increases robustness against adversarial inputs.
DNCF framework recovers real scenes from imperfect images robustly.
problem Recovering real scenes from imperfect images.
method Nonparametric deep network that learns physical image formation equations.
result DNCF framework robustly defends against adversarial attacks.