PG-IM uses neural-symbolic programs to manipulate images.
problem Creating holistic image representations and manipulations.
method PG-IM detects patterns, induces symbolic programs, and manipulates images using a neural network.
result PG-IM learns from a single image and achieves superior performance.
The method learns disentangled representations for localized image manipulations.
problem Image generating neural networks are viewed as black boxes with global effects.
method Localized ResNet Autoencoder with multiple loss functions.
result The network can transfer specific facial attributes like shape and color of eyes, hair, mouth, etc. between persons.
Localized adversarial training improves image classifiers' robustness.
problem State-of-the-art image classifiers fail on carefully manipulated adversarial images.
method Developed a localized adversarial attack and used it to train a robust classifier.
result Localized adversarial training increases robustness against adversarial inputs.
Many problems in image processing and computer vision (e.g. colorization, style transfer) can be posed as 'manipulating' an input image into a corresponding output image given a user-specified guiding signal. A holy-grail solution towards generic image manipulation should be able to efficiently alter an input image wit…
In this paper, we introduce an unsupervised learning approach to automatically discover, summarize, and manipulate artistic styles from large collections of paintings. Our method is based on archetypal analysis, which is an unsupervised learning technique akin to sparse coding with a geometric interpretation. When appl…
FaceSigns embeds a secret watermark in images to authenticate and detect deepfakes.
problem Realistic image and video manipulation threats, especially deepfakes.
method Semi-fragile watermarking using neural networks, robust to face-swapping but fragile to deepfake manipulations.
result FaceSigns can reliably detect deepfake content with high accuracy.
Scanning Transmission Electron Microscopy (STEM) has become the main stay for materials characterization on atomic level, with applications ranging from visualization of localized and extended defects to mapping order parameter fields. In the last several years, attention was attracted by potential of STEM to explore b…
Vision and learning have made significant progress that could improve robotics policies for complex tasks and environments. Learning deep neural networks for image understanding, however, requires large amounts of domain-specific visual data. While collecting such data from real robots is possible, such an approach lim…
In this work we propose a model that can manipulate individual visual attributes of objects in a real scene using examples of how respective attribute manipulations affect the output of a simulation. As an example, we train our model to manipulate the expression of a human face using nonphotorealistic 3D renders of a f…
With technological advances leading to an increase in mechanisms for image tampering, fraud detection methods must continue to be upgraded to match their sophistication. One problem with current methods is that they require prior knowledge of the method of forgery in order to determine which features to extract from th…
P3I learns holistic scene representations from a single image.
problem Inferring camera poses, object locations, and global scene structures from a single image.
method Combines search-based and gradient-based algorithms.
result P3I outperforms baselines on various image manipulation tasks.
Proposes a new method to generate unrestricted adversarial examples.
problem Generating unrestricted adversarial examples without norm constraints.
method Leveraging state-of-the-art generative models to manipulate image fine-grained aspects.
result Our adversarial images look indistinguishable from natural images and can bypass certified defenses.
Paper explores vulnerabilities in image authenticity detection methods, especially printing and scanning attacks.
problem Vulnerability of image authenticity detection models to printing and scanning attacks.
method Demonstrates and proposes a new machine learning model to counter these attacks.
result Proposed model outperforms state-of-the-art models when trained on images from a single printer.
Method estimates fingertip forces, torques, and curvatures from fingernail images.
problem Estimating fingertip forces and curvatures in various contact scenarios.
method Deformation and color distribution analysis of fingernail images using neural networks.
result High accuracy in predicting fingertip forces, torques, and curvatures.
Manipulating video content is easier than ever. Due to the misuse potential of manipulated content, multiple detection techniques that analyze the pixel data from the videos have been proposed. However, clever manipulators should also carefully forge the metadata and auxiliary header information, which is harder to do …
Paper proposes structured semantic perturbations to improve adversarial attacks.
problem Vulnerability of deep neural networks to adversarial attacks.
method Manipulates semantic attributes via disentangled latent codes.
result Demonstrates the effectiveness of structured semantic perturbations.
Diffusion models enhance robotic manipulation through probabilistic multi-modal learning.
problem Enhancing robotic manipulation through robust and multi-modal learning.
method Probabilistic diffusion models integrating imitation and reinforcement learning.
result Diffusion models improve grasp learning, trajectory planning, and data augmentation in robotics.
We present a generative autoencoder that provides fast encoding, faithful reconstructions (eg. retaining the identity of a face), sharp generated/reconstructed samples in high resolutions, and a well-structured latent space that supports semantic manipulation of the inputs. There are no current autoencoder or GAN model…
Accurate estimation of the positions and shapes of microscale objects is crucial for automated imaging-guided manipulation using a non-contact technique such as optical tweezers. Perception methods that use traditional computer vision algorithms tend to fail when the manipulation environments are crowded. In this paper…
New method learns various data manipulation schemes for model training.
problem Improving model training with data manipulation.
method Adapts RL reward learning algorithm for data manipulation learning.
result Significant improvement in classification performance.
Unified CLIP space manipulations improve GAN adaptation with a single target image.
problem Overfitting or underfitting in fine-tuning a pre-trained generator with a single target image.
method Two-step training strategy: latent optimization in CLIP space followed by generator fine-tuning with CLIP space consistency loss.
result Our model generates diverse outputs with the target texture and outperforms baseline models.
Robotic clothing manipulation improved with fashion image analysis techniques.
problem Automated identification of clothing categories and landmarks for robotic tasks.
method Training data augmentation methods and rotation invariant convolutions.
result Our approach outperforms state-of-the-art models on unseen datasets.
Robots detect and recognize objects in real-time for better manipulation.
problem Real-time object detection and recognition for humanoid robots.
method Modified YOLOv3 algorithm, quantization, and re-arrangement of layers for low-compute NAO robots.
result Robots can perform real-time detection, recognition, and localization of objects.
This work proposes a model to disentangle image factors effectively and control their manipulation.
problem Controlling disentanglement during image editing while preserving object identity.
method Encoder-decoder architecture with decorrelation regularization and soft target representations.
result The model successfully disentangles image factors and manipulates them effectively.
Robotic grasping system learns to target objects from a single image.
problem Robotic grasping in unstructured environments.
method Distributed reinforcement learning, active vision, synchronous SGD.
result System learns to grasp unseen objects without retraining.
Notes on Whitney towers in 4-manifolds, focusing on local surface manipulations and invariants.
problem Classifying and understanding Whitney towers in 4-manifolds.
method Local manipulations of surfaces, definitions of Whitney towers and trees, geometric Jacobi identities, classification of twisted Whitney towers.
result Classification of order n twisted Whitney towers in the 4-ball and related invariants.
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
A new method for disentangled latent spaces in VAEs that can manipulate attributes.
problem Disentangled representation of attributes in latent spaces of VAEs.
method Attribute-based regularization loss to enforce monotonic relationships between attributes and latent codes.
result Manipulation of attributes in latent spaces post-training.
Method interprets GAN latent space via latent variable correlation analysis.
problem Understanding the inner workings of GANs.
method Analyzing correlation between latent variables and semantic contents in generated images.
result A method for controllable semantic content generation in GANs.
Paper presents a new port-Hamiltonian model for vehicle manipulators.
problem Complex mechanical systems' energy flow and conservation.
method Derives port-Hamiltonian dynamics from Hamiltonian reduction theory.
result Establishes mathematical equivalence with existing formulations.
InvGAN combines generative and inference models for photo-realistic image manipulation.
problem GANs lack an inference model for image editing and downstream tasks.
method Train inference and generative models together to adapt and converge.
result InvGAN embeds real images into a high-quality generative model's latent space.
Simpler method for separating and manipulating latent attributes in autoencoders.
problem Separating and manipulating latent attributes in autoencoders.
method Matrix subspace projection
result Our method allows for changing selected attributes while preserving other information.
RG-Flow combines RG and sparse priors for hierarchical image disentanglement.
problem Disentangling and manipulating image representations at different scales.
method Hierarchical flow model using RG and sparse prior distributions.
result RG-Flow enables semantic manipulation and style mixing at different image scales.
Weak supervision enables learning causal representations from unstructured data.
problem Learning high-level causal representations from unstructured data like images.
method Weakly supervised setting with paired samples before and after interventions. Implicit latent causal models using variational autoencoders.
result Models can reliably identify causal structure and disentangle causal variables.
Facial analysis models are increasingly used in applications that have serious impacts on people's lives, ranging from authentication to surveillance tracking. It is therefore critical to develop techniques that can reveal unintended biases in facial classifiers to help guide the ethical use of facial analysis technolo…
Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data. However, manipulating such representation to perform meaningful and controllable transformations in the data space remains challenging without some form of supervision. While previous w…
Researchers create a flickering attack to fool video recognition networks.
problem Adversarial manipulation of video classification networks.
method Introducing a flickering temporal perturbation to fool video classifiers.
result Achieved high fooling ratio and temporal-invariant perturbation.
We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed …
Study on costs of manipulating AMM-based price oracles.
problem Cost of manipulation in AMM-based on-chain price oracles.
method Analyzes the robustness of AMM-based oracles to strategic manipulation, considering different aggregation methods and market conditions.
result Manipulation costs depend on the total quote depth and can be minimized by optimal liquidity weights.
Defense against adversarial attacks by manipulating feature thickness.
problem Vulnerability of machine learning models to adversarial attacks.
method Feature Manipulation (FM)-Defense using a combo-variational autoencoder.
result Detection and purification of adversarial examples with high accuracy.
Paper proposes a method for optimizing local policies for trajectory-centric reinforcement learning.
problem Challenges in global policy optimization for non-linear systems and poor performance of open-loop trajectory optimization.
method Formulates trajectory optimization and local policy synthesis as a single optimization problem and solves it as a nonlinear programming instance.
result Demonstrates improved performance of the proposed technique under simplifying assumptions.
Proposes a new framework for image generation using classification latent space representations.
problem Combining discriminative and dense representations for image generation and reconstruction.
method Discriminative modeling framework using manipulated supervised latent representations.
result Higher classification accuracy and visually realistic image generation compared to existing models.
The study examines how backrun auctions can protect traders from price manipulation.
problem Price manipulation by arbitrageurs in batched trading venues.
method Developed a laminated queueing model to study price manipulation and introduced a price manipulation coefficient.
result Bound the price manipulation coefficient and found it approximated by a 'zeta value' with measurable parameters.
Agent learns third-person manipulation tasks from a single video.
problem Learning from third-person videos to perform novel tasks.
method Decoupling high-level task generation from low-level action prediction.
result Agent successfully learns and performs tasks in unseen scenarios.
Deep neural networks have achieved impressive experimental results in image classification, but can surprisingly be unstable with respect to adversarial perturbations, that is, minimal changes to the input image that cause the network to misclassify it. With potential applications including perception modules and end-t…
SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.
problem Gradient ascent in complex Q-functions leads to suboptimal solutions.
method SAVO actor generates multiple action proposals and truncates poor local optima.
result SAVO actor finds optimal actions more frequently and outperforms other architectures.
We are concerned with the vulnerability of computer vision models to distributional shifts. We formulate a combinatorial optimization problem that allows evaluating the regions in the image space where a given model is more vulnerable, in terms of image transformations applied to the input, and face it with standard se…
Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the …