3DB framework tests and debugs computer vision models using photorealistic simulation.
problem Discovering vulnerabilities and understanding model decision-making in computer vision systems.
method Unified framework using photorealistic simulation to test and debug vision models.
result Insights generated by 3DB transfer to the physical world, enabling robustness analysis.
Unified model for age-invariant face recognition with photorealistic face synthesis.
problem Reliable face recognition across ages remains challenging due to significant intra-class variations.
method Unified deep architecture for cross-age face synthesis and recognition, continuous face rejuvenation/aging, disentangled age-invariant face representations.
result Superior performance on CAFR and other cross-age datasets, promising generalizability to unconstrained face recognition.
AutoSimulate efficiently optimizes synthetic data generation.
problem Optimizing synthetic data generation for machine learning.
method Differentiable approximation of the objective function for efficient optimization.
result Significantly faster (up to 50x) and more efficient (up to 30x) synthetic data generation.
We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algor…
Efficiently distills pretrained text-to-image models without real data, improving FID and CLIP scores.
problem Slow iterative refinement process of diffusion-based text-to-image models.
method Guided Score identity Distillation with Long and Short Classifier-Free Guidance.
result Achieves state-of-the-art FID performance with competitive CLIP score.
This paper investigates how dataset properties affect GAN training outcomes.
problem GANs often fail to reach equilibrium due to instability or mode collapse.
method Experiments to identify patterns in dataset properties and their effects on GAN training.
result Patterns in dataset properties influence GAN training dynamics and outcomes.
The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images. To tackle…
Synthesizing high resolution photorealistic images has been a long-standing challenge in machine learning. In this paper we introduce new methods for the improved training of generative adversarial networks (GANs) for image synthesis. We construct a variant of GANs employing label conditioning that results in 128x128 r…
VUSFA improves transfer learning for target-driven navigation in AI2THOR.
problem Improving transfer reinforcement learning for complex visual navigation tasks.
method Introducing SFDP and Variational Information Bottlenecks to A3C agent.
result VUSFA achieves state-of-the-art performance and generalizability.
OBSER framework infers sub-environments from objects, outperforming scene-based methods.
problem Zero-shot recognition of environments from object distributions.
method Bayesian framework using metric and self-supervised learning models to estimate object distributions in latent space.
result OBSER framework reliably performs inference in open-world and photorealistic environments, outperforming scene-based methods.
Survey on GANs for generating visual arts, music, and literature.
problem Tackles the challenge of generating art using GANs.
method Uses generative adversarial networks (GANs) to generate visual arts, music, and literary text.
result Performance comparison and description of various GAN architectures presented.
Improved autoencoder for realistic face manipulation.
problem Face identity conservation and disentanglement in high-resolution images.
method Modified progressively growing autoencoder (PIONEER) with new normalization schemes.
result Significantly improved visual and quantitative results for face identity conservation.
Curiosity enhanced by audio-visual associations improves learning efficiency.
problem Challenges in reinforcement learning, especially predicting the future.
method Exploits multiple modalities (audio and vision) to predict novel associations.
result Improves exploration and learning efficiency in various environments.
We propose a new algorithm for training generative adversarial networks that jointly learns latent codes for both identities (e.g. individual humans) and observations (e.g. specific photographs). By fixing the identity portion of the latent codes, we can generate diverse images of the same subject, and by fixing the ob…
New approach uses FBSDE to sample complex distributions.
problem Sampling multidimensional distributions with known normalization constants.
method Reformulated FBSDE to avoid gradient estimation; numerical solution using Deep Learning.
result Unique solution to FBSDE proved; numerical method for sampling.
VNLA uses vision and language to guide agents in finding objects in indoor environments.
problem Guiding agents in finding objects in indoor environments via language.
method Developed I3L framework for imitation learning with indirect intervention.
result Significantly improved success rate of learning agents over baselines.
Action-bisimulation learns long-horizon controllability for reinforcement learning.
problem Learning relevant state features in high-dimensional observations for robust reinforcement learning.
method Action-bisimulation encoding, inspired by bisimulation invariance, extends single-step controllability to multi-step.
result Action-bisimulation pretraining improves sample efficiency in various environments.
A machine learning environment for detecting autonomous vehicle corner cases.
problem Testing autonomous driving software in the real world is difficult.
method Connecting CARLA simulation software to TensorFlow and custom AI client software.
result The system can identify situations where AI software fails to understand the scenario.
Paper proposes meshAdv to generate adversarial 3D meshes for visual recognition.
problem Vulnerability of deep neural networks to adversarial examples.
method Differentiable renderer to manipulate shape and texture of 3D meshes.
result 3D meshes effectively attack classifiers and object detectors.
Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.
problem Synthesizing novel views from a single image with large viewpoint changes.
method Shrink-wrapping a planar mesh sheet onto the input image, consistent with learned depth.
result Worldsheet consistently outperforms prior methods on single-image view synthesis.
Generative model creates realistic images with 3D understanding.
problem Lack of 3D understanding in existing image generation models.
method Disentangled 3D representation using shape, viewpoint, and texture.
result Generates more realistic images and enables 3D operations.
LANCA uses ANM to learn latent causal factors without supervision.
problem Learning latent causal factors without supervision.
method LANCA employs a deterministic Wasserstein Auto-Encoder coupled with a differentiable ANM Layer.
result LANCA outperforms baselines on physics and photorealistic environments.
Generative model for high-resolution video generation.
problem Challenges in generating high-resolution videos due to memory and training stability limitations.
method Progressive growing of sliced Wasserstein GANs (SWGAN) for incremental spatiotemporal information learning.
result Generated photorealistic face videos of 256x256x32 resolution with an inception score of 14.57.
Generative models struggle with class prediction on real data.
problem Evaluating generative models' ability to infer class labels.
method Trained classifiers on synthetic data generated by various models and tested on real data.
result Generative models from different classes outperform GANs on a new classification accuracy score (CAS).
Traditional classifiers can generate high-quality images comparable to generative models.
problem Separation between classifiers and generators in neural networks.
method Optimizing input gradients to produce images, using mask-based stochastic reconstruction, progressive-resolution technique, and distance metric loss.
result Traditional classifiers can generate high-fidelity images of 256imes256 resolution on ImageNet.