3DB framework tests and debugs computer vision models using photorealistic simulation.
problem Discovering vulnerabilities and understanding model decision-making in computer vision systems.
method Unified framework using photorealistic simulation to test and debug vision models.
result Insights generated by 3DB transfer to the physical world, enabling robustness analysis.
AutoSimulate efficiently optimizes synthetic data generation.
problem Optimizing synthetic data generation for machine learning.
method Differentiable approximation of the objective function for efficient optimization.
result Significantly faster (up to 50x) and more efficient (up to 30x) synthetic data generation.
A machine learning environment for detecting autonomous vehicle corner cases.
problem Testing autonomous driving software in the real world is difficult.
method Connecting CARLA simulation software to TensorFlow and custom AI client software.
result The system can identify situations where AI software fails to understand the scenario.
In this paper, we show how novel transfer reinforcement learning techniques can be applied to the complex task of target driven navigation using the photorealistic AI2THOR simulator. Specifically, we build on the concept of Universal Successor Features with an A3C agent. We introduce the novel architectural contributio…
Curiosity enhanced by audio-visual associations improves learning efficiency.
problem Challenges in reinforcement learning, especially predicting the future.
method Exploits multiple modalities (audio and vision) to predict novel associations.
result Improves exploration and learning efficiency in various environments.
We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algor…
Generative Adversarial Networks are a new family of generative models, frequently used for generating photorealistic images. The theory promises for the GAN to eventually reach an equilibrium where generator produces pictures indistinguishable for the training set. In practice, however, a range of problems frequently p…
Action-bisimulation learns long-horizon controllability for reinforcement learning.
problem Learning relevant state features in high-dimensional observations for robust reinforcement learning.
method Action-bisimulation encoding, inspired by bisimulation invariance, extends single-step controllability to multi-step.
result Action-bisimulation pretraining improves sample efficiency in various environments.
Efficiently distills pretrained text-to-image models without real data, improving FID and CLIP scores.
problem Slow iterative refinement process of diffusion-based text-to-image models.
method Guided Score identity Distillation with Long and Short Classifier-Free Guidance.
result Achieves state-of-the-art FID performance with competitive CLIP score.
Despite the remarkable progress in face recognition related technologies, reliably recognizing faces across ages still remains a big challenge. The appearance of a human face changes substantially over time, resulting in significant intra-class variations. As opposed to current techniques for age-invariant face recogni…
The increasingly photorealistic sample quality of generative image models suggests their feasibility in applications beyond image generation. We present the Neural Photo Editor, an interface that leverages the power of generative neural networks to make large, semantically coherent changes to existing images. To tackle…
Synthesizing high resolution photorealistic images has been a long-standing challenge in machine learning. In this paper we introduce new methods for the improved training of generative adversarial networks (GANs) for image synthesis. We construct a variant of GANs employing label conditioning that results in 128x128 r…
OBSER framework infers sub-environments from objects, outperforming scene-based methods.
problem Zero-shot recognition of environments from object distributions.
method Bayesian framework using metric and self-supervised learning models to estimate object distributions in latent space.
result OBSER framework reliably performs inference in open-world and photorealistic environments, outperforming scene-based methods.
We present a generative autoencoder that provides fast encoding, faithful reconstructions (eg. retaining the identity of a face), sharp generated/reconstructed samples in high resolutions, and a well-structured latent space that supports semantic manipulation of the inputs. There are no current autoencoder or GAN model…
Survey on GANs for generating visual arts, music, and literature.
problem Tackles the challenge of generating art using GANs.
method Uses generative adversarial networks (GANs) to generate visual arts, music, and literary text.
result Performance comparison and description of various GAN architectures presented.
We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to …
We propose a new algorithm for training generative adversarial networks that jointly learns latent codes for both identities (e.g. individual humans) and observations (e.g. specific photographs). By fixing the identity portion of the latent codes, we can generate diverse images of the same subject, and by fixing the ob…
Highly expressive models such as deep neural networks (DNNs) have been widely applied to various applications. However, recent studies show that DNNs are vulnerable to adversarial examples, which are carefully crafted inputs aiming to mislead the predictions. Currently, the majority of these studies have focused on per…
New approach uses FBSDE to sample complex distributions.
problem Sampling multidimensional distributions with known normalization constants.
method Reformulated FBSDE to avoid gradient estimation; numerical solution using Deep Learning.
result Unique solution to FBSDE proved; numerical method for sampling.
Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of o…
Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.
problem Synthesizing novel views from a single image with large viewpoint changes.
method Shrink-wrapping a planar mesh sheet onto the input image, consistent with learned depth.
result Worldsheet consistently outperforms prior methods on single-image view synthesis.
LANCA uses ANM to learn latent causal factors without supervision.
problem Learning latent causal factors without supervision.
method LANCA employs a deterministic Wasserstein Auto-Encoder coupled with a differentiable ANM Layer.
result LANCA outperforms baselines on physics and photorealistic environments.
The extension of image generation to video generation turns out to be a very difficult task, since the temporal dimension of videos introduces an extra challenge during the generation process. Besides, due to the limitation of memory and training stability, the generation becomes increasingly challenging with the incre…
Deep generative models (DGMs) of images are now sufficiently mature that they produce nearly photorealistic samples and obtain scores similar to the data distribution on heuristics such as Frechet Inception Distance (FID). These results, especially on large-scale datasets such as ImageNet, suggest that DGMs are learnin…
Traditional classifiers can generate high-quality images comparable to generative models.
problem Separation between classifiers and generators in neural networks.
method Optimizing input gradients to produce images, using mask-based stochastic reconstruction, progressive-resolution technique, and distance metric loss.
result Traditional classifiers can generate high-fidelity images of 256imes256 resolution on ImageNet. The simulator is an R package that streamlines the process of performing simulations by creating a common infrastructure that can be easily used and reused across projects. Methodological statisticians routinely write simulations to compare their methods to preexisting ones. While developing ideas, there is a temptatio…
PAMS is a Python-based platform for simulating artificial markets.
problem Simulating complex market behaviors for research and education.
method Developed as a Python-based simulator with deep learning integration.
result Demonstrated effectiveness through agent price prediction studies.
The interpretability of machine learning, particularly for deep neural networks, is crucial for decision making in real-world applications. One approach is replacing the un-interpretable machine learning model with a surrogate model, which has a simple structure for interpretation. Another approach is understanding the…
Smartfluidnet accelerates Eulerian fluid simulation with neural networks.
problem Current neural network methods for Eulerian fluid simulation lack flexibility and generalization.
method Smartfluidnet automates model generation and dynamic switching to meet user requirements.
result Smartfluidnet achieves 1.46x and 590x speedup compared to state-of-the-art models, with better simulation quality.
Proposes a new simulator for complex arrival processes.
problem Modeling and simulating complex arrival processes with non-stationary and multi-dimensional rates.
method Integrates Monte Carlo and GANs to model a broad class of arrival processes.
result Consistent and efficient estimation of the simulator using Wasserstein distance.
ACE improves GBI for simulators by approximating cost functions, making inference more efficient.
problem Inference for misspecified simulators is overly restrictive.
method Amortized cost estimation (ACE) for Generalized Bayesian Inference (GBI).
result ACE provides accurate cost predictions and more efficient inference.
New method improves sample-efficiency in neural posterior estimation using simulator gradients.
problem High-fidelity posterior estimation with complex physical simulations is time-consuming.
method Neural Posterior Estimation (NPE) with differentiable simulators and gradient information.
result Improves sample-efficiency in posterior density estimation.
Simulates risk-neutral markets using neural spline flows.
problem Creating realistic risk-neutral market simulations.
method Developed a low-dimensional martingale representation and used neural spline flows for sampling.
result The calibrated simulator is closest to historical data with respect to Kullback-Leibler divergence.
Generative Adversarial Networks simulate elevator group control without extensive data.
problem Lack of historical real-world data for system testing.
method Used GANs to generate simulation data for a multi-car elevator system.
result GANs can be used as substitutes for expensive simulation runs, but fine-tuning is needed.
Enhances SBI accuracy with multilevel Monte Carlo for expensive simulators.
problem Limited accuracy in SBI due to expensive simulators.
method Multilevel Monte Carlo techniques for cost-effective SBI.
result Significant enhancement in SBI accuracy with fixed computational budget.
Bayesian neural networks improve simulation-based inference with limited data.
problem Inaccurate inference in data-poor regimes with limited or expensive simulations.
method Bayesian neural networks for posterior approximation, accounting for computational uncertainty.
result Bayesian neural networks produce well-calibrated posteriors with few simulations.
Improved nested simulation for financial risk measurement.
problem Efficiently estimating nested risk measures in financial engineering.
method Reusing inner simulation outputs to improve efficiency and accuracy.
result The proposed approach outperforms standard nested simulation and regression methods.
The paper develops a new simulation technique for estimating conditional expectations in financial models.
problem Estimating conditional expectations in financial models with expensive simulation of endogenous variables.
method Introduces a hierarchical simulation scheme with oversimplified defaults to address variance issues.
result The hierarchical simulation technique significantly improves the success of neural net regression for conditional expectation estimation.
Study proposes a new approach for deep hedging using artificial market simulations.
problem Challenges in selecting the best model for underlying asset simulations in deep hedging.
method Proposes artificial market simulations to replicate financial market stylized facts.
result Achieves similar performance to traditional approaches without mathematical finance models.
Improved inference efficiency for complex simulations.
problem Challenges in performing inference under resource-intensive stochastic simulators.
method Active sequential neural posterior estimation (ASNPE) integrating active learning into posterior estimation.
result Improved sample efficiency with low computational overhead.
Simulation workflow is a top-level model for the design and control of simulation process. It connects multiple simulation components with time and interaction restrictions to form a complete simulation system. Before the construction and evaluation of the component models, the validation of upper-layer simulation work…
Method improves simulation accuracy by mitigating distribution shift in hybrid systems.
problem Mitigating distribution shift in machine-learning augmented hybrid simulation.
method Tangent-space regularized estimator to control distribution shift.
result Marked improvements in simulation accuracy, especially for systems with high distribution shift.
New simulation model predicts financial market dynamics with high accuracy.
problem Extreme difficulty in financial market projections due to human behavioural complexity.
method Agent-based modeling with a hierarchical knowledge architecture to simulate diverse human groups.
result Simulator achieves 13.29% deviation in crisis scenarios and lower mean square error under normal conditions.
New emulator bridges simulators using conditional optimal transport.
problem Bridging simulators with minimal distortion.
method Flow-based approach to learn likelihood transport, COT-FM for optimal matching.
result Emulator accurately captures full correction between simulators.
New method uses low-fidelity simulations to efficiently infer parameters of high-fidelity models.
problem Challenges in inferring parameters of computationally expensive high-fidelity models.
method Multifidelity simulation-based inference using transfer learning and adaptive selection of high-fidelity parameters.
result Significant reduction in the number of high-fidelity simulations required for inference.
Moate Simulation improves accuracy and speed of financial derivative pricing.
problem Efficiently pricing financial derivatives with high accuracy.
method Discrete time simulation of probability distributions using Moate Simulation.
result Moate Simulation provides highly accurate distributions for financial derivatives pricing.
Study shows current simulations are insufficient for optimal neural network training in cosmology.
problem Insufficient training data for neural networks in cosmological inference.
method Empirical neural scaling law and Cramer-Rao bound to forecast training simulations needed.
result Current simulation suites do not provide sufficient training data for optimal neural network performance.
Improved flow-based inference speeds up and boosts accuracy for complex simulations.
problem Challenging inverse problems in astronomy, such as modeling strong gravitational lens systems.
method Refines flow-based generative models with simulator feedback for posterior inference.
result Improves accuracy by 53% and speeds up inference by up to 67x.