arXiv research

DreamFusion uses text-to-image diffusion models to create 3D images efficiently.

Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.

StyleNeRF generates high-resolution images with 3D consistency and style control.

LION generates high-quality 3D shapes using hierarchical latent diffusion models.

Existing networks directly learn feature representations on 3D point clouds for shape analysis. We argue that 3D point clouds are highly redundant and hold irregular (permutation-invariant) structure, which makes it difficult to achieve inter-class discrimination efficiently. In this paper, we propose a two-faceted sol…

BPI models 2D patterns on multiple planes and 3D scene from a single image.

Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of o…

We propose a novel multi-texture synthesis model based on generative adversarial networks (GANs) with a user-controllable mechanism. The user control ability allows to explicitly specify the texture which should be generated by the model. This property follows from using an encoder part which learns a latent representa…

This paper proposes a framework which is able to generate a sequence of three-dimensional human dance poses for a given music. The proposed framework consists of three components: a music feature encoder, a pose generator, and a music genre classifier. We focus on integrating these components for generating a realistic…

We propose a data-driven 3D shape design method that can learn a generative model from a corpus of existing designs, and use this model to produce a wide range of new designs. The approach learns an encoding of the samples in the training corpus using an unsupervised variational autoencoder-decoder architecture, withou…

Topology-GS improves 3D GS for better structural and feature integrity.

GTA improves transformer-based NVS models by encoding geometric structure.

A benchmark evaluates ioUS-to-MR synthesis methods for brain tumor surgery.

In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, invol…

We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative…

Tree-based synthesis improves forecast accuracy in GDP and inflation.

We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algor…

Speech synthesis from EEG features using RNN.

SED integrates synthesis, execution, and debugging for neural program synthesis.

The recent success of raw audio waveform synthesis models like WaveNet motivates a new approach for music synthesis, in which the entire process --- creating audio samples from a score and instrument information --- is modeled using generative neural networks. This paper describes a neural music synthesis model with fl…

The discovery of processes for the synthesis of new materials involves many decisions about process design, operation, and material properties. Experimentation is crucial but as complexity increases, exploration of variables can become impractical using traditional combinatorial approaches. We describe an iterative met…

A neural program synthesis method with iterative fix operations.

New approach generates better synthetic data for neural program synthesis.

Paper introduces a method for generating interlocutor-aware facial gestures in dyadic settings.

We prove that spherical spectral analysis and synthesis hold in Damek-Ricci spaces and derive two-radius theorems.

Inverse Drum Machine separates drum mixes using transcription and synthesis.

The development of computed tomography (CT) image reconstruction methods that significantly reduce patient radiation exposure while maintaining high image quality is an important area of research in low-dose CT (LDCT) imaging. We propose a new penalized weighted least squares (PWLS) reconstruction method that exploits …

Language models predict inorganic synthesis conditions and temperatures.

Programming-by-example (PBE) is a synthesis paradigm that allows users to generate functions by simply providing input-output examples. While a promising interaction paradigm, synthesis is still too slow for realtime interaction and more widespread adoption. Existing approaches to PBE synthesis have used automated reas…

CrossBeam learns to search more efficiently in program synthesis.

We propose a framework for synthesis of geological images based on an exemplar image. We synthesize new realizations such that the discrepancy in the patch distribution between the realizations and the exemplar image is minimized. Such discrepancy is quantified using a kernel method for two-sample test called maximum m…

BOiLS optimizes circuit quality using Bayesian optimization.

AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.

Coupling the high-fidelity generation capabilities of label-conditional image synthesis methods with the flexibility of unconditional generative models, we propose a semantic bottleneck GAN model for unconditional synthesis of complex scenes. We assume pixel-wise segmentation labels are available during training and us…

Diffusion models outperform GANs in image synthesis quality.

Develops BPDS for better financial portfolio decisions.

Speech-driven visual speech synthesis involves mapping features extracted from acoustic speech to the corresponding lip animation controls for a face model. This mapping can take many forms, but a powerful approach is to use deep neural networks (DNNs). However, a limitation is the lack of synchronized audio, video, an…

WaveCycleGAN has recently been proposed to bridge the gap between natural and synthesized speech waveforms in statistical parametric speech synthesis and provides fast inference with a moving average model rather than an autoregressive model and high-quality speech synthesis with the adversarial training. However, the …

Model for directed synthesis of audio textures using multi-scale RNNs.

We present a program synthesis-oriented dataset consisting of human written problem statements and solutions for these problems. The problem statements were collected via crowdsourcing and the program solutions were extracted from human-written solutions in programming competitions, accompanied by input/output examples…

Proposes a synthesis algorithm using Conformal Prediction for improved Deep Learning performance.

InVAErt networks use data-driven methods for system synthesis and identifiability analysis.

Generative adversarial networks (GANs) are a recent approach to train generative models of data, which have been shown to work particularly well on image data. In the current paper we introduce a new model for texture synthesis based on GAN learning. By extending the input noise distribution space from a single vector …

This paper improves speech synthesis using a DGP with SRU for naturalness.

We study knots in 3d Chern-Simons theory with complex gauge group $SL(N,\mathbb{C})$, in the context of its relation with 3d $\mathcal{N}=2$ theory (the so-called 3d-3d correspondence). The defect has either co-dimension 2 or co-dimension 4 inside the 6d $(2,0)$ theory, which is compactified on a 3-manifold $\hat{M}$. …

The study examines optimal synthesis in a radially symmetric Grushin space with conditions on the weight function.

Leveraging new data sources is a key step in accelerating the pace of materials design and discovery. To complement the strides in synthesis planning driven by historical, experimental, and computed data, we present an automated method for connecting scientific literature to synthesis insights. Starting from natural la…

Bayesian spatial predictive synthesis improves spatial data predictions.