DreamFusion uses text-to-image diffusion models to create 3D images efficiently.
problem Lack of large-scale 3D datasets and efficient architectures for 3D synthesis.
method Adapting a 2D diffusion model to 3D synthesis using a loss based on probability density distillation.
result A 3D model can be optimized from a 2D diffusion model, allowing for text-to-3D synthesis.
Worldsheet wraps a 3D mesh sheet onto a single image to synthesize novel views.
problem Synthesizing novel views from a single image with large viewpoint changes.
method Shrink-wrapping a planar mesh sheet onto the input image, consistent with learned depth.
result Worldsheet consistently outperforms prior methods on single-image view synthesis.
StyleNeRF generates high-resolution images with 3D consistency and style control.
problem Generating high-resolution images with fine details and 3D consistency.
method Integrates NeRF into a style-based generator for efficient high-resolution image synthesis.
result Synthesizes high-resolution images at interactive rates with high 3D consistency and style control.
LION generates high-quality 3D shapes using hierarchical latent diffusion models.
problem Creating high-quality 3D shapes for digital artists.
method Hierarchical Latent Point Diffusion Model (LION) with a global shape latent and point-structured latent space.
result LION achieves state-of-the-art generation performance on ShapeNet benchmarks.
Existing networks directly learn feature representations on 3D point clouds for shape analysis. We argue that 3D point clouds are highly redundant and hold irregular (permutation-invariant) structure, which makes it difficult to achieve inter-class discrimination efficiently. In this paper, we propose a two-faceted sol…
BPI models 2D patterns on multiple planes and 3D scene from a single image.
problem Understanding and editing images with multiple 2D planes and 3D scene from a single image.
method Box Program Induction (BPI) with neural networks and search-based algorithm.
result Holistic, structured scene representation enables 3D-aware image editing.
Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of o…
Generative model creates realistic dance poses from music.
problem Generating human-like dance poses from music.
method Music feature encoder, pose generator, music genre classifier integrated.
result Generative autoregressive model synthesizes dance sequences up to 5,000 frames.
We propose a novel multi-texture synthesis model based on generative adversarial networks (GANs) with a user-controllable mechanism. The user control ability allows to explicitly specify the texture which should be generated by the model. This property follows from using an encoder part which learns a latent representa…
We propose a data-driven 3D shape design method that can learn a generative model from a corpus of existing designs, and use this model to produce a wide range of new designs. The approach learns an encoding of the samples in the training corpus using an unsupervised variational autoencoder-decoder architecture, withou…
Topology-GS improves 3D GS for better structural and feature integrity.
problem Compromised pixel-level and feature-level integrity in 3D GS.
method Incorporates Local Persistent Voronoi Interpolation (LPVI) and PersLoss based on persistent homology.
result Topology-GS outperforms existing methods in PSNR, SSIM, and LPIPS metrics.
GTA improves transformer-based NVS models by encoding geometric structure.
problem Suboptimal positional encoding for 3D vision tasks.
method Geometry-aware attention mechanism encoding geometric structure of tokens.
result GTA improves learning efficiency and performance of NVS models.
A benchmark evaluates ioUS-to-MR synthesis methods for brain tumor surgery.
problem Difficult interpretation of ioUS images for brain tumor surgery.
method Six generators trained under four inference regimes and two targets on public data.
result SynDiff-2.5D best preserved downstream segmentation (U_Dice=0.55).
In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, invol…
Tree-based synthesis improves forecast accuracy in GDP and inflation.
problem Improving forecast accuracy in GDP and inflation.
method Developed a nonparametric synthesis function using regression trees.
result Tree-based synthesis leads to improved forecast accuracy.
We propose a systematic learning-based approach to the generation of massive quantities of synthetic 3D scenes and arbitrary numbers of photorealistic 2D images thereof, with associated ground truth information, for the purposes of training, benchmarking, and diagnosing learning-based computer vision and robotics algor…
Method synthesizes 4D CMR images from XCAT model using GAN and SPADE.
problem Synthesizing realistic 4D CMR images with annotations and adaptable styles.
method Hybrid GAN approach with XCAT anatomical ground truth and SPADE for semantic preservation.
result Synthesized images with modality-specific features learned from real CMR data.
Speech synthesis from EEG features using RNN.
problem Speech synthesis from EEG data.
method Recurrent Neural Network (RNN) regression model to predict acoustic features from EEG features.
result Feasibility of synthesizing speech directly from EEG features demonstrated.
SED integrates synthesis, execution, and debugging for neural program synthesis.
problem Challenges in synthesizing complex programs that match specifications.
method SED combines synthesis, execution, and debugging to improve neural program generation.
result SED reduces error rates and outperforms standard decoding methods.
The recent success of raw audio waveform synthesis models like WaveNet motivates a new approach for music synthesis, in which the entire process --- creating audio samples from a score and instrument information --- is modeled using generative neural networks. This paper describes a neural music synthesis model with fl…
System uses machine learning and automated reasoning to speed up PBE synthesis.
problem Slow synthesis in PBE due to domain-specific knowledge and large training datasets.
method Preprocess SyGuS PBE problems with a neural network to reduce search space, then use automated reasoning for faster solution.
result System outperforms all competing tools in the 2019 SyGuS Competition for the PBE Strings track by 47.65%.
The discovery of processes for the synthesis of new materials involves many decisions about process design, operation, and material properties. Experimentation is crucial but as complexity increases, exploration of variables can become impractical using traditional combinatorial approaches. We describe an iterative met…
A neural program synthesis method with iterative fix operations.
problem Creating correct programs from input-output examples.
method Combines encoder-decoder synthesis with a differentiable fixer.
result Improves synthesis accuracy by reducing discrepancies between outputs and desired outputs.
New approach generates better synthetic data for neural program synthesis.
problem Current approaches to neural program synthesis generalize poorly to real data.
method Adversarial approach to control synthetic data distributions.
result Proposed method outperforms current approaches.
Paper introduces a method for generating interlocutor-aware facial gestures in dyadic settings.
problem Generating appropriate non-verbal behavior for conversational agents in dyadic settings.
method Probabilistic method using multi-modal cues from the interlocutor to synthesize facial gestures.
result The model successfully leverages multi-modal input from the interlocutor to generate more appropriate behavior.
We prove that spherical spectral analysis and synthesis hold in Damek-Ricci spaces and derive two-radius theorems.
Generative model creates realistic scenes from pixel-wise labels.
problem Creating photo-realistic scenes from pixel-wise labels.
method Semantic bottleneck GAN model combining conditional and unconditional generation networks.
result Model outperforms state-of-the-art models in unsupervised image synthesis.
Inverse Drum Machine separates drum mixes using transcription and synthesis.
problem Separating individual drum tracks from mixed recordings.
method Analysis-by-synthesis framework combining deep learning and automatic transcription.
result Separation quality comparable to supervised methods requiring isolated stems.
The development of computed tomography (CT) image reconstruction methods that significantly reduce patient radiation exposure while maintaining high image quality is an important area of research in low-dose CT (LDCT) imaging. We propose a new penalized weighted least squares (PWLS) reconstruction method that exploits …
Language models predict inorganic synthesis conditions and temperatures.
problem Limited data and heuristic approaches constrain inorganic synthesis planning.
method Language models without fine-tuning predict precursor conditions and temperatures.
result Language models achieve high accuracy in predicting synthesis conditions and temperatures.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
We propose a framework for synthesis of geological images based on an exemplar image. We synthesize new realizations such that the discrepancy in the patch distribution between the realizations and the exemplar image is minimized. Such discrepancy is quantified using a kernel method for two-sample test called maximum m…
BOiLS optimizes circuit quality using Bayesian optimization.
problem Optimizing circuits with complex search spaces.
method Adapting Bayesian optimization to logic synthesis, using Gaussian process kernels and trust-region constrained acquisitions.
result Demonstrated superior performance in sample efficiency and QoR values.
AutoDiff combines auto-encoder and diffusion model for realistic tabular data synthesis.
problem Generating realistic synthetic tabular data with heterogeneous features.
method Employing auto-encoder architecture to handle tabular data's complexity.
result Synthetic tables from AutoDiff show good statistical fidelity and perform well in machine learning tasks.
Diffusion models outperform GANs in image synthesis quality.
problem Improving image synthesis quality using generative models.
method Developed and optimized diffusion models, and introduced classifier guidance for conditional synthesis.
result Achieved superior image synthesis quality with diffusion models, matching or exceeding GANs in FID scores.
Develops BPDS for better financial portfolio decisions.
problem Model uncertainty in financial time series forecasting.
method Bayesian dynamic modelling and predictive decision synthesis.
result Improved predictive and decision outcomes compared to traditional Bayesian analysis.
Program synthesis struggles with complex spatial relationships in image classification.
problem Challenges in solving Synthetic Visual Reasoning Test problems.
method Quantitative reanalysis of human and machine performance, improved program synthesis classifier, categorization of SVRT problems.
result Program synthesis is constrained by spatial relationships in images, not just shape specification.
Speech-driven visual speech synthesis involves mapping features extracted from acoustic speech to the corresponding lip animation controls for a face model. This mapping can take many forms, but a powerful approach is to use deep neural networks (DNNs). However, a limitation is the lack of synchronized audio, video, an…
WaveCycleGAN has recently been proposed to bridge the gap between natural and synthesized speech waveforms in statistical parametric speech synthesis and provides fast inference with a moving average model rather than an autoregressive model and high-quality speech synthesis with the adversarial training. However, the …
Model for directed synthesis of audio textures using multi-scale RNNs.
problem Challenges in modeling complex audio textures with traditional methods.
method Combining multi-scale RNNs with a conditioning strategy for user-directed synthesis.
result Demonstrated improved performance on various audio texture datasets.
We present a program synthesis-oriented dataset consisting of human written problem statements and solutions for these problems. The problem statements were collected via crowdsourcing and the program solutions were extracted from human-written solutions in programming competitions, accompanied by input/output examples…
Proposes a synthesis algorithm using Conformal Prediction for improved Deep Learning performance.
problem Assessing the quality of synthesised data for high-stake domains.
method Conformal Prediction framework for generating data from high-confidence feature space regions.
result Training sets extended with confident synthesised data improved Deep Learning performance by up to 61 percentage points F1-score.
InVAErt networks use data-driven methods for system synthesis and identifiability analysis.
problem Model synthesis and identifiability analysis for complex systems.
method Deterministic encoder and decoder, normalizing flow, variational encoder, loss function penalty coefficients, latent space sampling.
result Validation through various system types, demonstrating effectiveness of the framework.
Generative adversarial networks (GANs) are a recent approach to train generative models of data, which have been shown to work particularly well on image data. In the current paper we introduce a new model for texture synthesis based on GAN learning. By extending the input noise distribution space from a single vector …
This paper improves speech synthesis using a DGP with SRU for naturalness.
problem Improving naturalness in synthetic speech.
method Deep Gaussian process with a recurrent architecture using SRU.
result SRU-DGP outperforms other models in naturalness of synthetic speech.
The study examines optimal synthesis in a radially symmetric Grushin space with conditions on the weight function.
problem Optimal synthesis in a radially symmetric Grushin space with a weight function.
method Analysis of the geometry of R3 with a weighted Carnot-Carathéodory metric, providing conditions for Grushin-like structure, and describing optimal synthesis. result Sufficient conditions on the weight function ensure a Grushin-like structure, and the candidate cut time coincides with the true cut time in the integrable case.
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery. To complement the strides in synthesis planning driven by historical, experimental, and computed data, we present an automated method for connecting scientific literature to synthesis insights. Starting from natural la…
We study knots in 3d Chern-Simons theory with complex gauge group SL(N,C), in the context of its relation with 3d N=2 theory (the so-called 3d-3d correspondence). The defect has either co-dimension 2 or co-dimension 4 inside the 6d (2,0) theory, which is compactified on a 3-manifold M^. …