Improved PixelCNN model with faster training and better performance.
problem Generating high-quality images with tractable likelihood.
method Discretized logistic mixture likelihood, pixel conditioning, downsampling, short-cut connections, dropout regularization.
result State-of-the-art log likelihood results on CIFAR-10.
PixelCNN models can achieve state-of-the-art results on CIFAR-10 with exact likelihood computation.
problem Dequantization gap in modeling discrete data like images.
method Introducing subset flows to allow exact computation of likelihoods for discrete data.
result PixelCNN models trained with exact likelihood computation achieve state-of-the-art results on CIFAR-10.
PixelVAE++ improves generative models for natural images by combining VAE and PixelCNN.
problem Challenges in constructing powerful generative models for natural images.
method Introduces PixelVAE++, a VAE with three types of latent variables and a PixelCNN++ for the decoder, reusing a part of the decoder as an encoder.
result Achieves state-of-the-art performance on binary data sets and CIFAR-10.
Bayesian PixelCNN improves semi-supervised learning in MRI data.
problem Lack of robust uncertainty measures in deep generative models for MRI data.
method Extended PixelCNN to volumetric MRI data and reformulated as a deep Gaussian process.
result Improved semi-supervised learning performance in MRI data.
A new model learns from few examples using latent variables.
problem Learning from few examples with powerful neural networks.
method Variational Homoencoder (VHE) for hierarchical latent variable model.
result VHE outperforms existing models on test set likelihood and one-shot tasks.
Probabilistic inpainting learns multiple plausible images from missing data.
problem Generating multiple plausible images from missing data in images.
method Building a PixelCNN model that learns a distribution of images conditioned on visible pixels.
result The method produces diverse and realistic inpaintings.
AIQN improves generative models with quantile regression.
problem Improving perceptual quality and diversity in generative models.
method AIQN uses autoregressive implicit quantile networks for generative modeling.
result AIQN achieves superior perceptual quality and diversity without sacrificing diversity.
A method speeds up generation in convolutional autoregressive models.
problem Slow generation in convolutional autoregressive models.
method Cache hidden states to avoid redundant computation.
result Up to 21x and 183x speedups in generation for Wavenet and PixelCNN++ models.
LMConv improves autoregressive models for image generation and completion.
problem Limited generation order in autoregressive models restricts their applicability.
method Introduces LMConv, a modified 2D convolution that allows arbitrary masks to be applied to weights.
result LMConv achieves improved performance on image density estimation and coherent completions.
A method for user-controlled semantic image filling.
problem Generating coherent images with user-specified semantics.
method Deep generative model combining encoder, latent variables, and PixelCNN.
result User can control the inpainting of unobserved pixels while maintaining semantic coherence.
Generative models struggle with out-of-distribution data, but new methods show they can be improved.
problem Generative models struggle with assigning higher likelihood to out-of-distribution (OOD) data.
method Empirical challenge of previous reasoning, new data-set separability, and novel model architectures.
result Generative models can be improved by changing the data representation space and using PCA for anomaly detection.
TensorFlow Distributions simplifies deep learning with flexible probability tools.
problem Creating efficient and flexible probability models for deep learning.
method Two basic abstractions (Distributions and Bijectors) for fast, differentiable computation.
result Modular construction of complex distributions and transformations.
Plug-and-play multimodal controller improves class-conditional image generation.
problem Generating class-conditional images from user-specified labels.
method Introduces a `multimodal controller` to generate multimodal data without additional learning parameters.
result Multimodal controlled generative models produce higher quality class-conditional images and novel modalities.
ResNet learns to compress information during training.
problem Applying information bottleneck principle to deep learning models.
method Trained PixelCNN++ models to measure mutual information between ResNet hidden layers and input data.
result Compression occurs in both classification and autoencoding training regimes.
Parallelizes feedforward computation using nonlinear equation solving.
problem Sequential nature of feedforward computation limits parallelization.
method Frame feedforward computation as solving nonlinear equations; use Jacobi or Gauss-Seidel methods for parallel updates.
result Accelerates feedforward computation with reduced parallelizable iterations.
Tensor networks improve generative modeling of natural images.
problem Exponential decay of correlation in Matrix Product States limits their use for complex data.
method Introduced Tree Tensor Networks (TTN) for 2D data, developed efficient learning and sampling algorithms.
result TTN outperforms Matrix Product States in keeping pixel correlations and log-likelihood scores.
Generative models can confuse house numbers with dog images.
problem Generative models struggle with distinguishing between different datasets.
method Investigated flow-based models, VAEs, and PixelCNNs trained on CIFAR-10.
result Generative models assign higher likelihood to house numbers than to dog images.