Wasserstein GANs fail to approximate Wasserstein distance, leading to their success.
problem Approximating Wasserstein distance in deep generative models.
method Analysis of differences between theoretical setup and training reality.
result Wasserstein GANs' success is due to their failure to approximate Wasserstein distance.
Vanilla GANs are connected to Wasserstein distance for better understanding.
problem Understanding the statistical properties of Vanilla GANs.
method Connecting Vanilla GANs to Wasserstein distance and proving an oracle inequality.
result An oracle inequality for Vanilla GANs in Wasserstein distance is obtained.
Paper bridges f-GANs and WGANs for better image generation.
problem Learning high-dimensional distributions using GANs.
method List constraints, minimize Lagrangian relaxation, propose KL-Wasserstein GAN.
result Empirical success on synthetic and real-world image generation benchmarks.
Orthogonal Wasserstein GANs improve image quality without gradient norm regularization.
problem Wasserstein-GANs' gradient norm regularization limits the distribution's fidelity.
method Substituted gradient norm regularization with orthogonality constraints on weight matrices.
result Orthogonal Wasserstein GANs achieve better image quality and generalization.
Generative adversarial networks (GANs) have received a tremendous amount of attention in the past few years, and have inspired applications addressing a wide range of problems. Despite its great potential, GANs are difficult to train. Recently, a series of papers (Arjovsky & Bottou, 2017a; Arjovsky et al. 2017b; and Gu…
Wasserstein distance improves GANs by reducing training difficulties.
problem Training difficulties and arbitrary hyperparameters in GANs.
method Estimating Wasserstein distance for generative modeling.
result Various ways to estimate Wasserstein distance for generative models.
Improved GAN performance using higher-order Wasserstein moments.
problem Stabilizing and enhancing GANs for better mode coverage and stability.
method Deriving and training a GAN with a modified Wasserstein distance using higher-order moments.
result Training a GAN with higher-order Wasserstein moments improves performance, even with increased computational cost.
Generative Adversial Networks (GANs) have made a major impact in computer vision and machine learning as generative models. Wasserstein GANs (WGANs) brought Optimal Transport (OT) theory into GANs, by minimizing the 1-Wasserstein distance between model and data distributions as their objective function. Since then, W…
The study improves PAC-Bayesian bounds for adversarial generative models.
problem Improving generalization bounds for adversarial generative models.
method Extending PAC-Bayesian theory to generative models, developing bounds for Wasserstein and total variation distances.
result New training objectives for Wasserstein and Energy-Based GANs.
Paper proposes robust estimators for GANs under Wasserstein contamination.
problem Robust estimation of distributions under contamination.
method Wasserstein GAN-based estimators for location, covariance, and regression.
result Proposed estimators are minimax optimal in many scenarios.
Paper relaxes the Lipschitz constraint in WGANs to improve performance.
problem WGANs do not always outperform other GAN variants due to imperfect implementation of the Lipschitz condition.
method Proposes a new dual form of Wasserstein distance (Sobolev duality) that relaxes the Lipschitz constraint but maintains gradient property.
result SWGAN, based on Sobolev duality, outperforms existing methods in experiments.
The paper explores multidimensional critic output in GANs, improving convergence and diversity.
problem Underexplored in GANs literature, multidimensional critic output.
method Generalized Wasserstein GAN framework, SRVT block, maximal p-centrality discrepancy.
result High-dimensional critic output improves GAN performance in convergence and diversity.
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimizat…
Generative adversarial nets (GANs) and variational auto-encoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, image-to-image translation and feature learning. However, to model high-dimensional distributions, sequential training and stacked architectures …
Wasserstein GANs are shown to have hidden convexity, enabling exact solutions with convex optimization.
problem Non-convex and non-concave optimization in GANs.
method Convex duality analysis of Wasserstein GANs with two-layer neural network discriminators.
result Wasserstein GANs can be solved exactly with convex optimization under certain conditions.
iWGAN improves GANs by stabilizing training and preventing mode collapse.
problem Stable and effective training of GANs with mode collapse.
method iWGAN combines auto-encoders and WGANs using iterative primal dual optimization.
result iWGAN provides a clear stopping criterion and mitigates mode collapse.
MonoFlow rethinks GANs using Wasserstein gradient flows.
problem Inconsistencies between GAN theory and practice.
method Unified generative modeling framework based on Wasserstein gradient flows.
result Adversarial training can be seen as particle flow optimization.
MWGAN tackles multi-marginal matching problem with Wasserstein GAN.
problem Learning mappings to match a source domain to multiple target domains with cross-domain correlations.
method Develops a novel Multi-marginal Wasserstein GAN (MWGAN) with inner- and inter-domain constraints to minimize Wasserstein distance.
result Theoretical and empirical evaluations show MWGAN's effectiveness on balanced and imbalanced translation tasks.
Generative adversarial networks are a class of generative algorithms that have been widely used to produce state-of-the-art samples. In this paper, we investigate GAN to perform anomaly detection on time series dataset. In order to achieve this goal, a bibliography is made focusing on theoretical properties of GAN and …
Generative adversarial networks (GANs) are one of the most popular approaches when it comes to training generative models, among which variants of Wasserstein GANs are considered superior to the standard GAN formulation in terms of learning stability and sample quality. However, Wasserstein GANs require the critic to b…
GANICE improves GAN-based causal inference by minimizing averaged Wasserstein risk.
problem Estimating interventional outcome distributions and quantiles in causal inference.
method GANICE uses extended Wasserstein distance and a cellwise critic to minimize averaged Wasserstein risk.
result GANICE achieves minimax optimality and consistently outperforms existing methods.
SGD finds global optima in WGANs for 1-layer generators.
problem Training GANs with WGANs requires global optimality, which is hard.
method Used SGD to train 1-layer generator networks in WGANs.
result SGD converges to global solution in polynomial time and sample complexity.
This paper explains the math behind a generative adversarial network (GAN) model and why it is hard to be trained. Wasserstein GAN is intended to improve GANs' training by adopting a smooth metric for measuring the distance between two probability distributions.
Simplified GAN model shows how discriminator improves generalization.
problem Understanding GAN's generalization ability and avoiding memorization.
method Analyzing a simplified GAN model with early stopping and Wasserstein metric.
result Generalization error escapes from curse of dimensionality with early stopping.
Paper introduces a new method to improve GANs by leveraging the discriminator's energy.
problem Improving the quality of generated samples in GANs.
method Discriminator Contrastive Divergence (DCD) method.
result Significant improvement in generation quality on synthetic and real-world data.
The paper introduces a new ODE approach to improve Wasserstein GANs.
problem Improving Wasserstein GANs for better training results.
method Derives an ODE representing the gradient flow of Wasserstein-1 loss and proposes a new model W1-FE.
result W1-FE outperforms WGAN in training experiments across various dimensions.
The paper introduces a new Wasserstein distance for approximating posteriors in inverse problems.
problem Approximating posterior measures in inverse problems using conditional Wasserstein distances.
method Introduces a conditional Wasserstein distance with restricted couplings and derives its dual.
result Shows that conditional Wasserstein GANs can yield favorable properties for posterior sampling.
Paper optimizes WGAN parameters for non-Gaussian data.
problem Optimizing parameters for non-Gaussian data in WGAN.
method Characterization of optimal solutions for population WGAN beyond LQG setting, using sliced Wasserstein framework.
result Closed-form optimal parameters for non-linear activation functions and non-Gaussian data derived.
Generative Adversarial Networks (GANs) have become a powerful framework to learn generative models that arise across a wide variety of domains. While there has been a recent surge in the development of numerous GAN architectures with distinct optimization metrics, we are still lacking in our understanding on how far aw…
New findings suggest minimax optimality doesn't guarantee distribution learning for GANs.
problem Understanding when GANs can truly learn the underlying distribution.
method Using cryptographic assumptions and ReLU network generators, the paper shows that achieving minimax optimality is insufficient for distribution learning.
result Achieving minimax optimality is insufficient for distribution learning in the usual statistical sense.
This report has several purposes. First, our report is written to investigate the reproducibility of the submitted paper On the regularization of Wasserstein GANs (2018). Second, among the experiments performed in the submitted paper, five aspects were emphasized and reproduced: learning speed, stability, robustness ag…
New method simulates multivariate extreme events using GANs and Aitchison coordinates.
problem Simulating multivariate extreme events for economic risk assessment.
method Wasserstein-Aitchison GAN approach combining tail dependence and marginal tail modeling.
result Strong performance in capturing tail dependence and generating accurate extreme observations.
Generative adversarial networks (GANs) are an exciting alternative to algorithms for solving density estimation problems---using data to assess how likely samples are to be drawn from the same distribution. Instead of explicitly computing these probabilities, GANs learn a generator that can match the given probabilisti…
Generative Adversarial Networks (GANs) are one of the most practical methods for learning data distributions. A popular GAN formulation is based on the use of Wasserstein distance as a metric between probability distributions. Unfortunately, minimizing the Wasserstein distance between the data distribution and the gene…
While Generative Adversarial Networks (GANs) have empirically produced impressive results on learning complex real-world distributions, recent works have shown that they suffer from lack of diversity or mode collapse. The theoretical work of Arora et al. suggests a dilemma about GANs' statistical properties: powerful d…
In this paper, we investigate the underlying factor that leads to failure and success in the training of GANs. We study the property of the optimal discriminative function and show that in many GANs, the gradient from the optimal discriminative function is not reliable, which turns out to be the fundamental cause of fa…
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only low-quality samples or fail to converge. We find that these problems are often du…
Wasserstein GANs with Gradient Penalty compute a different optimal transport problem called congested transport.
problem Training generative models to produce high-quality synthetic data.
method Wasserstein GANs with Gradient Penalty (WGAN-GP) approach to calculate the Wasserstein 1 distance.
result WGAN-GP computes the minimum of the congested transport problem, not the Wasserstein 1 distance.
Study extends DRO with IPMs, linking robustness to regularization and GANs.
problem Addressing robustness of deep neural networks to adversarial attacks.
method Distributionally Robust Optimization (DRO) with Integral Probability Metrics (IPMs).
result DRO under any IPM corresponds to a family of regularization penalties.
A new GAN loss function based on cumulant generating functions improves stability and robustness.
problem Improving the stability and performance of GANs.
method Cumulant GAN loss function based on variational R{é}nyi divergence.
result Cumulant GAN achieves linear convergence to Nash equilibrium and superior performance in image generation.
Estimates intrinsic dimension of data for GANs.
problem Estimating intrinsic dimension of high-dimensional data.
method Uses Wasserstein distances for estimation.
result Provides sample complexity bounds for GANs.
The paper explores theoretical insights into WGANs for better understanding and stability.
problem Stabilizing the training process of GANs.
method Theoretical analysis and statistical convergence study of WGANs.
result Theoretical properties and convergence of WGANs are clarified.
Adaptive framework for learning latent space dimensions in GANs.
problem Inadequate latent space dimensions lead to poor generative models for complex data.
method Proposes a novel framework (LWGAN) that adaptively learns latent dimensions of data manifolds.
result Proves that the estimated intrinsic dimension is a consistent estimate of the true data manifold dimension.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
HW2MP-GAN tackles ancient handwritten text recognition.
problem Automatic text recognition from ancient handwritten records.
method Conditional Generative Adversarial Network (HW2MP-GAN) with Sliced Wasserstein distance and U-Net architectures.
result HW2MP-GAN outperforms state-of-the-art models in image-to-image translation and handwritten recognition.
This study analyzes how well GANs approximate distributions from small samples.
problem Understanding how well GANs approximate distributions from limited data.
method Analysis of GANs using integral probability metrics and Hölder classes.
result GANs can adaptively learn low-dimensional structures or Hölder densities.
Proposes Sig-Wasserstein GANs for generating time series with temporal dependence.
problem Challenges in generating time series with temporal dependence and high-dimensional data.
method Integrates Wasserstein-GANs with signature feature extraction for conditional time series generation.
result Consistently outperforms state-of-the-art benchmarks in similarity and predictive ability.
Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications. However, this framework has two main drawbacks: (i) Wasserstein-1 (or Earth-Mover) distance is restrictive such that WGANs cannot always fit data geometry well; (ii) It …