Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
This paper compares two GAN architectures to reduce mode collapse.
problem Reducing mode collapse in Generative Adversarial Networks (GANs).
method Explains and compares PacGAN and VEEGAN models.
result PacGAN performs slightly better than vanilla GAN in terms of mode collapse.
SR-GANs combat mode collapse in GANs by monitoring and compensating spectral distributions.
problem Mode collapse in GANs.
method Spectral regularization (SR-GANs) to combat spectral collapse.
result SR-GANs prevent mode collapse and outperform SN-GANs in experiments.
Generative adversarial networks (GANs) are innovative techniques for learning generative models of complex data distributions from samples. Despite remarkable recent improvements in generating realistic images, one of their major shortcomings is the fact that in practice, they tend to produce samples with little divers…
This work explains GAN mode collapse and convergence issues via optimal transportation theory.
problem GANs struggle with convergence and mode collapse due to discontinuous optimal transportation mappings.
method The study connects GANs to optimal transportation theory, testing hypotheses about discontinuity and proposing a new method to approximate continuous Brenier potentials.
result The supports of real data distributions are often non-convex, leading to discontinuous optimal transportation mappings and mode collapse in GANs.
Theoretical work on mode collapse in variational inference models.
problem Mode collapse in variational inference models, where models focus on a few modes instead of all possible ones.
method Theoretical investigation of mode collapse in Gaussian mixture models, identifying key low-dimensional statistics and equations governing their evolution.
result Mode collapse is present even in favorable scenarios, driven by mean alignment and vanishing weight mechanisms.
MFVI mode collapse explained; RoVI proposed to mitigate.
problem Mode collapse in MFVI for mixture distributions.
method Introducing ε-separateness, deriving bounds, proposing RoVI.
result MFVI optimizers collapse to a single component when components are ε-separated.
Meta-CoTGAN improves adversarial text generation by preventing mode collapse.
problem Mode collapse in adversarial text generation.
method Meta-Cooperative Training Paradigm with a language model.
result Meta-CoTGAN effectively slows down mode collapse and improves generation quality and diversity.
New metrics improve scRNA-seq perturbation modeling by reducing mode collapse.
problem Outperformed by simple mean prediction in scRNA-seq perturbation modeling.
method Introduce DEG-aware metrics (WMSE, Rw2(Δ)) and negative/positive baselines. result WMSE loss function reduces mode collapse and improves model performance.
GAN+VER improves GANs by regularizing entropy to reduce mode collapse.
problem Mode collapse in GANs where the generator fails to capture all modes.
method Maximizing a variational lower bound on the entropy of generated samples.
result Significant improvement in evaluation metrics for real and generated samples.
New model explains GAN training dynamics and mode collapse.
problem Mode collapse in GANs, where generators fail to reproduce diversity.
method Simplified model using particles in output space, coupled by universal kernel.
result Gradient regularizers can optimally yield convergence through critical damping.
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
MMCGAN uses explicit manifold learning to improve GAN performance.
problem GAN mode collapse and unstable training.
method Introduces Minimum Manifold Coding (MMC) as a prior to guide GAN training.
result MMCGAN effectively alleviates mode collapse and stabilizes GAN training.
New model generates data without mode collapse or mixture.
problem Mode collapse and mode mixture in generative models.
method Inspired by AE-OT, improved sample generation algorithm.
result Free from mode collapse and mixture issues.
GANs mode collapse solved with Bures distance.
problem GANs mode collapse or mode dropping.
method Use Bures distance to match real and fake batch diversity in feature space.
result Diversity matching reduces mode collapse and improves sample quality.
DoPaNet uses multiple discriminators to prevent mode collapse in GANs.
problem Mode collapse in adversarial training.
method Employing multiple discriminators and a classifier to guide the generator.
result DoPaNet effectively covers the target distribution and outperforms competing methods.
Improved RL training for DMs reduces mode collapse and preserves diversity.
problem Mode collapse and training instability in RL fine-tuned diffusion models.
method Dynamic hierarchical RL training with sliding-window parameter regularisation.
result Models trained with HRF achieve better preservation of diversity in downstream tasks.
Stable GFlowNets prevent loss spikes and mode collapse in training.
problem Unstable training of GFlowNets leading to loss spikes and mode collapse.
method Assessed sensitivity of GFlowNet objectives, derived loss-to-TV bounds, and proposed Stable GFlowNets.
result Stable GFlowNets improve training behavior and distributional fidelity.
Generative model learns diverse generations using DPP to avoid mode collapse.
problem Mode collapse in generative models, limiting output to few modes.
method Inspired by DPP, proposes an unsupervised penalty loss to enhance diversity.
result Generative DPP approach consistently resists mode-collapse and outperforms state-of-the-art methods.
Solves GAN mode collapse by assigning minibatches to multiple discriminators.
problem Mode collapse in GANs where models generate similar samples.
method Multiple discriminators, microbatching, and changing tasks.
result Promotes sample diversity in generated sets.
InfoQGAN uses mutual information to improve QGANs, overcoming mode collapse and feature disentanglement issues.
problem Mode collapse and lack of feature control in QGANs.
method Integrates InfoGAN principles with variational quantum circuit, classical discriminator, and MINE for mutual information optimization.
result InfoQGAN effectively mitigates mode collapse and achieves robust feature disentanglement.
PresGANs improve GANs by mitigating mode collapse and enhancing log-likelihood.
problem GANs struggle with mode collapse and lack a reliable way to evaluate generalization.
method PresGANs add noise to density networks and use entropy regularization to stabilize training and capture all modes.
result PresGANs reduce the gap in predictive log-likelihood between GANs and VAEs.
A new GAN model α-GAN with tunable loss function addresses gradient vanishing and mode collapse issues.
problem Addressing vanishing gradients and mode collapse in GANs.
method Introduced a tunable GAN α-GAN using a supervised α-loss function. result Holistic understanding of α-GAN related to Arimoto divergence and convergence properties. Intervention loss stabilizes GAN training and reduces mode collapse.
problem Stabilizing GAN training and mitigating mode collapse.
method Intervention loss regularization term introduced into GAN objective.
result Improved GAN training stability and reduced mode collapse.
New method trains neural samplers without simulation, but fails due to mode collapse.
problem Training neural samplers without simulation.
method Time-dependent normalizing flow with Langevin preconditioning.
result Langevin preconditioning is crucial for avoiding mode collapse.
Compares GAN and VAE models, highlighting differences in fidelity and mode collapse.
problem Understanding and comparing GAN and VAE models.
method Empirical comparison of GAN and VAE models on MNIST and CelebA datasets.
result Hypothesis explaining differences in fidelity and mode collapse between GAN and VAE models.
iWGAN improves GANs by stabilizing training and preventing mode collapse.
problem Stable and effective training of GANs with mode collapse.
method iWGAN combines auto-encoders and WGANs using iterative primal dual optimization.
result iWGAN provides a clear stopping criterion and mitigates mode collapse.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.
method Differentiable variational inference with variational dropout and automatic relevance determination.
result State-of-the-art accuracy with up to twice fewer non-zero parameters.
Generative adversarial networks (GANs) often suffer from unpredictable mode-collapsing during training. We study the issue of mode collapse of Boundary Equilibrium Generative Adversarial Network (BEGAN), which is one of the state-of-the-art generative models. Despite its potential of generating high-quality images, we …
In this paper, we show that Generative Adversarial Networks (GANs) suffer from catastrophic forgetting even when they are trained to approximate a single target distribution. We show that GAN training is a continual learning problem in which the sequence of changing model distributions is the sequence of tasks to the d…
Deep generative models provide powerful tools for distributions over complicated manifolds, such as those of natural images. But many of these methods, including generative adversarial networks (GANs), can be difficult to train, in part because they are prone to mode collapse, which means that they characterize only a …
Paper proposes OKGAN to improve GAN training, addressing mode collapse and cycling.
problem Challenges in GAN training, including mode collapse and cycling.
method Kernel-based non-parametric discriminator for online training.
result OKGAN mitigates training issues and performs better than other GAN formulations.
SPECTRE uses spectral conditioning to generate larger graphs without mode collapse.
problem Overcoming expressivity and mode collapse in one-shot graph generators.
method SPECTRE generates graph Laplacian spectrum and eigenvectors to model graph structure.
result SPECTRE outperforms state-of-the-art deep autoregressive generators in fidelity and speed.
New method prevents mode collapse in deep SVDD for anomaly detection.
problem Mode collapse in deep SVDD due to architectural constraints.
method Two regularizers: noise injection and minibatch variance penalization.
result Regularized deep SVDD outperforms state-of-the-art methods.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
CNFs model conditional densities efficiently and avoid mode collapse.
problem Capturing strong correlations between output dimensions in structured prediction tasks.
method Conditional normalizing flows (CNFs) with a likelihood-based objective.
result CNFs achieve competitive performance in super-resolution and vessel segmentation tasks.
SIG combines explicit and implicit models for better sample diversity.
problem Mode collapsing in GANs and multi-modality samples.
method Introduces semi-implicit generator (SIG) as a hierarchical model trained in maximum likelihood.
result SIG improves GANs' ability to generate diverse samples, stabilizing training and resisting mode collapse.
Survey of GANs and autoencoders, addressing mode collapse and likelihood issues.
problem Addressing mode collapse and likelihood issues in GANs and autoencoders.
method Explains various GAN and autoencoder variants, their applications, and methods to resolve issues.
result Various methods to resolve mode collapse and improve likelihood in GANs and autoencoders.
NS-GAN mode collapse due to sample weighting inversion, solved with MM-nsat.
problem Mode collapse in GANs due to sample weighting inversion.
method Preserves MM-GAN sample weighting while avoiding saturation by rescaling gradients.
result MM-nsat improves mode coverage, stability, and FID on MNIST and CIFAR-10.
A new hybrid VAE-GAN framework improves mode coverage and quality.
problem Mode collapse and poor sample quality in GANs and VAEs.
method Integrates a 'Best-of-Many-Samples' reconstruction cost and a stable synthetic likelihood estimate.
result Significant improvement in mode coverage and quality compared to hybrid VAE-GANs and plain GANs.
End-to-end learnable Gaussian mixture priors improve diffusion models' exploration and expressiveness.
problem Challenges in diffusion models when priors differ from target distributions.
method End-to-end learnable Gaussian mixture priors (GMPs) with iterative refinement.
result Significant performance improvements across various benchmark problems.
We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch. Our approach forces the single generator not to constrain its output to satisfy a single discriminator,…
Unbalanced GANs stabilize GAN training by pre-training the generator with VAE.
problem Stable training of GANs to avoid mode collapses and improve image quality.
method Pre-train GAN generator with VAE, balance generator and discriminator training, prevent discriminator's early convergence.
result Unbalanced GANs reduce mode collapses and outperform ordinary GANs in stability, convergence, and image quality.
This work tackles GAN training instability through parallel tempering.
problem Training instability and mode collapse in GANs.
method Introduces a parallel tempering framework to stabilize GAN training.
result Significantly reduces gradient variance and improves training efficiency.
New findings show language models can't simultaneously avoid hallucinations and capture all language richness.
problem Achieving both valid output and full language richness in language generation.
method Investigates language generation within a statistical setting, focusing on consistency and breadth.
result For most collections of candidate languages, a language model cannot simultaneously avoid hallucinations and capture all language richness.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
problem Estimating KL divergences for normalizing flows efficiently and accurately.
method Path-gradient estimators for reverse and forward KL divergences.
result Path-gradient estimators lead to faster convergence and better approximation results.
PAS method improves UDA by progressively refining subspaces for reliable pseudo-labels.
problem Mitigating mode collapse in unsupervised domain adaptation.
method Progressive Adaptation of Subspaces (PAS) approach.
result PAS effectively mitigates mode collapse and improves performance in UDA and PDA.
Unified evaluation framework for sampling methods.
problem Lack of a standardized evaluation framework for sampling methods.
method Introduces a benchmark suite and performance criteria for sampling methods.
result Insights into strengths and weaknesses of existing sampling methods.