Improves text-to-image diffusion models using GFlowNets.
problem Aligning diffusion models with text descriptions.
method Post-training diffusion models with GFlowNets to generate high-reward images.
result Effective alignment of large-scale text-to-image diffusion models with reward information.
DreamFusion uses text-to-image diffusion models to create 3D images efficiently.
problem Lack of large-scale 3D datasets and efficient architectures for 3D synthesis.
method Adapting a 2D diffusion model to 3D synthesis using a loss based on probability density distillation.
result A 3D model can be optimized from a 2D diffusion model, allowing for text-to-3D synthesis.
Improved text-to-image and multimodal understanding through adaptive generation order optimization.
problem Determining optimal generation sequences in text-to-image synthesis and multimodal understanding.
method Introduced a learnable control module trained via Group Relative Policy Optimization (GRPO) to determine the generation order.
result Learning the control block substantially improves text-to-image alignment and multimodal understanding in DLMs.
New text-to-image diffusion models improve scene understanding for AI agents.
problem Fine-grained scene understanding for AI agents from text and images.
method Pre-trained text-to-image diffusion models optimized for generating images from text prompts.
result Policies learned with Stable Control Representations outperform state-of-the-art approaches on various control tasks.
Training-free method improves text-to-image generation quality.
problem Error accumulation in simultaneous token updates of masked diffusion models.
method Training-free self-correction framework exploiting inductive biases.
result Significantly improved generation quality on text-to-image tasks.
Novel alignment framework for text-to-image generation using diffusion models and flow matching.
problem Improving text-to-image generation with minimal computational resources.
method Proposes a novel alignment framework that decomposes the score function into pre-trained score plus a conditional expectation of the reward.
result Achieves comparable performance to finetuning-based models with reduced computational cost.
Self-play fine-tuning improves diffusion models for text-to-image generation.
problem Plateauing performance of diffusion models after data saturation.
method Self-play fine-tuning (SPIN-Diffusion) using competition among model versions.
result Significantly improved model performance and human preference alignment.
PAC-Bayesian theory improves text-to-image models by enforcing alignment and generalization.
problem Text-to-image models struggle with complex prompts, misaligning modifiers and neglecting certain elements.
method Proposes a Bayesian approach with custom priors over attention distributions to enforce desirable properties.
result Achieves state-of-the-art results across multiple metrics on standard benchmarks.
Improved text-to-image alignment using iterative VQA feedback.
problem Misalignment between text prompts and generated images, especially for complex inputs.
method Decompose complex prompts into assertions, evaluate each using VQA, combine scores iteratively.
result Significantly higher correlation with human ratings compared to CLIP, BLIP scores.
Gradient-based framework for optimizing text prompts in diffusion models.
problem Efficiently optimizing prompts in text-to-image diffusion models with large domain space and non-differentiable embeddings.
method Formulated as discrete optimization over language space, designed compact subspaces, and introduced shortcut text gradient.
result Empirically discovered prompts that enhance or destroy image faithfulness.
Improved image compression with diffusion models outperforming state-of-the-art methods.
problem Difficulties in replicating text-to-image success in image compression.
method Two-stage approach combining autoencoder targeting MSE followed by score-based decoder.
result Significantly improved perceptual quality at a given bit-rate, measured by FID score.
Efficiently distills pretrained text-to-image models without real data, improving FID and CLIP scores.
problem Slow iterative refinement process of diffusion-based text-to-image models.
method Guided Score identity Distillation with Long and Short Classifier-Free Guidance.
result Achieves state-of-the-art FID performance with competitive CLIP score.
GLASS Flows improves flow and diffusion model performance by optimizing sampling efficiency.
problem Efficiency bottleneck in sampling Markov transitions for flow and diffusion models.
method Introduces GLASS Flows, a new sampling paradigm that simulates a 'flow matching model within a flow matching model' to sample Markov transitions efficiently.
result Eliminates the trade-off between stochastic evolution and efficiency in large-scale text-to-image models.
PoGDiff improves diffusion models on imbalanced datasets.
problem Diffusion models struggle with imbalanced text-image pairs.
method PoGDiff replaces ground-truth with a Product of Gaussians.
result PoGDiff enhances generation accuracy and quality on imbalanced datasets.
Generates coherent storybooks from plain text using diffusion models.
problem Ensuring coherency in a sequence of images for storytelling applications.
method Combines pre-trained LLM and text-guided Latent Diffusion Model for zero-shot generation.
result Outperforms state-of-the-art image editing baselines in generating coherent storybooks.
AutoAnchor uses cross-attention to improve text-to-image model unlearning.
problem Mitigating harmful or copyrighted content in text-to-image models.
method Two-stage framework that automatically synthesizes manifold-proximal anchors using cross-attention consistency loss.
result Effective robust and unbiased unlearning across various baselines.
New samplers improve compositional generation with diffusion models.
problem Improving compositional generation with diffusion models.
method Score-based interpretation, energy-based parameterization, Metropolis-corrected samplers.
result New samplers enable successful compositional generation across various tasks.
A new method learns continuous guidance weights to improve diffusion model quality and distributional alignment.
problem Improving perceptual quality and distributional alignment of samples from conditional diffusion models.
method Learned continuous guidance weights ωc,(s,t) are used to minimize distributional mismatch and reward guided sampling. result Improvements in Fréchet inception distance (FID) for image generation and better image-prompt alignment in text-to-image applications.
Paper proposes a faster, higher-quality RL method for text-to-image models.
problem Improving image quality and prompt alignment in RL post-training of text-to-image models.
method Online RL variant that reduces variance by sampling paired trajectories and optimizing flow velocity.
result Method converges faster and yields higher output quality and prompt alignment.
PaGoDA reduces diffusion model training costs by 64x.
problem Diffusion models are computationally expensive during training.
method Three-stage pipeline: downsampled training, distillation, progressive super-resolution.
result PaGoDA achieves state-of-the-art performance with reduced training costs.
Improves diversity of text-to-image models without sacrificing FID.
problem Lack of diversity and tendency to recreate training set images.
method Adds sparse repellency terms to diffusion SDE to guide trajectories away from a reference set.
result Improves diversity of diffusion models with minimal impact on FID.
Few-step distillation improves T2I models without real images or CFG trade-offs.
problem Challenges in accelerating T2I models with high-resolution and CFG.
method Score identity distillation (SiD) for few-step generation, with adversarial loss and new guidance strategies.
result State-of-the-art performance on SDXL at 1024x1024 resolution, robust to real images absence.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
New method improves image generation for inverse problems using text prompts.
problem Suboptimal performance of existing latent diffusion models for inverse problems.
method Prompt tuning and latent variable projection to optimize text embeddings.
result P2L method outperforms existing methods on various inverse problem tasks.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.
EM Distillation simplifies diffusion models to one-step generators.
problem Efficient sampling from complex diffusion models with minimal loss of quality.
method EM Distillation, a maximum likelihood approach based on Expectation-Maximization.
result EM Distillation outperforms existing one-step generative methods in FID scores.
New algorithm speeds up diffusion model sampling 4-14 times.
problem Time-consuming sampling from diffusion models.
method Parallelizing autoregressive process through fixed-point iteration.
result ParaTAA reduces inference steps by 4-14 times.
Single-stage neural architecture improves text-to-image synthesis.
problem Combining text and image generation is challenging.
method Used deep residual networks and sentence interpolation.
result Achieved state-of-the-art performance with single-stage training.
Generates garden paintings from text descriptions using deep learning.
problem Lack of firsthand material for traditional Chinese garden reconstruction.
method Deep learning model trained on text and paintings of Ming Dynasty gardens.
result Model generates garden paintings in Ming Dynasty style based on textual descriptions.
New diffusion models handle constrained domains, improving generative tasks.
problem Diffusion models struggle with manifolds defined by inequality constraints.
method Developed two noising processes: logarithmic barrier metric and reflected Brownian motion.
result Demonstrated practical utility on synthetic and real-world tasks.
TDS provides exact samples for conditional distributions in diffusion models.
problem Lack of exact sampling methods for diffusion models.
method Sequential Monte Carlo (SMC) algorithm with twisting technique.
result TDS offers more accurate approximations with fewer particles compared to heuristics.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
New metric compares noisy neural trajectories using optimal transport.
problem Existing metrics fail to capture differences in noisy, dynamic neural responses.
method Proposed an optimal transport distance metric for Gaussian processes.
result Metric effectively compares neural dynamics in different systems.
Generative model improves safety in self-driving simulators and human motion generation.
problem Improving generative models for constrained domains like safety-critical applications.
method Developed Gen-neG, a denoising diffusion model that uses oracle-assisted guidance.
result Empirically validated Gen-neG for collision avoidance and safety-guarded human motion generation.
Unified model improves sampling speed and quality.
problem Difficult to balance sampling speed and quality.
method Multistep Consistency Models combining consistency and diffusion models.
result Improved sampling quality with reduced steps.
Study evaluates ensemble methods for zero-shot uncertainty quantification with diffusion models.
problem Quantifying uncertainty in zero-shot regression problems using diffusion models.
method Used diffusion probabilistic models for ensemble prediction and evaluated their effectiveness on various regression tasks.
result Ensemble methods consistently improve model prediction accuracy across different regression tasks.
Unified framework reduces NFEs for inverse problems.
problem High computational costs and degraded reconstruction quality in existing LDM-based inverse solvers.
method Consistency Regularised Gradient Flows for posterior sampling and prompt optimization.
result Significantly reduced computational cost with state-of-the-art performance.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
Develops a framework for multi-objective learning in diffusion models with limited labeled data.
problem Achieving good trade-offs in multi-objective learning with diffusion models requires a generalist model class with larger capacity than individual tasks.
method Proposes a two-stage training procedure: first fitting specialist models from limited paired data, then distilling them into a generalist model.
result Establishes generalization bounds showing the number of paired samples depends only on specialist model complexity.
New method uses off-the-shelf classifiers to improve diffusion generation without extra training.
problem Improving sample quality and controllability in conditional diffusion generation.
method Leveraging off-the-shelf classifiers in a training-free fashion with calibration and pre-conditioning techniques.
result Significant performance improvements (up to 20%) over existing guidance schemes.
A commonly used evaluation metric for text-to-image synthesis is the Inception score (IS) \cite{inceptionscore}, which has been shown to be a quality metric that correlates well with human judgment. However, IS does not reveal properties of the generated images indicating the ability of a text-to-image synthesis method…
New research shows CFG improves high-dimensional data generation.
problem Characterizing CFG's effect on high-dimensional distributions.
method High-dimensional analysis of CFG's impact on target distributions.
result CFG accurately reproduces the target distribution in high dimensions.
Measures mode separation in high-dimensional densities via a reversible diffusion process.
problem Quantifying how sharply a distribution fragments into barrier-separated clusters in high dimensions.
method A unique reversible diffusion process with f as stationary distribution, extracting SSA and DA from its autocovariance matrix.
result Empirical autocovariance spectrum and readouts (SSA, DA) quantify mode separation using only samples and pretrained score-based models.
Text-to-Image translation has been an active area of research in the recent past. The ability for a network to learn the meaning of a sentence and generate an accurate image that depicts the sentence shows ability of the model to think more like humans. Popular methods on text to image translation make use of Generativ…
DataInf efficiently approximates data influence in large models, improving transparency and identifying mislabeled data.
problem Efficiently estimating data influence in large-scale models like LoRA-tuned LLMs and diffusion models.
method DataInf uses a closed-form expression to approximate influence scores efficiently.
result DataInf outperforms existing methods in computational and memory efficiency, accurately identifying influential data points.
A new sampling method called Restart improves both speed and quality of generative processes.
problem Balancing speed and quality in generative processes involving differential equations.
method Alternates between adding noise and following ODE, improving both speed and quality.
result Surpasses previous SDE and ODE samplers in both speed and accuracy.
Paper explores how Rectified Flow adapts to low-dimensional data.
problem Improving sampling efficiency in low-dimensional data.
method Investigates Rectified Flow's adaptation to low-dimensional support and introduces a stochastic version.
result Shows improved sampling efficiency with O(k/ε) complexity. LVTINO improves high-definition video restoration with consistent temporal details.
problem Restoring high-definition video with fine spatial detail and temporal consistency.
method LVTINO uses Video Consistency Models to bypass frame-by-frame image-based LDMs, achieving perceptual quality and computational efficiency.
result Significant perceptual improvements over current methods, achieving state-of-the-art quality with minimal evaluations.