New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
The paper analyzes CFG in masked diffusion models and proposes a new method to improve sample quality.
problem Improving sample quality in conditional generation with CFG in masked diffusion models.
method Analyzing CFG's effect in a low-dimensional masked diffusion model, revealing early-stage guidance harms quality and late-stage improves it. Proposed a novel method to smooth transitions between distributions.
result The proposed method improves sample quality via a simple one-line code change.
A new method learns continuous guidance weights to improve diffusion model quality and distributional alignment.
problem Improving perceptual quality and distributional alignment of samples from conditional diffusion models.
method Learned continuous guidance weights ωc,(s,t) are used to minimize distributional mismatch and reward guided sampling. result Improvements in Fréchet inception distance (FID) for image generation and better image-prompt alignment in text-to-image applications.
GH-PID uses guided harmonic paths for efficient SOT with interpretable diagnostics.
problem Efficiently solving Stochastic Optimal Transport with hard terminal distributions and soft costs.
method Guided Harmonic Path-Integral Diffusion (GH-PID) framework with low-dimensional guidance.
result GH-PID generates geometry-aware, cost-reducing trajectories that match terminal distributions.
Recent advances in matrix completion enable data imputation in full-rank matrices by exploiting low dimensional (nonlinear) latent structure. In this paper, we develop a new model for high rank matrix completion (HRMC), together with batch and online methods to fit the model and out-of-sample extension to complete new …
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
Framework for training-free guidance in discrete diffusion models for molecular generation.
problem No equivalent training-free guidance methods for discrete diffusion models.
method Framework using guidance functions for discrete data.
result Demonstrated utility on molecular graph generation tasks.
Exact guidance for discrete data improves posterior sampling efficiency.
problem Inefficient guidance for discrete data in posterior sampling.
method Derive exact transition rate for desired distribution given learned discrete flow matching model.
result Significantly improved efficiency with single forward pass per sampling step.
Diffusion models improve sample quality with guidance, proving it works for general data distributions.
problem Theoretical understanding of guidance effect in diffusion models for general data distributions.
method Analyzing diffusion guidance under general data distributions, proving improvement in sample quality.
result Guidance improves the average reciprocal of the classifier probability, aligning with its motivation.
Research shows guidance in diffusion models does not sample from intended distribution, affecting boundary sampling.
problem Clarifying the misconception that guidance modifies the data distribution in diffusion models.
method Rigorous proof and fine-grained analysis of guidance dynamics in two cases: mixtures of compactly supported distributions and mixtures of Gaussians.
result Guidance leads to sampling more heavily from the boundary of the support of the conditional distribution as the parameter increases.
Regression problems assume every instance is annotated (labeled) with a real value, a form of annotation we call \emph{strong guidance}. In order for these annotations to be accurate, they must be the result of a precise experiment or measurement. However, in some cases additional \emph{weak guidance} might be given by…
Improved image quality in diffusion models by limiting guidance to a specific noise level range.
problem Improving image quality in diffusion models with guidance.
method Restricting guidance to a specific noise level range.
result Significantly improved FID score from 1.81 to 1.40 in ImageNet-512.
Unified approach to guided generation techniques.
problem Controlling the generative process of flow/diffusion models.
method Unified posterior and end-to-end guidance techniques.
result Unified posterior guidance as a greedy strategy of end-to-end guidance.
Jeffrey guidance extends diffusion-model control to more complex applications.
problem Controlling diffusion models beyond simple cases like conditional sampling.
method Leveraging Jeffrey's rule of conditioning to update marginal distributions towards a target distribution.
result Significant reductions in FID on CIFAR-10 and FFHQ with Inception embeddings as the target.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
New study on guidance in masked diffusion models, showing how it shapes sampling dynamics.
problem Understanding how guidance influences the sampling behavior of masked diffusion models.
method Derived explicit solution to guided reverse dynamics, analyzing effects in 1D and 2D.
result Guidance amplifies class-specific regions and suppresses shared regions, affecting covariance structures.
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
New method controls classifier guidance in diffusion models.
problem Improving classifier guidance in diffusion models.
method Cross-entropy control of classifier gradients.
result Effective guidance vectors with mean squared error O(dε). New method guides pretrained diffusion models without additional training.
problem Guidance methods for diffusion models often require extra training or are task-specific.
method Variational Control using Diffusion Trajectory Matching (DTM)
result Achieves state-of-the-art results on various problems.
Gradient guidance improves diffusion models for optimizing specific objectives.
problem Improving diffusion models for specific optimization tasks.
method Established a mathematical framework for gradient-guided diffusion, linking it to optimization theory. Developed a modified gradient guidance method and iteratively fine-tuned version.
result Gradient-guided diffusion models are essentially solutions to regularized optimization problems, preserving latent structure.
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
SFG improves on-manifold sampling without labels or additional training.
problem Guiding score-based models on manifolds without labeled data or extra training.
method Developed saddle-free guidance (SFG) that uses curvature of log density estimates.
result SFG achieves state-of-the-art metrics in image generation without labeled data or additional training.
Theoretical analysis of t-SNE for visualizing clustered data.
problem Understanding t-SNE for visualizing high-dimensional clustered data.
method Gradient descent approach and power iterations based on graph Laplacian.
result Asymptotic equivalence and limiting behavior of t-SNE's early exaggeration stage.
Proposes a thermodynamic work minimization framework for guiding generative models.
problem Guiding generative models in sparse-data regimes with limited target samples or constraints.
method Regularization framework inspired by thermodynamic work, introducing Path Guidance and Observable Guidance.
result Improves sample efficiency and reduces bias in molecular simulations.
Develops sparse portfolio strategy for high-dimensional assets.
problem Sparse wealth allocations in high dimensions are limited by existing approaches.
method Establishes theoretical bounds and empirical analysis of sparse weight estimators.
result Sparse portfolios are robust to recessions and can be used as a hedging vehicle.
Certified guidance ensures generative models always meet planning objectives.
problem Lack of guarantees for generated outputs satisfying planning objectives.
method Certified guidance modifies generative models without retraining, ensuring outputs meet STL specifications.
result Certified guidance produces models that are always correct, unlike non-certified methods.
New method improves training-free guidance for diffusion models, achieving state-of-the-art results.
problem Accurate, training-free guidance for conditional generation in diffusion models.
method Sequential Monte Carlo (SMC) framework with Multi-Level Monte Carlo (MLMC) variance reduction.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet datasets with significant cost reduction.
A new method for offline RL using diffusion models and self-weighted guidance.
problem Challenges in computing scores for offline RL using weight functions.
method Constructing a diffusion over actions and weights, using the diffusion model for guidance.
result Performs on par with state-of-the-art methods on challenging environments.
This paper proposes a new algorithm for learning guidance rewards in RL.
problem Long-term temporal credit assignment in sparse or delayed reward environments.
method Surrogate RL objective with trajectory-space smoothing to learn guidance rewards.
result Guidance rewards can be learned without additional neural networks and have intuitive interpretation.
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
Boost-and-Skip generates minority samples without guidance, faster and more effectively.
problem Generating minority samples in low-density regions of a data manifold.
method Boost-and-Skip approach with variance-boosted initialization and timestep skipping.
result Boost-and-Skip effectively promotes the emergence of underrepresented minority features.
System guides freehand obstetric ultrasound probe movements.
problem Reducing operator expertise for standard plane acquisition.
method Artificial neural network trained on real scans.
result 88.8% accuracy for goal prediction, 90.9% for action prediction.
New method uses off-the-shelf classifiers to improve diffusion generation without extra training.
problem Improving sample quality and controllability in conditional diffusion generation.
method Leveraging off-the-shelf classifiers in a training-free fashion with calibration and pre-conditioning techniques.
result Significant performance improvements (up to 20%) over existing guidance schemes.
NegToMe uses images to guide text-based models away from unwanted visual elements.
problem Insufficient text-based adversarial guidance for complex visual concepts.
method Negative token merging (NegToMe) using visual features from reference images.
result Significantly enhances output diversity and reduces visual similarity to copyrighted content.
Discriminator guidance improves autoregressive diffusion models for generating molecular graphs.
problem Improving the accuracy of autoregressive diffusion models for generating molecular graphs.
method Deriving ways to use a discriminator with a pretrained generative model in the discrete case, including optimal and sub-optimal scenarios.
result Using a discriminator can correct pretrained models and improve exact sampling from the data distribution.
LEAQI learns to query an expert less often for noisy guidance.
problem Active imitation learning with noisy guidance to reduce expert query complexity.
method Developed LEAQI, a difference classifier that predicts expert disagreement with noisy heuristic.
result Significantly fewer queries to the expert with comparable or better accuracy.
Action guidance helps agents learn true objectives in games with sparse rewards.
problem Training agents in games with sparse rewards requires significant exploration.
method Action guidance, a novel technique that combines exploration with reward shaping.
result Action guidance enables agents to optimize true objectives efficiently.
New research shows CFG improves high-dimensional data generation.
problem Characterizing CFG's effect on high-dimensional distributions.
method High-dimensional analysis of CFG's impact on target distributions.
result CFG accurately reproduces the target distribution in high dimensions.
A method for disentangling text representations without supervision.
problem Challenges in learning disentangled representations of natural language.
method Information-theoretic guidance to induce independent style and content embeddings.
result High quality disentangled representations in terms of content and style preservation.
A new method for sampling from posterior distributions in Bayesian inverse problems.
problem Sampling from posterior distributions in Bayesian inverse problems is challenging due to intractable terms.
method Proposes a novel approach that decomposes the transitions, allowing a trade-off between complexity of guidance term and prior transitions.
result Validated through experiments on various inverse problems, including challenging cases with latent diffusion models as priors.
Improved image translation using asymmetric gradient guidance.
problem Trade-off between style transformation and content preservation in diffusion models.
method Asymmetric gradient guidance to guide reverse diffusion sampling.
result Our method outperforms state-of-the-art models in image translation tasks.
Discrete diffusion models improve text and image inference.
problem Challenges in posterior sampling with discrete diffusion models.
method Anchored Posterior Sampling (APS) with quantized expectation and anchored remasking.
result APS achieves state-of-the-art performance on various tasks.
New framework improves variational inference for high-dimensional posteriors.
problem Challenges in choosing variational objectives and approximating families for high-dimensional posteriors.
method Conceptual framework and experimental tools to understand and optimize variational objectives and families.
result For moderate-to-high-dimensional posteriors, exclusive KL divergence is recommended due to optimization ease; for low-dimensional, heavy-tailed variational families are effective.
Novel alignment framework for text-to-image generation using diffusion models and flow matching.
problem Improving text-to-image generation with minimal computational resources.
method Proposes a novel alignment framework that decomposes the score function into pre-trained score plus a conditional expectation of the reward.
result Achieves comparable performance to finetuning-based models with reduced computational cost.
Unsupervised discovery of latent representations, in addition to being useful for density modeling, visualisation and exploratory data analysis, is also increasingly important for learning features relevant to discriminative tasks. Autoencoders, in particular, have proven to be an effective way to learn latent codes th…
High-dimensional diffusion models suffer from distorted samples due to CFG.
problem Distortions in high-dimensional guided diffusion models.
method Analytical tools from statistical physics, dynamic mean-field theory.
result Distortions arise in high-dimensional settings due to class separability issues.
New method distills cloud models into edge-friendly ones.
problem Cloud-to-edge model compression with limited data exchange.
method Two-step workflow of deprivatization and distillation.
result Outperforms previous state-of-the-art approaches on various benchmarks.