Jeffrey guidance extends diffusion-model control to more complex applications.
problem Controlling diffusion models beyond simple cases like conditional sampling.
method Leveraging Jeffrey's rule of conditioning to update marginal distributions towards a target distribution.
result Significant reductions in FID on CIFAR-10 and FFHQ with Inception embeddings as the target.
Jeffreys Flow improves robustness of Boltzmann generators for rare event sampling.
problem Rare events and metastable trapping in sampling physical systems with rough energy landscapes.
method Introduces Jeffreys Flow, a robust generative framework using Parallel Tempering distillation and symmetric Jeffreys divergence to mitigate mode collapse and improve mode coverage.
result Minimizing Jeffreys divergence suppresses mode collapse and corrects inaccuracies in multi-modal distributions.
Paper proves Jeffrey's update rule minimizes relative entropy.
problem Improving Bayesian learning algorithms.
method More concise proof of Jeffrey's update rule.
result Jeffrey's update rule reduces relative entropy.
Due to the success of the bag-of-word modeling paradigm, clustering histograms has become an important ingredient of modern information processing. Clustering histograms can be performed using the celebrated k-means centroid-based algorithm. From the viewpoint of applications, it is usually required to deal with symm…
New method samples Jeffreys prior for objective Bayesian inference.
problem Sampling from Jeffreys prior is challenging.
method Metropolis-Adjusted Langevin Algorithm
result Samples can be directly used in Bayesian methods.
Bayesian neural networks update beliefs with soft evidence, improving accuracy and calibration.
problem Updating neural network weights with uncertain or soft evidence.
method Developed two algorithms to approximate Jeffrey's rule for updating neural network weights.
result Jeffrey-based methods outperform traditional approaches in accuracy and calibration, especially in noisy data.
New optimal prior avoids bias in complex models with limited data.
problem Bias in inference from limited data using Jeffreys prior.
method Developed a principled choice of measure that avoids bias, dependent on data quantity.
result Optimal prior leads to unbiased inference in complex models.
Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques fo…
A new Weyl prior is proposed for Bayesian statistics, offering a more canonical choice for parameter α.
problem Choosing a prior distribution for Bayesian inference.
method Proposed a new Weyl prior based on the Weyl structure on a statistical manifold.
result The Weyl prior is a special case of the α-parallel prior with α = -n, where n is the dimension of the statistical manifold.
Optimality of TS with noninformative priors proven for Pareto model.
problem Optimality of Thompson Sampling with noninformative priors for Pareto bandits.
method Proved optimality of TS with certain probability matching priors, showed suboptimality with others, and found effectiveness of truncation procedures.
result TS with certain probability matching priors achieves optimal regret bound for Pareto model.
Consider the space RΔ of rational functions of several variables with poles on a fixed arrangement Δ of hyperplanes. We obtain a decomposition of RΔ as a module over the ring of differential operators with constant coefficients. We generalize to the space RΔ the notions of principal part and of residue, and …
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …
We study online learning under logarithmic loss with regular parametric models. Hedayati and Bartlett (2012b) showed that a Bayesian prediction strategy with Jeffreys prior and sequential normalized maximum likelihood (SNML) coincide and are optimal if and only if the latter is exchangeable, and if and only if the opti…
This work formalizes guidance in diffusion models and introduces a stochastic control framework.
problem Lack of a solid theoretical foundation for guidance scheduling in diffusion models.
method Introduces a stochastic optimal control framework to cast guidance scheduling as an adaptive optimization problem.
result Establishes a principled foundation for more effective guidance in diffusion models.
Automated feature selection is important for text categorization to reduce the feature size and to speed up the learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity…
Framework for training-free guidance in discrete diffusion models for molecular generation.
problem No equivalent training-free guidance methods for discrete diffusion models.
method Framework using guidance functions for discrete data.
result Demonstrated utility on molecular graph generation tasks.
Exact guidance for discrete data improves posterior sampling efficiency.
problem Inefficient guidance for discrete data in posterior sampling.
method Derive exact transition rate for desired distribution given learned discrete flow matching model.
result Significantly improved efficiency with single forward pass per sampling step.
This study investigates self-supervised learning with Wasserstein distance on tree structures.
problem Improving self-supervised learning methods using Wasserstein distance.
method Utilized Tree-Wasserstein distance (TWD) and Jeffrey divergence regularization for training.
result A simple combination of softmax function and Tree-Wasserstein distance outperforms cosine similarity-based methods.
Diffusion models improve sample quality with guidance, proving it works for general data distributions.
problem Theoretical understanding of guidance effect in diffusion models for general data distributions.
method Analyzing diffusion guidance under general data distributions, proving improvement in sample quality.
result Guidance improves the average reciprocal of the classifier probability, aligning with its motivation.
The paper explores how to handle uncertain evidence in probabilistic models.
problem Handling uncertain evidence in probabilistic models and stochastic simulators.
method The paper considers distributional evidence, Jeffrey's rule, and virtual evidence as methods for interpreting uncertain evidence.
result The paper provides guidelines on how to account for uncertain evidence and highlights the importance of careful consideration.
MsIGN tackles high-dimensional Bayesian inference using multiscale structure.
problem High-dimensional Bayesian inference challenges due to the curse of dimensionality.
method MsIGN generates samples from coarse to fine scale, minimizing Jeffreys divergence.
result MsIGN outperforms previous approaches in posterior approximation and mode capture.
Research shows guidance in diffusion models does not sample from intended distribution, affecting boundary sampling.
problem Clarifying the misconception that guidance modifies the data distribution in diffusion models.
method Rigorous proof and fine-grained analysis of guidance dynamics in two cases: mixtures of compactly supported distributions and mixtures of Gaussians.
result Guidance leads to sampling more heavily from the boundary of the support of the conditional distribution as the parameter increases.
Regression problems assume every instance is annotated (labeled) with a real value, a form of annotation we call \emph{strong guidance}. In order for these annotations to be accurate, they must be the result of a precise experiment or measurement. However, in some cases additional \emph{weak guidance} might be given by…
Improved image quality in diffusion models by limiting guidance to a specific noise level range.
problem Improving image quality in diffusion models with guidance.
method Restricting guidance to a specific noise level range.
result Significantly improved FID score from 1.81 to 1.40 in ImageNet-512.
Unified approach to guided generation techniques.
problem Controlling the generative process of flow/diffusion models.
method Unified posterior and end-to-end guidance techniques.
result Unified posterior guidance as a greedy strategy of end-to-end guidance.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
New study on guidance in masked diffusion models, showing how it shapes sampling dynamics.
problem Understanding how guidance influences the sampling behavior of masked diffusion models.
method Derived explicit solution to guided reverse dynamics, analyzing effects in 1D and 2D.
result Guidance amplifies class-specific regions and suppresses shared regions, affecting covariance structures.
Develops a statistical test for IV, improving feature selection reliability.
problem Lack of statistical justification in conventional IV-based feature selection.
method Establishes connection with Jeffreys divergence and proposes a nonparametric hypothesis test.
result The J-Divergence test provides rigorous guarantees and is more reliable than traditional IV thresholds.
The paper analyzes CFG in masked diffusion models and proposes a new method to improve sample quality.
problem Improving sample quality in conditional generation with CFG in masked diffusion models.
method Analyzing CFG's effect in a low-dimensional masked diffusion model, revealing early-stage guidance harms quality and late-stage improves it. Proposed a novel method to smooth transitions between distributions.
result The proposed method improves sample quality via a simple one-line code change.
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
We announce the following result and give several applications: A Hamiltonian T-space (for T a torus) with isolated fixed points is cobordant to a disjoint union of weighted projective spaces which are constructed from its fixed point data. The applications concern the Duistermaat-Heckman formula, the topological J…
New method controls classifier guidance in diffusion models.
problem Improving classifier guidance in diffusion models.
method Cross-entropy control of classifier gradients.
result Effective guidance vectors with mean squared error O(dε). New method guides pretrained diffusion models without additional training.
problem Guidance methods for diffusion models often require extra training or are task-specific.
method Variational Control using Diffusion Trajectory Matching (DTM)
result Achieves state-of-the-art results on various problems.
Gradient guidance improves diffusion models for optimizing specific objectives.
problem Improving diffusion models for specific optimization tasks.
method Established a mathematical framework for gradient-guided diffusion, linking it to optimization theory. Developed a modified gradient guidance method and iteratively fine-tuned version.
result Gradient-guided diffusion models are essentially solutions to regularized optimization problems, preserving latent structure.
Jeffrey and Kirwan suggested expressions for intersection pairings on the reduced space of a Hamiltonian G-space in terms of multiple residues. In this paper we prove a residue formula for symplectic volumes of reduced spaces of a quasi-Hamiltonian SU(2)-space. The definition of quasi-Hamiltonian G-spaces was recently …
Self-guidance controls image generation by extracting properties from diffusion model representations.
problem Generating images from text descriptions is challenging due to the complexity of visual details.
method Self-guidance uses internal representations of diffusion models to control image generation.
result Properties like object shape, location, and appearance can be extracted and used to steer image generation.
SFG improves on-manifold sampling without labels or additional training.
problem Guiding score-based models on manifolds without labeled data or extra training.
method Developed saddle-free guidance (SFG) that uses curvature of log density estimates.
result SFG achieves state-of-the-art metrics in image generation without labeled data or additional training.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
Proposes a thermodynamic work minimization framework for guiding generative models.
problem Guiding generative models in sparse-data regimes with limited target samples or constraints.
method Regularization framework inspired by thermodynamic work, introducing Path Guidance and Observable Guidance.
result Improves sample efficiency and reduces bias in molecular simulations.
We show that if the connected sum of two knots with coprime Alexander polynomials is doubly slice, then the Ozsváth-Szabó correction terms as smooth double sliceness obstructions vanish for both knots. Recently, Jeffrey Meier gave smoothly slice knots that are topologically doubly slice, but not smoothly doubly slice. …
Certified guidance ensures generative models always meet planning objectives.
problem Lack of guarantees for generated outputs satisfying planning objectives.
method Certified guidance modifies generative models without retraining, ensuring outputs meet STL specifications.
result Certified guidance produces models that are always correct, unlike non-certified methods.
New method improves training-free guidance for diffusion models, achieving state-of-the-art results.
problem Accurate, training-free guidance for conditional generation in diffusion models.
method Sequential Monte Carlo (SMC) framework with Multi-Level Monte Carlo (MLMC) variance reduction.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet datasets with significant cost reduction.
A new method for offline RL using diffusion models and self-weighted guidance.
problem Challenges in computing scores for offline RL using weight functions.
method Constructing a diffusion over actions and weights, using the diffusion model for guidance.
result Performs on par with state-of-the-art methods on challenging environments.
Let M be a smooth manifold and G a compact connected Lie group acting on M by isometries. In this paper, we study the equivariant cohomology of X=T∗M, and relate it to the cohomology of the Marsden-Weinstein reduced space via certain residue formulae. In case that X is a compact symplectic mani…
Using the notion of equivariant Kirwan map, as defined by Goldin, we prove that -- in the case of Hamiltonian torus actions with isolated fixed points -- Tolman and Weitsman's description of the kernel of the Kirwan map can be deduced directly from the residue theorem of Jeffrey and Kirwan. A characterization of the ke…
Bayesian method improves extreme quantile estimation with zero coverage error.
problem Estimating extreme quantiles with zero coverage error in small samples.
method Bayesian quantile estimation using Jeffreys prior.
result Bayesian method results in zero coverage error, unlike maximum likelihood.
We construct geometric shrinkage priors for Kählerian signal filters. Based on the characteristics of Kähler manifolds, an efficient and robust algorithm for finding superharmonic priors which outperform the Jeffreys prior is introduced. Several ansätze for the Bayesian predictive priors are also suggested. In particul…
This paper proposes a new algorithm for learning guidance rewards in RL.
problem Long-term temporal credit assignment in sparse or delayed reward environments.
method Surrogate RL objective with trajectory-space smoothing to learn guidance rewards.
result Guidance rewards can be learned without additional neural networks and have intuitive interpretation.