Improved fast rates for decision making with forward-KL regularization in contextual bandits.
problem Improving fast rates for decision making with forward-KL regularization in contextual bandits.
method Streamlined analysis of forward-KL-regularized offline CBs, exploiting the pessimism principle and convex-analytical pipeline.
result First ildeO(ε−1) upper bounds in tabular and general function approximation settings. TSC uses HMC and adaptive transport maps to optimize forward KL for variational inference.
problem Variational inference underestimates uncertainty when minimizing reverse KL.
method TSC uses Hamiltonian Monte Carlo and adaptive transport maps to optimize KL(p||q).
result TSC achieves competitive performance in training variational autoencoders on large-scale data.
VISA improves inference efficiency for complex models.
problem Efficient approximate inference in computationally intensive models.
method Sequential sample-average approximations within a trust region.
result VISA achieves comparable accuracy with computational savings.
We quantify forgetting in post-training models, distinguishing mass and drift.
problem Understanding and preventing forgetting in post-training generative models.
method Developed theoretical results under a two-mode mixture abstraction, formalizing mass and drift forgetting.
result Forgetting can be precisely quantified based on divergence direction, geometric overlap, and training regime.
Paper proposes f-DPG for aligning language models with preferences.
problem Aligning language models with user preferences.
method Uses f-divergence to approximate target distributions and minimizes a forward KL from it using DPG.
result Jensen-Shannon divergence often outperforms forward KL divergence, leading to significant improvements.
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
problem Estimating KL divergences for normalizing flows efficiently and accurately.
method Path-gradient estimators for reverse and forward KL divergences.
result Path-gradient estimators lead to faster convergence and better approximation results.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
A new method combines VI and IS to improve Bayesian inference accuracy.
problem Bayesian inference often underestimates posterior tails, leading to miscalibration and degeneracy.
method Proposes a novel combination of optimization and sampling techniques using the forward KL divergence.
result The method guarantees asymptotic consistency and fast convergence to optimal IS and variational approximations.
A new framework predicts hidden Markov model regimes online.
problem Efficiently identify hidden Markov model regimes in streaming data.
method Develops a predictive-first optimisation framework for streaming HMMs, approximating the full posterior predictive distribution.
result The method provides competitive prequential performance compared to Online EM and Sequential Monte Carlo.
Unified framework for unlearning in diffusion models using KL divergence and likelihood constraints.
problem Removing undesirable data or concepts while preserving utility of pretrained models.
method Constrained optimization framework based on reverse and forward KL divergences, and likelihood constraints.
result Our KL-constrained approach achieves superior retention-unlearning tradeoffs compared to weight-based baselines.
Unified framework for efficient trans-dimensional Bayesian inference using VI and NFs.
problem Efficient trans-dimensional Bayesian inference with reduced computational cost.
method Variational inference with normalizing flows to train transport proposals.
result Our approach minimizes reverse KL divergence and reduces computational cost.
The paper proves learning-curve monotonicity for maximum likelihood estimators in various parametric settings.
problem Establishing monotonicity guarantees for maximum likelihood estimators.
method Variants of GPT-5.2 Pro were used to derive the results.
result The paper proves monotonicity for maximum likelihood estimators in Gaussian and Gamma variables.
New method improves variational inference for better posterior approximation.
problem Challenges in minimizing inclusive KL divergence for amortized variational inference.
method Likelihood-tempered sequential Monte Carlo samplers to estimate inclusive KL gradient.
result SMC-Wake method fits variational distributions more accurately than existing methods.
New method uses neural networks to efficiently approximate Bayesian inference for complex models.
problem Efficiently approximating Bayesian inference for complex models with varying temperatures.
method Fully amortized neural posterior estimator trained on a single forward pass.
result Achieves competitive posterior approximations across various temperatures and benchmarks.
Variational inference improves training of generative flow networks.
problem Training generative flow networks efficiently and accurately.
method Define variational objectives in terms of KL divergences and optimize convex combinations.
result Variational inference methods can reduce the variance of gradients in training generative flow networks.
This work develops a unified framework for RLHF with general f-divergence regularization.
problem Theoretical understanding of general f-divergence regularization in RLHF. method Holistic approach across f-divergence class, two algorithms based on distinct sampling principles. result Provably efficient algorithms with O(logT) regret and O(1/T) sub-optimality gap. This paper solves the multiple reference model problem in RLHF with exact solutions and sample complexity guarantees.
problem Limitations of single reference models in aligning LLMs with human feedback.
method Integrates multiple reference models into RLHF frameworks, addressing theoretical challenges with exact solutions and sample complexity guarantees.
result First exact solution to the multiple reference model problem in reverse KL-regularized RLHF.
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.
New methods minimize GFlowNet training divergences for better sampling.
problem Training GFlowNets with KL divergence leads to biased and high-variance estimators.
method Design and implement efficient estimators for four divergence measures.
result Properly minimizing these divergences yields a provably correct and effective training scheme.
This paper introduces f-DPO, a generalized approach to Direct Preference Optimization using diverse divergence constraints.
problem Aligning large language models with human preferences while mitigating safety risks.
method Incorporates diverse divergence constraints to simplify the relationship between reward and optimal policy, eliminating the need for estimating the normalizing constant.
result Optimizes LLMs to align with human preferences more efficiently and under a broader set of divergence constraints.
Unified theory for semi-implicit variational inference, bridging approximation and optimization.
problem Developing a statistical theory for semi-implicit variational inference.
method Unified theory combining approximation and optimization analyses.
result Unified theory characterizes SIVI's ability to recover target distributions and governs asymptotic behavior.
Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.