Improved fast rates for decision making with forward-KL regularization in contextual bandits.
problem Improving fast rates for decision making with forward-KL regularization in contextual bandits.
method Streamlined analysis of forward-KL-regularized offline CBs, exploiting the pessimism principle and convex-analytical pipeline.
result First ildeO(ε−1) upper bounds in tabular and general function approximation settings. TSC uses HMC and adaptive transport maps to optimize forward KL for variational inference.
problem Variational inference underestimates uncertainty when minimizing reverse KL.
method TSC uses Hamiltonian Monte Carlo and adaptive transport maps to optimize KL(p||q).
result TSC achieves competitive performance in training variational autoencoders on large-scale data.
Forward-Euler fails for simulating Wasserstein gradient flows with KL divergence.
problem Simulating Wasserstein gradient flows with forward-Euler discretization fails for KL divergence.
method Forward-Euler discretization for Wasserstein gradient flows with KL divergence.
result Forward-Euler discretization can be incorrect for Wasserstein gradient flows with KL divergence.
SRFE clarifies KL divergences without unifying learning frameworks.
problem Inductive biases of KL divergences and their limitations.
method Introducing SRFE, a log-moment-based functional of the likelihood ratio.
result SRFE recovers KL divergences as limits and reveals a mean-variance tradeoff.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
problem Estimating KL divergences for normalizing flows efficiently and accurately.
method Path-gradient estimators for reverse and forward KL divergences.
result Path-gradient estimators lead to faster convergence and better approximation results.
Variational inference improves training of generative flow networks.
problem Training generative flow networks efficiently and accurately.
method Define variational objectives in terms of KL divergences and optimize convex combinations.
result Variational inference methods can reduce the variance of gradients in training generative flow networks.
We quantify forgetting in post-training models, distinguishing mass and drift.
problem Understanding and preventing forgetting in post-training generative models.
method Developed theoretical results under a two-mode mixture abstraction, formalizing mass and drift forgetting.
result Forgetting can be precisely quantified based on divergence direction, geometric overlap, and training regime.
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
Paper proposes f-DPG for aligning language models with preferences.
problem Aligning language models with user preferences.
method Uses f-divergence to approximate target distributions and minimizes a forward KL from it using DPG.
result Jensen-Shannon divergence often outperforms forward KL divergence, leading to significant improvements.
VISA improves inference efficiency for complex models.
problem Efficient approximate inference in computationally intensive models.
method Sequential sample-average approximations within a trust region.
result VISA achieves comparable accuracy with computational savings.
Beta diffusion generates bounded data using multiplicative transitions.
problem Generating data within specific ranges.
method Integrates demasking and denoising with scaled and shifted beta distributions.
result KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
Unified framework for unlearning in diffusion models using KL divergence and likelihood constraints.
problem Removing undesirable data or concepts while preserving utility of pretrained models.
method Constrained optimization framework based on reverse and forward KL divergences, and likelihood constraints.
result Our KL-constrained approach achieves superior retention-unlearning tradeoffs compared to weight-based baselines.
New method improves variational inference for better posterior approximation.
problem Challenges in minimizing inclusive KL divergence for amortized variational inference.
method Likelihood-tempered sequential Monte Carlo samplers to estimate inclusive KL gradient.
result SMC-Wake method fits variational distributions more accurately than existing methods.
Improved analysis for diffusion models reduces KL divergence error dependence on data dimension and discretization step size.
problem Analyze the convergence of diffusion-based generative models under minimal assumptions.
method Model the generation process as a composition of reverse ODE and noising steps, leveraging Wasserstein-type error control and noise addition.
result Achieved a linear dependence on data dimension and improved dependence on discretization step size for KL divergence error.
We consider the nonlinear Kalman filtering problem using Kullback-Leibler (KL) and α-divergence measures as optimization criteria. Unlike linear Kalman filters, nonlinear Kalman filters do not have closed form Gaussian posteriors because of a lack of conjugacy due to the nonlinearity in the likelihood. In this paper …
This paper solves the multiple reference model problem in RLHF with exact solutions and sample complexity guarantees.
problem Limitations of single reference models in aligning LLMs with human feedback.
method Integrates multiple reference models into RLHF frameworks, addressing theoretical challenges with exact solutions and sample complexity guarantees.
result First exact solution to the multiple reference model problem in reverse KL-regularized RLHF.
Cryo-EM reconstruction is reformulated as a stochastic inverse problem to handle structural heterogeneity.
problem Handling structural heterogeneity in cryo-EM 3D reconstruction.
method Formulated as a stochastic inverse problem over probability measures, using variational discrepancy and Wasserstein gradient flow.
result Validated approach using synthetic examples, demonstrating recovery of continuous structural distributions.
Develops a method for learning proposals in nested importance samplers.
problem Improving sampling quality in complex distributions.
method Nested Variational Inference (NVI) using forward or reverse KL divergence.
result Optimizing nested objectives leads to improved sample quality.
A new method selects variables efficiently for fast and accurate dynamic system identification.
problem Efficiently selecting variables for scalable Gaussian processes.
method Forward variable selection using Karhunen-Loève decomposition and Gibbs sampling.
result Method yields competitive accuracies and inference times for dynamic systems.
The paper proves learning-curve monotonicity for maximum likelihood estimators in various parametric settings.
problem Establishing monotonicity guarantees for maximum likelihood estimators.
method Variants of GPT-5.2 Pro were used to derive the results.
result The paper proves monotonicity for maximum likelihood estimators in Gaussian and Gamma variables.
New methods minimize GFlowNet training divergences for better sampling.
problem Training GFlowNets with KL divergence leads to biased and high-variance estimators.
method Design and implement efficient estimators for four divergence measures.
result Properly minimizing these divergences yields a provably correct and effective training scheme.
A new geometric concept, the dead direction, bridges singular learning theory and information geometry.
problem The gap between singular learning theory and information geometry.
method Introducing the dead direction, a unit vector along degenerating Fisher metric, and showing its KL order can be recovered.
result The KL order of the dead direction can be recovered as the decay rate of the directional Fisher curvature, providing a handle on singular geometry.
Flow-based models generate data with improved theoretical guarantees.
problem Theoretical analysis of flow-based generative models.
method Proximal gradient descent in Wasserstein space for JKO flow model.
result KL guarantee of data generation by JKO flow model is O(ε2). Unified theory for semi-implicit variational inference, bridging approximation and optimization.
problem Developing a statistical theory for semi-implicit variational inference.
method Unified theory combining approximation and optimization analyses.
result Unified theory characterizes SIVI's ability to recover target distributions and governs asymptotic behavior.
Bayesian PINNs solve noisy PDE problems with physics constraints.
problem Uncertainty quantification in noisy PDE problems.
method Bayesian framework combining PINNs and HMC/VI for posterior estimation.
result HMC outperforms VI for noisy data.
Discrete diffusion models improve data generation for discrete data like language and graphs.
problem Adapting diffusion models to discrete state spaces for better data generation.
method Formulated as CTMCs, used uniformization of continuous Markov chains for sampling.
result Derive guarantees for sampling from any distribution on a hypercube, aligning with state-of-the-art achievements.
Proximal Diffusion Models improve generative model efficiency.
problem Improving generative model efficiency and accuracy.
method Developed Proximal Diffusion Models using proximal maps instead of scores.
result Proximal Diffusion Models achieve faster convergence and higher accuracy.
This paper introduces f-DPO, a generalized approach to Direct Preference Optimization using diverse divergence constraints.
problem Aligning large language models with human preferences while mitigating safety risks.
method Incorporates diverse divergence constraints to simplify the relationship between reward and optimal policy, eliminating the need for estimating the normalizing constant.
result Optimizes LLMs to align with human preferences more efficiently and under a broader set of divergence constraints.
Unified framework for efficient trans-dimensional Bayesian inference using VI and NFs.
problem Efficient trans-dimensional Bayesian inference with reduced computational cost.
method Variational inference with normalizing flows to train transport proposals.
result Our approach minimizes reverse KL divergence and reduces computational cost.
Paper analyzes inclusive KL inference using Wasserstein gradient flows.
problem Analyzing inclusive KL inference with mathematical tools.
method Gradient flows derived from PDE analysis.
result Unified view of existing sampling algorithms as inclusive-KL inference.
Bayesian neural networks improve uncertainty calibration without sacrificing accuracy.
problem Bayesian neural networks struggle with uncertainty calibration and high-dimensional geometry.
method Model uncertainty only in weight directions using a von Mises-Fisher posterior on the unit sphere, deriving a compact KL term.
result A lightweight, dimension-aware variational unit improves calibration without sacrificing accuracy.
Paper analyzes Langevin dynamics for multimodal Gaussian mixtures, controlling errors across dimensions.
problem Challenges in obtaining stable diffusion-based samplers in high- and infinite-dimensional settings.
method Study of preconditioned Annealed Langevin Dynamics (ALD) for Gaussian mixtures, focusing on Euler-Maruyama (EM) and exponential-integrator schemes.
result Proves dimension-uniform KL bounds for the exponential-integrator scheme, allowing arbitrarily small divergence with dimension.
A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB…
TREK uses distillation to help students solve hard problems.
problem Stalled progress on hard prompts when current policy lacks useful reasoning trajectories.
method TREK combines distillation and reinforcement learning to expand student support.
result TREK significantly improves student performance on mathematical reasoning and agentic tasks.
A new method combines VI and IS to improve Bayesian inference accuracy.
problem Bayesian inference often underestimates posterior tails, leading to miscalibration and degeneracy.
method Proposes a novel combination of optimization and sampling techniques using the forward KL divergence.
result The method guarantees asymptotic consistency and fast convergence to optimal IS and variational approximations.
This work develops a unified framework for RLHF with general f-divergence regularization.
problem Theoretical understanding of general f-divergence regularization in RLHF. method Holistic approach across f-divergence class, two algorithms based on distinct sampling principles. result Provably efficient algorithms with O(logT) regret and O(1/T) sub-optimality gap. Causal KL improves on existing metrics for evaluating causal models.
problem Insufficient discrimination between causal models using edit-distance and KL divergence.
method Introducing Causal KL, an augmented KL divergence that considers causal relationships.
result Causal KL variants effectively distinguish between observationally equivalent models.
New algorithm minimizes inclusive KL for VI, improving accuracy.
problem Improving variational inference accuracy with KL(p||q).
method Markovian score climbing (MSC) using stochastic gradients.
result MSC converges to local optimum of inclusive KL without bias.
Sharp analysis improves RLHF sample complexity with KL-regularization.
problem Improving RLHF sample complexity with KL-regularization.
method Sharp analysis of KL-regularized contextual bandits and RLHF.
result Achieved an O(1/ε) sample complexity when ε is sufficiently small.
Kurdyka-Lojasiewicz (KL) exponent plays an important role in estimating the convergence rate of many contemporary first-order methods. In particular, a KL exponent of 21 for a suitable potential function is related to local linear convergence. Nevertheless, KL exponent is in general extremely hard to estimate. I…
Tail-Safe hedging uses reinforcement learning with a safety layer to manage financial risks.
problem Managing financial risks in derivatives trading with robustness and explainability.
method Combines distributional reinforcement learning with a CBF-QP safety layer to enforce financial constraints.
result Improves risk management without degrading central performance and avoids hard constraint violations.
The paper develops new algorithms for KL-divergence NMF, proving convergence and performance.
problem Improving NMF for nonnegative data with KL divergence.
method Collect and analyze properties of KL objective function, propose and test new algorithms.
result Guaranteed non-increasing objective function for one proposed algorithm, global convergence.
New method guarantees global convergence in variational inference.
problem Limited convergence to local optima in variational inference.
method Minimizes inclusive KL divergence using neural networks and neural tangent kernel.
result Gradient descent dynamics converge to a unique solution in function space.
This paper tightens the law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
problem Developing nonasymptotic concentration bounds for empirical KL_inf with optimal constants and rates.
method Presenting a tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
result A tight law of the iterated logarithm for empirical KL_inf, applicable to unbounded data.
Paper analyzes and improves KL-regularized RL for LLMs with logarithmic regret.
problem Improving efficiency of RL fine-tuning for large language models.
method Optimism-based KL-regularized online contextual bandit algorithm with novel regret analysis.
result Achieves an O(ηlog(NRT)⋅dR) logarithmic regret bound. Paper relaxes triangle inequality for KL divergence between Gaussian distributions.
problem KL divergence does not satisfy triangle inequality for Gaussian distributions.
method Investigates relaxed triangle inequality and finds supremum.
result Supremum of KL divergence is found and conditions for attaining it are determined.
Theory for RLHF generalization under reward shift and clipped KL.
problem Theoretical understanding of RLHF generalization, especially with reward shift and clipped KL.
method Developed generalization theory for RLHF, accounting for reward shift and clipped KL.
result Presented generalization bounds for RLHF, suggesting generalization error from sampling, reward shift, and KL clipping.