The paper evaluates biased methods for alpha-divergence minimization.
problem The impact of bias on solutions found for alpha-divergence minimization.
method Empirical evaluation of biased methods for alpha-divergence minimization, focusing on bias effects and dimensionality.
result Solutions are biased towards KL-divergence minimizers and require impractical computation in high dimensions to minimize alpha-divergence.
Unbiased methods for alpha-divergence minimization struggle in high dimensions.
problem The difficulty of unbiased alpha-divergence minimization in high dimensions.
method Signal-to-Noise Ratio (SNR) analysis of gradient estimators.
result The SNR of the gradient estimator worsens exponentially with dimensionality.
This research explores using Alpha-Divergences in variational dropout for better inference.
problem Improving variational inference methods using alternative divergences.
method Extending the Stochastic Gradient Variational Bayes (SGVB) framework with Alpha-Divergences.
result The α-divergence with αightarrow1 yields the lowest training error and optimizes the ELBO. The paper presents methods to improve uncertainty calibration in Bayesian Neural Networks.
problem Uncalibrated Bayesian Neural Networks often lead to overconfidence.
method The paper uses alpha-divergences from Information Geometry for calibration.
result Calibration using alpha-divergences provides better uncertainty estimates and is more efficient.
AlphaNet improves supernets training with alpha-divergence.
problem Improving the uncertainty distillation in weight-sharing NAS.
method Proposes alpha-divergence for better uncertainty distillation in supernets.
result Significant improvements in model performance across various FLOPs regimes.
Improved dropout inference for Bayesian neural networks using alpha-divergences.
problem Uncertainty underestimation in dropout variational inference.
method Proposed a re-parametrisation of alpha-divergence objectives for dropout networks.
result Improved uncertainty estimates and accuracy compared to VI in dropout networks.
Paper formalizes and analyzes a new bound for variational inference.
problem Lack of theoretical guarantees in variational algorithms.
method Introduces VR-IWAE bound, a generalization of IWAE.
result VR-IWAE bound leads to unbiased gradient estimators.
New α-divergence loss function improves neural density ratio estimation.
problem Optimization challenges in existing DRE methods, especially overfitting and high sample requirements.
method Derived α-divergence loss function (α-Div) for neural density ratio estimation. result The α-divergence loss function (α-Div) offers stable and effective optimization for DRE. New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two…
This paper studies geometrical structure of the manifold of escort probability distributions and shows its new applicability to information science. In order to realize escort probabilities we use a conformal transformation that flattens so-called alpha-geometry of the space of discrete probability distributions, which…
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…
New divergences extend Bregman and skew Jensen, including f-divergences.
problem Developing new divergences to include f-divergences.
method Introducing g-Bregman and skew g-Jensen divergences, showing they include f-divergences.
result g-divergences generalize existing divergences and inequalities.
New algorithms optimize nonlinear Kalman filtering using divergence measures.
problem Nonlinear Kalman filtering without closed-form solutions.
method Proposes novel algorithms for KL and α-divergence optimization.
result Improves performance on radar and sensor tracking problems.
The paper shows how policy regularization acts like an adversary to improve robustness.
problem Improving robustness of learned policies in reinforcement learning.
method Using convex duality, the paper characterizes adversarial reward perturbations and provides generalization guarantees.
result Policy regularization acts as an adversary to improve robustness against worst-case reward perturbations.
This paper examines methods to estimate uncertainty in neural networks for dialogue policy optimization.
problem Efficient exploration in neural network-based dialogue policy optimization.
method Extensive benchmark of deep Bayesian methods to extract uncertainty estimates from DQN.
result Combining uncertainty estimation methods with DQN improves sample efficiency and user experience.
Improved Bayesian inference using power priors with historical data.
problem Improving Bayesian inference with historical data.
method Generalized power priors that adapt to the α parameter of Amari's α-divergence. result Improved performance through appropriate choices of the α parameter. Estimates Markov chains from samples, solving two related prediction and estimation problems.
problem Estimating an unknown Markov chain from its samples.
method Considered two problems: predicting conditional distribution and estimating transition matrix, using KL-divergence and various f-divergences. result Resolved estimation problem for all sufficiently smooth f-divergences, including KL-, L2, Chi-squared, Hellinger, and Alpha-divergences.