A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper introduces a diagnostic for approximate inference methods.
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
MIM learns useful representations with high mutual information.
DAIS minimizes symmetrized KL divergence between initial and target distributions.
Improves reliability of BBVI optimization methods.
We analyze variational inference for highly symmetric graphical models such as those arising from first-order probabilistic models. We first show that for these graphical models, the tree-reweighted variational objective lends itself to a compact lifted formulation which can be solved much more efficiently than the sta…
We develop a method to combine Markov chain Monte Carlo (MCMC) and variational inference (VI), leveraging the advantages of both inference approaches. Specifically, we improve the variational distribution by running a few MCMC steps. To make inference tractable, we introduce the variational contrastive divergence (VCD)…
NGD improves multivariate Gaussian inference by optimizing Fisher information.
A new variational inference method using sliced Wasserstein distance is proposed.
Computing the partition function of a discrete graphical model is a fundamental inference challenge. Since this is computationally intractable, variational approximations are often used in practice. Recently, so-called gauge transformations were used to improve variational lower bounds on . In this paper, we pro…
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, an…
We formalize the problem of learning interdomain correspondences in the absence of paired data as Bayesian inference in a latent variable model (LVM), where one seeks the underlying hidden representations of entities from one domain as entities from the other domain. First, we introduce implicit latent variable models,…
This work introduces a fixed-point optimization for variational inference.
New guarantees for VI in symmetric cases, extending previous results.
DisCoPyro combines category theory with machine learning for program learning.
The paper improves guarantees for VI in symmetric cases.
Paper introduces TSSDMN for modeling dynamic multilayer networks.
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators, and are closely related to Variational Monte Carlo methods from computational physics. As such, the…
Proposes an alternative method to train RBMs with binary synapses using Bayesian learning rule.
Develops geometric causal models for causal inference from dependent data.
Study finds lower bounds for energy on fibred manifolds using fiberwise symmetrization.
Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate infe…
This paper introduces a new technique for quantifying the approximation error of a broad class of probabilistic inference programs, including ones based on both variational and Monte Carlo approaches. The key idea is to derive a subjective bound on the symmetrized KL divergence between the distribution achieved by an a…
A new method for self-attention models that improves uncertainty estimation.
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
Paper proposes Walsh-Hadamard Variational Inference for efficient approximate inference in large models.
The paper classifies Landsberg spherically symmetric Finsler metrics in various dimensions.
Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …
Variational Inference shows promise for Bayesian GARCH model estimation.
A tutorial on variational inference for high-dimensional models.
Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
A new particle algorithm improves mean-field variational inference.
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
The paper introduces validated variational inference with practical error bounds.
DDVI uses diffusion models for variational inference, improving latent variable model performance.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
We investigate existence and stability of rotationally symmetric critical immersions of variational problems of higher order which were considered by Nitsche.
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameter…
NVGD uses neural networks to infer distributions without kernel choices.
A new form of the variational autoencoder (VAE) is proposed, based on the symmetric Kullback-Leibler divergence. It is demonstrated that learning of the resulting symmetric VAE (sVAE) has close connections to previously developed adversarial-learning methods. This relationship helps unify the previously distinct techni…
This work improves variational inference by reducing gradient variance.
SoftCVI uses contrastive estimation to infer complex posteriors.
New method reduces inference variance for faster optimization.
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…