A new EVI framework improves ParVI methods by maintaining variational structure and reducing KL-divergence.
problem Improving variational inference methods for better approximation of target distributions.
method EVI framework that minimizes the VI objective function based on an energy-dissipation law, including a new 'Approximation-then-Variation' scheme.
result The new scheme significantly decreases KL-divergence and outperforms existing ParVI methods in fidelity.
A new framework for Einstein-Hilbert action with topological variations.
problem Understanding critical points and dimensionality in Einstein-Hilbert action.
method Localized Einstein-Hilbert variational principle, topology on Sobolev configurations, topological variations.
result No critical points in dimension 4, higher dimensions free of this problem.
Derive Dirichlet scalar curvature energy functional variation formula
problem Dirichlet scalar curvature energy functional
method First variation formula
result Introduce Dirichlet-Einstein metrics
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
Develops a new framework for analyzing MFVI algorithms.
problem Analyzes mean field variational inference (MFVI) formulations.
method Inspired by variational Bayesian formulations, represents MFVI problem in three ways: gradient flow, Fokker-Planck-like equations, and diffusion process.
result Establishes rigorous guarantees for convergence of time-discretized coordinate ascent variational inference algorithms.
We develop a framework for incorporating structured graphical models in the \emph{encoders} of variational autoencoders (VAEs) that allows us to induce interpretable representations through approximate variational inference. This allows us to both perform reasoning (e.g. classification) under the structural constraints…
New framework improves stochastic optimization for variational inference.
problem Improving variational posterior approximations in high-dimensional models.
method Developed a robust stochastic optimization framework using Markov chains.
result Demonstrated improved accuracy and robustness across diverse models.
DIVA clusters dynamic data without needing cluster count, outperforming baselines.
problem Clustering complex, dynamic data without prior knowledge of cluster count.
method Nonparametric Dirichlet Process Mixtures with memoized online variational inference.
result DIVA outperforms state-of-the-art in classifying complex data with changing features.
AutoBayes simplifies variational inference by composing models and optimizing them.
problem Generalized variational inference complexities and optimization challenges.
method Compositional framework exploiting chain rules for automatic differentiation.
result Optimized models and parameterized statistical games can be locally optimized.
A new framework for recycling Gaussian process approximations.
problem Efficiently combining multiple Gaussian process approximations.
method Construct variational ensembles using a dictionary of fitted Gaussian processes.
result Framework allows for various tasks and scalability.
New comparison shows differences in how value is incorporated in AIF and CAI.
problem Clarifying the relationship between Active Inference and Control-as-Inference.
method Formal comparison of AIF and CAI frameworks.
result Primary difference is how value is incorporated into generative models.
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…
Variational autoencoders provide a principled framework for learning deep latent-variable models and corresponding inference models. In this work, we provide an introduction to variational autoencoders and some important extensions.
Optimal hedging framework with variational preferences under convex risk measures.
problem Optimal hedging with variational preferences under convex risk measures.
method Theoretical hedging optimization framework with dual representation of risk measures and utilities.
result Derivation of optimality and indifference pricing conditions.
A framework uses variational Bayes for solving inverse problems efficiently.
problem Solving inverse problems in various dimensions with flexibility and accuracy.
method Variational Bayes approximations with message passing and factor graph approach.
result Efficient algorithm updates for higher dimensions and computational advantage over MCMC.
This paper provides a unifying theoretical framework for stochastic optimization algorithms by means of a latent stochastic variational problem. Using techniques from stochastic control, the solution to the variational problem is shown to be equivalent to that of a Forward Backward Stochastic Differential Equation (FBS…
Two complementary approaches have been extensively used in signal and image processing leading to novel results, the sparse representation methodology and the variational strategy. Recently, a new sparsity based model has been proposed, the cosparse analysis framework, which may potentially help in bridging sparse appr…
Proposes an amortized variational framework for Deep Q Networks.
problem Efficient exploration in deep reinforcement learning.
method Amortized variational inference for action value function approximation.
result Significantly less learning parameters and better performance.
Proposes a framework to maximize mutual information in VAE models for better latent code representation.
problem Lack of explicit measurement of the quality of learned representations in VAE models.
method Variational Mutual Information Maximization Framework for VAE.
result Maximizes mutual information between latent codes and observations, improving latent code representation.
Recent advances in Neural Variational Inference allowed for a renaissance in latent variable models in a variety of domains involving high-dimensional data. While traditional variational methods derive an analytical approximation for the intractable distribution over the latent variables, here we construct an inference…
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) …
Unified deep learning framework solves various optimal transport problems.
problem Solving variational problems in optimal transport with computational challenges.
method Unified deep learning framework leveraging dual formulation of Lagrangians.
result Outperforms previous approaches in single-cell trajectory inference.
New framework improves variational inference for high-dimensional posteriors.
problem Challenges in choosing variational objectives and approximating families for high-dimensional posteriors.
method Conceptual framework and experimental tools to understand and optimize variational objectives and families.
result For moderate-to-high-dimensional posteriors, exclusive KL divergence is recommended due to optimization ease; for low-dimensional, heavy-tailed variational families are effective.
A framework for disentangling class-related and class-independent factors in data.
problem Learning disentangled representations in variational autoencoders.
method Attention mechanism in latent space, mixture models, Bhattacharyya coefficient, semi-supervised training.
result Disentangles class-related and class-independent factors of variation.
This paper learns variational models and solvers for inverse problems from incomplete data.
problem Solving inverse problems with partially observed data.
method Joint learning of variational cost and gradient-based solver as neural networks.
result Joint learning leads to improved reconstruction performance.
Improved vector quantization using Gaussian mixtures for better codebook utilization.
problem Training instability and information loss in discrete vector quantization.
method Generalized vector quantization with Gaussian mixture model and aggregated categorical posterior evidence lower bound.
result GM-VQ improves codebook utilization and reduces information loss without heuristics.
This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in compl…
We develop a variational framework for SDEs driven by fractional noise.
problem Capturing long-term dependencies in SDEs driven by fractional noise.
method Markov approximation of fractional Brownian motion, variational inference, neural networks.
result Efficient variational inference of posterior path measures for neural-SDEs.
Recent advances in neural variational inference have spawned a renaissance in deep latent variable models. In this paper we introduce a generic variational inference framework for generative and conditional models of text. While traditional variational methods derive an analytic approximation for the intractable distri…
A new ParVI framework improves particle-based variational inference methods.
problem Non-trivial kernel design in particle-based variational inference methods.
method Proposes a generalized Wasserstein gradient descent (GWG) framework with broader regularizers.
result Demonstrates strong convergence guarantees and effectiveness on simulated and real data.
Enhanced DeepONet framework with uncertainty quantification for complex operators.
problem Learning complex operators with uncertainty quantification.
method Generalised variational inference (GVI) using Rényi's α-divergence.
result Superior predictive accuracy and uncertainty quantification.
Framework identifies causal direction from single data setting.
problem Identify causal direction from single observational data.
method VCEI framework based on ICM principle and artificial variation.
result VCEI is competitive to other frameworks in identifying causal direction.
Many biological data analysis processes like Cytometry or Next Generation Sequencing (NGS) produce massive amounts of data which needs to be processed in batches for down-stream analysis. Such datasets are prone to technical variations due to difference in handling the batches possibly at different times, by different …
The paper uses persistent homology to estimate recurrence times in multi-variate time series.
problem Estimating recurrence times in multi-variate time series with different cyclic behaviors.
method Persistent homology framework with three specialized methods.
result Validated methods on real-world data, including a new benchmark dataset.
Paper improves variational inference for complex models.
problem Improving statistical accuracy of variational inference in high-dimensional models.
method Developed a general framework for MFVI and proposed a partially grouped VI algorithm.
result Proposed algorithm works and outperforms vanilla MFVI in mixed membership stochastic blockmodel.
This paper refines human labeling as a measurement process, revealing four sources of variation.
problem Systematic variation in human labeling obscures model learning.
method Introduces a statistical framework to decompose labeling outcomes.
result Empirical evidence for four components of labeling variation.
This paper presents a novel variational inference framework for deriving a family of Bayesian sparse Gaussian process regression (SGPR) models whose approximations are variationally optimal with respect to the full-rank GPR model enriched with various corresponding correlation structures of the observation noises. Our …
Develops inference combinators for probabilistic programs using neural networks.
problem Creating efficient proposals for probabilistic program inference.
method Inference combinators using neural network parameterization of proposals.
result Correct by construction variational methods tailored to specific models.
Geometric framework analyzes bias in variational inference for posterior functionals.
problem Analyzing the bias of posterior functionals under variational approximations.
method Developed a geometric framework to evaluate the bias of posterior functionals using the variational tangent space.
result The leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
A distributed framework for reducing high-dimensional matrix-variate time series data.
problem Reducing dimensionality of high-dimensional, heterogeneous matrix-variate time series data.
method Data partitioning, distributed two-dimensional tensor PCA, aggregation, final PCA, factor matrix computation.
result Preserves latent matrix structure, improves computational efficiency and information utilization.
BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the tedious task of implementing the variational Bayesian update equations, the user can construct model…
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
Marginal MAP problems are notoriously difficult tasks for graphical models. We derive a general variational framework for solving marginal MAP problems, in which we apply analogues of the Bethe, tree-reweighted, and mean field approximations. We then derive a "mixed" message passing algorithm and a convergent alternati…
Geometric analysis improves convergence of variational inference.
problem Challenges in analyzing convergence of variational inference due to non-convexity and non-smoothness.
method Exploits exponential family structure and Bregman divergences to geometrically analyze the optimization landscape.
result Establishes non-asymptotic convergence rates for gradient descent algorithms.
Gaussian process classification is a popular method with a number of appealing properties. We show how to scale the model within a variational inducing point framework, outperforming the state of the art on benchmark datasets. Importantly, the variational formulation can be exploited to allow classification in problems…
Robust VB framework for large datasets with outliers.
problem Handling outliers and contamination in large datasets.
method Divide and conquer approach with geometric median aggregation.
result VM-Posterior distribution preserves contraction properties.
Develops variational framework for LQG risk-sensitive MFGs with major-minor interactions.
problem Risk-sensitive optimal control in LQG systems with major-minor interactions.
method Variational approach, nonlinear necessary and sufficient condition of optimality, equivalent risk-neutral measure, Markovian closed-loop best-response strategies.
result Derives optimal control strategies for LQG risk-sensitive MFGs with major-minor interactions, establishing Nash and ε-Nash equilibria.