A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New variational principle found for non-variational differential equations.
Improved Bayesian uncertainty quantification using variational bagging.
Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control varia…
Adaptive variational Bayes framework improves inference adaptively.
The paper introduces structured variational families to improve scalability in black-box variational inference.
Develops methods for structured variational inference with star-structured models.
The paper improves Gaussian process regression by optimizing hyperparameters.
The paper explores variational principles for equations of maximal symmetry, providing new insights and results.
Improves bandit convex optimization with gradient variations.
Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…
New formulas for coassociative submanifolds' volume variation.
Variational language models seek to estimate the posterior of latent variables with an approximated variational posterior. The model often assumes the variational posterior to be factorized even when the true posterior is not. The learned variational posterior under this assumption does not capture the dependency relat…
Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a g…
he celebrated formula of Schlafli relates the variation of the dihedral angles of a smooth family of polyhedra in a space form and the variation of volume. We give a smooth analogue of this classical formula -- our result relates the variation of the volume bounded by a hypersurface moving in a general Einstein manifol…
Extends tracking guarantees for time-varying variational inequalities.
This paper belongs to the realm of conformal geometry and deals with Euclidean submanifolds that admit smooth variations that are infinitesimally conformal. Conformal variations of Euclidean submanifolds is a classical subject in differential geometry. In fact, already in 1917 Cartan classified parametrically the Eucli…
Expanding on previous work, this note generalizes geometric structures results.
The paper analyzes variational autoencoders for state space models with risk bounds.
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…
We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trad…
We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte Carlo (MCMC) algorithm -- Collapsed Gibbs sampling. We compare their time complexity and performance.…
Variational inference for latent variable models is prevalent in various machine learning problems, typically solved by maximizing the Evidence Lower Bound (ELBO) of the true data likelihood with respect to a variational distribution. However, freely enriching the family of variational distribution is challenging since…
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
Recent progress in variational inference has paid much attention to the flexibility of variational posteriors. One promising direction is to use implicit distributions, i.e., distributions without tractable densities as the variational posterior. However, existing methods on implicit posteriors still face challenges of…
Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
Quantum neural tangent kernels help understand variational quantum circuits in machine learning.
New insights into variational inference using Monte Carlo estimates.
We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variational inference and thermodynamic integration that results in a tighter lower bound to the log marginal likelihood than the standard variatio…
PVI improves SIVI by directly optimizing ELBO without parametric assumptions.
We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …
The inverse problem of the calculus of variations asks whether a given system of partial differential equations (PDEs) admits a variational formulation. We show that the existence of a presymplectic form in the variational bicomplex, when horizontally closed on solutions, allows us to construct a variational formulatio…
Guarantees convergence for black-box variational inference without modifications.
Variational inference has become one of the most widely used methods in latent variable modeling. In its basic form, variational inference employs a fully factorized variational distribution and minimizes its KL divergence to the posterior. As the minimization can only be carried out approximately, this approximation i…
We propose a novel hierarchical model for multitask bipartite ranking. The proposed approach combines a matrix-variate Gaussian process with a generative model for task-wise bipartite ranking. In addition, we employ a novel trace constrained variational inference approach to impose low rank structure on the posterior m…
The motivations for using variational inference (VI) in neural networks differ significantly from those in latent variable models. This has a counter-intuitive consequence; more expressive variational approximations can provide significantly worse predictions as compared to those with less expressive families. In this …
We consider a square-integrable semimartingale and investigate the convex order relations between its discrete, continuous and predictable quadratic variation. As the main results, we show that if the semimartingale has conditionally independent increments and symmetric jump measure, then its discrete realized variance…
Newton's method solves variational problems on manifolds.
The study provides statistical guarantees for Bayesian variational boosting.
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more s…
Robust VB framework for large datasets with outliers.
Introduces VSMD to improve generative diffusion processes without high costs.
Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.
Paper improves variance control in importance weighted variational bounds.
This paper develops scalable control variates for Monte Carlo methods using stochastic optimization.
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…
We introduce a new approximation of -divergences for machine learning.