arXiv research

We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …

Variational inference methods for latent variable statistical models have gained popularity because they are relatively fast, can handle large data sets, and have deterministic convergence guarantees. However, in practice it is unclear whether the fixed point identified by the variational inference algorithm is a local…

The paper improves Gaussian process regression by optimizing hyperparameters.

Develops methods for structured variational inference with star-structured models.

Bayesian supertrees method uses variational Bayes for overlapping taxon subsets.

Minimal submanifolds are found as energy concentration sets in variational problems.

Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the po…

Bayesian method improves SOM training for dynamic data.

We derive a formula for the first variation of horizontal perimeter measure for $C^2$ hypersurfaces of completely general sub-Riemannian manifolds, allowing for the existence of characteristic points. For $C^2$ hypersurfaces in vertically rigid sub-Riemannian manifolds we also produce a second variation formula for var…

We show how the homogeneous variational bicomplex provides a useful formalism for describing a number of properties of single-integral variational problems, and we introduce a subsequence of one of the rows of the bicomplex which is locally exact with respect to the variational derivative. We are therefore able to reco…

Introduces a variational framework for indefinite Lagrangians with specific symmetries.

This work improves variational inference by reducing gradient variance.

The paper analyzes uncertainty quantification in sparse Gaussian process regression with a Brownian motion prior.

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Usin…

New scalable variational Bayes methods for Hawkes processes.

New method disentangles shared and private latent factors in multimodal data.

GMVAE improves open-set classification by clustering latent representations.

The paper studies the dimension of limit sets using variational principles and stationary measures.

We consider undiscounted reinforcement learning in Markov decision processes (MDPs) where both the reward functions and the state-transition probabilities may vary (gradually or abruptly) over time. For this problem setting, we propose an algorithm and provide performance guarantees for the regret evaluated against the…

We consider the probabilistic analogue to neural network matrix factorization (Dziugaite & Roy, 2015), which we construct with Bayesian neural networks and fit with variational inference. We find that a linear model fit with variational inference can attain equivalent predictive performance to the regular neural networ…

We consider the variational complex on infinite jet space and the complex of variational derivatives for Lagrangians of multidimensional paths and study relations between them. The discussion of the variational (bi)complex is set up in terms of a flat connection in the jet bundle. We extend it to supercase using a part…

Adapts score matching for missing data in flexible settings.

Meta-learning framework improves model performance on few-shot classification tasks.

A setting for global variational geometry on Grassmann fibrations is presented. The integral variational functionals for finite dimensional immersed submanifolds are studied by means of the fundamental Lepage equivalent of a homogeneous Lagrangian, which can be regarded as a generalization of the well-known Hilbert for…

Hidden regular variation defines a subfamily of distributions satisfying multivariate regular variation on $\mathbb{E} = [0, \infty]^d \backslash \{(0,0, ..., 0) \}$ and models another regular variation on the sub-cone $\mathbb{E}^{(2)} = \mathbb{E} \backslash \cup_{i=1}^d \mathbb{L}_i$, where $\mathbb{L}_i$ is the $i…

The paper is mainly devoted to systematic developments and applications of geometric aspects of second-order variational analysis that are revolved around the concept of parabolic regularity of sets. This concept has been known in variational analysis for more than two decades while being largely underinvestigated. We …

Research in several fields now requires the analysis of data sets in which multiple high-dimensional types of data are available for a common set of objects. In particular, The Cancer Genome Atlas (TCGA) includes data from several diverse genomic technologies on the same cancerous tumor samples. In this paper we introd…

New method calibrates predictions in chaotic systems using variational inference.

Variational inference algorithms have proven successful for Bayesian analysis in large data settings, with recent advances using stochastic variational inference (SVI). However, such methods have largely been studied in independent or exchangeable data settings. We develop an SVI algorithm to learn the parameters of hi…

Variational reduction simplifies Lagrangian systems with scaling symmetries.

Combines variational and evolutionary optimization for generative models.

This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.

Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a g…

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control varia…

Empirical Bayes rates via variational approximations and prior decomposition.

New framework improves variational inference for high-dimensional posteriors.

In this work we perform outlier detection using ensembles of neural networks obtained by variational approximation of the posterior in a Bayesian neural network setting. The variational parameters are obtained by sampling from the true posterior by gradient descent. We show our outlier detection results are comparable …

New method scales Gaussian processes with derivatives using variational inference.

We study system design problems stated as parameterized stochastic programs with a chance-constraint set. We adopt a Bayesian approach that requires the computation of a posterior predictive integral which is usually intractable. In addition, for the problem to be a well-defined convex program, we must retain the conve…

We introduce the variational filtering EM algorithm, a simple, general-purpose method for performing variational inference in dynamical latent variable models using information from only past and present variables, i.e. filtering. The algorithm is derived from the variational objective in the filtering setting and cons…

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …

Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact inference is often intractable. Markov chain Monte Carlo techniques make approximat…

Integrative analysis of disparate data blocks measured on a common set of experimental subjects is a major challenge in modern data analysis. This data structure naturally motivates the simultaneous exploration of the joint and individual variation within each data block resulting in new insights. For instance, there i…

The paper learns perturbation sets from data to improve robustness in machine learning.

Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defi…

Modeling sequential data has become more and more important in practice. Some applications are autonomous driving, virtual sensors and weather forecasting. To model such systems so called recurrent models are used. In this article we introduce two new Deep Recurrent Gaussian Process (DRGP) models based on the Sparse Sp…

Paper variates Navier-Stokes-Fourier system for thermodynamic consistency.

ProJIVE integrates multiple data types to explain joint and individual variation.