arXiv research

Rotates MFVI for better Gaussian approximations.

MFVI mode collapse explained; RoVI proposed to mitigate.

This paper considers a new family of variational distributions motivated by Sklar's theorem. This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i.e. with a complexity linear in the dimension of state space. Then, the proposed variational densi…

New kernel interprets 3D anisotropic data with rotations and improved predictions.

FACTM combines FA with correlated topic modeling for structured data integration.

We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embeddi…

The paper classifies and constructs rotational surfaces with constant astigmatism in space forms.

The paper learns pose variations within shape populations using constrained mixtures of factor analyzers.

The paper classifies surfaces with constant skew curvature in 3-space forms.

We study variational systems for space curves, for which the Lagrangian or action principle has a Euclidean symmetry, using the Rotation Minimising frame, also known as the Normal, Parallel or Bishop frame. Such systems have previously been studied using the Frenet-Serret frame. The Rotation Minimising frame has many a…

OCEAN infers online task identities from context variables.

New methods improve multi-agent reinforcement learning by addressing rotational dynamics.

Improved variational approximation for deep Wishart process models.

Characterizes curves for minimal surfaces in de Sitter space.

Planar neural networks learn image transformations from sequences.

Study minimizes CR surfaces in Heisenberg group with rotational symmetry.

New method learns disentangled discrete representations using categorical variational autoencoders.

We consider an infinite 3-dimensional elastic continuum whose material points experience no displacements, only rotations. This framework is a special case of the Cosserat theory of elasticity. Rotations of material points are described mathematically by attaching to each geometric point an orthonormal basis which give…

This paper shows how rotating, cropping, and translating images improves a reinforcement learning agent's ability to generalize.

There are many forms of feature information present in video data. Principle among them are object identity information which is largely static across multiple video frames, and object pose and style information which continuously transforms from frame to frame. Most existing models confound these two types of represen…

RIO uses rotation-equivariance to train robust inertial odometry models.

If a piece of the contour of a picture is missing to the eye vision, then the brain tends to complete it using some kind of sub-Riemannian geodesics of the unit tangent bundle of the plane, R2xS1. These geodesics can be obtained by lifting extremal curves of a total curvature type energy in the plane. We completely sol…

We classify all rotational surfaces in Euclidean space whose principal curvatures $κ_1$ and $κ_2$ satisfy the linear relation $κ_1=aκ_2+b$, where $a$ and $b$ are two constants. We give a variational characterization of these surfaces in terms of its generating curve. As a consequence of our classification, we find clos…

A stationary rotating surface is a compact surface in Euclidean space whose mean curvature $H$ at each point $x$ satisfies $2H(x)=a r^2+b$, where $r$ is the distance from $x$ to a fixed straight-line $L$, and $a$ and $b$ are constants. These surfaces are solutions of a variational problem that describes the shape of a …

Proposes a new model for RANS simulations with uncertainty.

Quantum method prices options by evolving a state in imaginary time.

New dataset tests mental rotation from single images, improving model understanding of 3D scenes.

We present Ordinary Differential Equation Variational Auto-Encoder (ODE$^2$VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE$^2$VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…

Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …

We consider probabilistic PCA and related factor models from a Bayesian perspective. These models are in general not identifiable as the likelihood has a rotational symmetry. This gives rise to complicated posterior distributions with continuous subspaces of equal density and thus hinders efficiency of inference as wel…

Develops computational methods for simulating rigid body dynamics on SO(3).

Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). …

Variational Inference shows promise for Bayesian GARCH model estimation.

New normalizing flows model molecular crystal structures.

A tutorial on variational inference for high-dimensional models.

Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable stati…

Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.

A new particle algorithm improves mean-field variational inference.

We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …

DDVI uses diffusion models for variational inference, improving latent variable model performance.

Boosting Variational Inference improves posterior approximations with adaptive step-sizes.

Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying th…

Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…

This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…

Variational inference offers scalable and flexible tools to tackle intractable Bayesian inference of modern statistical models like Bayesian neural networks and Gaussian processes. For largely over-parameterized models, however, the over-regularization property of the variational objective makes the application of vari…

FibQuant improves KV-cache compression for long-context inference.

Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameter…

NVGD uses neural networks to infer distributions without kernel choices.