arXiv research

Improves bandit convex optimization with gradient variations.

Paper proposes a novel model to improve n-ary cross-sentence relation extraction by addressing noisy data and non-consecutive sentences.

In this paper, we consider the connectedness of planar self-affine set $T(A,\mathcal{D})$ arising from an integral expanding matrix $A$ with characteristic polynomial $f(x)=x^2+bx+c$ and a digit set $\mathcal{D}=\{0,1,\dots, m\}v$. The necessary and sufficient conditions only depending on $b,c,m$ are given for the $T(A…

CNNs, RNNs, GCNs, and CapsNets have shown significant insights in representation learning and are widely used in various text mining tasks such as large-scale multi-label text classification. However, most existing deep models for multi-label text classification consider either the non-consecutive and long-distance sem…

In the paper, we focus on the connectedness of planar self-affine sets $T(A,{\mathcal{D}})$ generated by an integer expanding matrix $A$ with $|\det (A)|=3$ and a collinear digit set ${\mathcal{D}}=\{0,1,b\}v$, where $b>1$ and $v\in {\mathbb{R}}^2$ such that $\{v, Av\}$ is linearly independent. We discuss the domain of…

Improved sampling method using regularized Stein Variational Gradient Flow.

New algorithm learns HMM parameters on Riemannian manifolds.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

This work improves variational inference by reducing gradient variance.

This work proposes a new method for variational inference using Wasserstein gradient descent.

Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditiona…

This paper bridges variational inference and Wasserstein gradient flows.

New path-gradient estimator for continuous normalizing flows.

VarGrad reduces variance in ELBO gradient estimation for variational inference.

New method reduces inference variance for faster optimization.

The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterizat…

This paper shows equivalence between SVGD and BBVI using kernel gradient flows.

Proposes VSGD optimizer combining probabilistic and gradient-based methods.

Square-root natural-gradient improves variational inference convergence.

In the Teichmüller space of a hyperbolic surface of finite type, we construct geodesic lines for Thurston's asymmetric metric having the property that when they are traversed in the reverse direction, they are also geodesic lines (up to reparametrization). The lines we construct are special stretch lines in the sense o…

Paper formulates particle flow using variational inference and Fisher-Rao gradient flow.

New method for constrained sampling using gradient flows.

Improves gradient estimation for discrete distributions with variance reduction techniques.

Variational inference approximates the posterior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solutions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approximation for the gradients. This enables variatio…

Variational inference has experienced a recent surge in popularity owing to stochastic approaches, which have yielded practical tools for a wide range of model classes. A key benefit is that stochastic variational inference obviates the tedious process of deriving analytical expressions for closed-form variable updates…

Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} minimizing the KL divergence to match the target distribution in an amortize fashion. Consequently, we a…

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control varia…

We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …

A new ParVI framework improves particle-based variational inference methods.

We introduce local expectation gradients which is a general purpose stochastic variational inference algorithm for constructing stochastic gradients through sampling from the variational distribution. This algorithm divides the problem of estimating the stochastic gradients over multiple variational parameters into sma…

DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.

New method reduces training cost by using approximate gradients.

BSVGD improves sampling for multimodal distributions using branching.

NVGD uses neural networks to infer distributions without kernel choices.

Paper improves variance control in importance weighted variational bounds.

A new method reduces variance in training discrete latent variable models.

It is well known that Markov chain Monte Carlo (MCMC) methods scale poorly with dataset size. A popular class of methods for solving this issue is stochastic gradient MCMC. These methods use a noisy estimate of the gradient of the log posterior, which reduces the per iteration computational cost of the algorithm. Despi…

The paper analyzes rates for a modified gradient descent method using Stein variational gradients.

BBVI with STL converges geometrically under perfect specification, with quadratic variance bound.

Paper improves REINFORCE for VI without restrictive assumptions.

Quantized Variational Inference improves ELBO optimization with fast convergence.

Variational Bayesian neural networks combine the flexibility of deep learning with Bayesian uncertainty estimation. However, inference procedures for flexible variational posteriors are computationally expensive. A recently proposed method, noisy natural gradient, is a surprisingly simple method to fit expressive poste…

A new method reduces complexity and uncertainty in neural networks.

Gradient-free method improves predictive accuracy for probabilistic models.

New method improves variational inference for better posterior approximation.

QBVI uses natural gradients for efficient Bayesian learning.

Improves understanding of stochastic NGVI convergence rates.

NVA combines variational posteriors, annealing, and natural-gradient learning for multimodal optimization.