arXiv research

VarGrad reduces variance in ELBO gradient estimation for variational inference.

This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making con…

In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variation…

This work interprets SFA through variational inference, relaxing linearity constraints.

Proposes variational Gaussian approximations for solving the Kushner equation.

New reweighted losses improve diffusion model training and image quality.

New method prevents neural network breakdown by combining trimmed loss and variation regularization.

We analyze bias-variance of margin losses.

Stable GFlowNets prevent loss spikes and mode collapse in training.

As deep Variational Auto-Encoder (VAE) frameworks become more widely used for modeling biomolecular simulation data, we emphasize the capability of the VAE architecture to concurrently maximize the timescale of the latent space while inferring a reduced coordinate, which assists in finding slow processes as according t…

Prb-GAN uses dropout and variational inference to improve GAN performance.

Geometric Mean Market Makers super-hedge impermanent loss without models.

Study optimal consumption for loss-averse agents considering past spending peaks.

Stochastic variational inference is an established way to carry out approximate Bayesian inference for deep models. While there have been effective proposals for good initializations for loss minimization in deep learning, far less attention has been devoted to the issue of initialization of stochastic variational infe…

AutoBayes simplifies variational inference by composing models and optimizing them.

Improved sampling via learned diffusions using variational losses.

In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…

DAEs can generate images without additional loss terms, inheriting VAE properties.

Neural networks solve variational inequalities for optimal stopping problems.

New bound shows variational algorithms may struggle with barren plateaus.

Paper provides finite-sample guarantees for Wasserstein DRO without dimensionality curse.

VarNet solves PDEs with deep neural networks using variational loss.

In this paper, we provide an information-theoretic interpretation of the Vector Quantized-Variational Autoencoder (VQ-VAE). We show that the loss function of the original VQ-VAE can be derived from the variational deterministic information bottleneck (VDIB) principle. On the other hand, the VQ-VAE trained by the Expect…

Score matching fails to train VAEs robustly, revealing autoencoding loss insights.

Proposes an amortized variational framework for Deep Q Networks.

Reparameterization of variational auto-encoders with continuous random variables is an effective method for reducing the variance of their gradient estimates. In the discrete case, one can perform reparametrization using the Gumbel-Max trick, but the resulting objective relies on an $\arg \max$ operation and is non-dif…

New $\alpha$-divergence loss function improves neural density ratio estimation.

A new upper bound for variational inference improves the efficiency of Bayesian deep learning.

Wasserstein distributionally robust optimization (DRO) has recently achieved empirical success for various applications in operations research and machine learning, owing partly to its regularization effect. Although connection between Wasserstein DRO and regularization has been established in several settings, existin…

We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters. We will establish the equivalence between our proposed surrogate objective and variational i…

This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein var…

We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…

We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical $\mathcal{L}^p$ losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the …

In recent years Variation Autoencoders have become one of the most popular unsupervised learning of complicated distributions.Variational Autoencoder (VAE) provides more efficient reconstructive performance over a traditional autoencoder. Variational auto enocders make better approximaiton than MCMC. The VAE defines a …

CosmoVAE uses deep learning to fill in missing parts of the cosmic microwave background map.

New algorithms minimize dynamic regret for strongly convex losses.

GCVAE improves disentanglement in VAEs while balancing reconstruction error.

This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidd…

Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique …

A new loss function ED simplifies training energy-based models without scores.

TA-VAAL improves active learning by better utilizing task structures and overall data distribution.

Learning binary classifiers only from positive and unlabeled (PU) data is an important and challenging task in many real-world applications, including web text classification, disease gene identification and fraud detection, where negative samples are difficult to verify experimentally. Most recent PU learning methods …

This work provides statistical guarantees for VAEs using PAC-Bayesian theory.

We reinterpreting the variational inference in a new perspective. Via this way, we can easily prove that EM algorithm, VAE, GAN, AAE, ALI(BiGAN) are all special cases of variational inference. The proof also reveals the loss of standard GAN is incomplete and it explains why we need to train GAN cautiously. From that, w…

This work focuses on dynamic regret of online convex optimization that compares the performance of online learning to a clairvoyant who knows the sequence of loss functions in advance and hence selects the minimizer of the loss function at each step. By assuming that the clairvoyant moves slowly (i.e., the minimizers c…

The choice of activation function can significantly influence the performance of neural networks. The lack of guiding principles for the selection of activation function is lamentable. We try to address this issue by introducing our variational neural networks, where the activation function is represented as a linear c…

Warm starts improve variational quantum algorithms by avoiding barren plateaus.

A framework connects VAEs to GLMs for better model initialization and performance.