New unbiased gradient estimators for complex optimization problems.
problem Unbiased and variance-limited gradient estimation for conditional stochastic optimization.
method Developed multilevel Monte Carlo gradient estimators for conditional stochastic optimization problems.
result Unbiased and finite variance gradient estimators for conditional stochastic optimization problems.
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
A number of optimization approaches have been proposed for optimizing nonconvex objectives (e.g. deep learning models), such as batch gradient descent, stochastic gradient descent and stochastic variance reduced gradient descent. Theory shows these optimization methods can converge by using an unbiased gradient estimat…
New method for unbiased regression reduces excess risk.
problem Least squares regression with optimal solution and Hessian matrix.
method Averaged stochastic gradient descent with time-average estimator.
result Unbiased estimator with O(1/k) expected excess risk.
Proposes unbiased estimators for training mixture of experts models.
problem Efficiently training large-scale mixture of experts models on modern hardware.
method Two unbiased estimators based on principled stochastic assignment procedures.
result Both estimators are more effective and robust than biased alternatives.
Unbiased gradient estimation improves VAE performance.
problem Training VAEs via maximum likelihood is difficult due to intractable integrals.
method Introduced unbiased estimators of the log-likelihood gradient using coupled Markov chains.
result Unbiased estimators lead to better predictive performance in VAEs.
Paper proposes an unbiased optimization method for Bayesian experimental design.
problem Maximizing expected information gain in Bayesian experimental design.
method Randomized multilevel Monte Carlo (MLMC) method combined with stochastic gradient descent.
result An unbiased estimator for the gradient of expected information gain.
UREs lead to overfitting in complex models, especially in complementary label learning.
problem Overfitting in weakly supervised learning with complementary labels.
method Proposed a surrogate complementary loss (SCL) framework to reduce gradient variance.
result SCL mitigates overfitting and improves URE-based methods.
The recently proposed Unbiased Online Recurrent Optimization algorithm (UORO, arXiv:1702.05043) uses an unbiased approximation of RTRL to achieve fully online gradient-based learning in RNNs. In this work we analyze the variance of the gradient estimate computed by UORO, and propose several possible changes to the meth…
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss functions and training nonconvex deep neural networks. The theory assumes that on…
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
Unbiased methods for alpha-divergence minimization struggle in high dimensions.
problem The difficulty of unbiased alpha-divergence minimization in high dimensions.
method Signal-to-Noise Ratio (SNR) analysis of gradient estimators.
result The SNR of the gradient estimator worsens exponentially with dimensionality.
We show a connection between the Fourier spectrum of Boolean functions and the REINFORCE gradient estimator for binary latent variable models. We show that REINFORCE estimates (up to a factor) the degree-1 Fourier coefficients of a Boolean function. Using this connection we offer a new perspective on variance reduction…
AR-DAE approximates entropy gradient for machine learning models.
problem Intractable computation of entropy gradient for continuous distributions.
method Amortized residual denoising autoencoder (AR-DAE) to approximate entropy gradient.
result AR-DAE provides an unbiased gradient approximation for entropy.
Learning in models with discrete latent variables is challenging due to high variance gradient estimators. Generally, approaches have relied on control variates to reduce the variance of the REINFORCE estimator. Recent work (Jang et al. 2016, Maddison et al. 2016) has taken a different approach, introducing a continuou…
This work improves texture segmentation by automatically tuning hyperparameters for Total-Variation.
problem The challenge is to automatically select hyperparameters for Total-Variation texture segmentation.
method The approach involves extending Stein's unbiased gradient estimator to handle correlated Gaussian noise, leading to an automatic tuning method.
result The method provides an automatic way to select hyperparameters for Total-Variation texture segmentation.
A new variational method for SSMs improves inference efficiency.
problem Hard variational inference for state space models.
method Proposes variational marginal particle filter (VMPF) based on Rao-Blackwellization.
result VMPF provides tighter variational bounds and sometimes benefits from unbiased reparameterization.
A new unbiased Hessian estimator for expectation-based objectives.
problem Estimating Hessian for objectives with non-reparameterizable nodes.
method GO Hessian estimator for expectation-based objectives.
result GO Hessian provides unbiased and low-variance estimation of Hessian.
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
PES method reduces bias in gradient estimation for unrolled graphs.
problem High variance and bias in gradient estimation for unrolled computation graphs.
method Divide graph into unrolls, apply ES update, accumulate correction terms.
result PES provides unbiased, low-variance gradient estimates.
End-to-end training of DBMs with improved gradient estimation.
problem Biased gradient estimation in DBMs, especially with high-dimensional states.
method Unbiased contrastive divergence using MH coupling and local mode initialization.
result End-to-end training of DBMs without greedy pretraining, achieving FID score of 10.33 for MNIST.
In applications of Gaussian processes where quantification of uncertainty is of primary interest, it is necessary to accurately characterize the posterior distribution over covariance parameters. This paper proposes an adaptation of the Stochastic Gradient Langevin Dynamics algorithm to draw samples from the posterior …
A large class of machine learning techniques requires the solution of optimization problems involving spectral functions of parametric matrices, e.g. log-determinant and nuclear norm. Unfortunately, computing the gradient of a spectral function is generally of cubic complexity, as such gradient descent methods are rath…
Paper presents a new policy gradient theorem using weak derivatives for reinforcement learning.
problem Continuous state-action reinforcement learning problems.
method Introduced an alternative policy gradient theorem using weak derivatives.
result The new approach yields algorithms that converge almost surely to stationary points of the value function.
Due to the high variance of policy gradients, on-policy optimization algorithms are plagued with low sample efficiency. In this work, we propose Augment-Reinforce-Merge (ARM) policy gradient estimator as an unbiased low-variance alternative to previous baseline estimators on tasks with binary action space, inspired by …
PLUMAGE improves large model training efficiency and stability.
problem Accelerator memory and networking constraints during large model training.
method Probabilistic Low rank Unbiased Minimum Variance Gradient Estimator (PLUMAGE) that resolves bias and variance issues.
result PLUMAGE reduces training loss by 28% on average across the GLUE benchmark.
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …
New approach improves model generalization through distributionally robust learning.
problem Improving model generalization in machine learning.
method Stochastic gradient descent applied to the outer minimization problem, with gradient estimation through multi-level Monte Carlo randomization.
result Our approach yields significant benefits over previous work in numerical experiments.
Deep latent variable models have become a popular model choice due to the scalable learning algorithms introduced by (Kingma & Welling, 2013; Rezende et al., 2014). These approaches maximize a variational lower bound on the intractable log likelihood of the observed data. Burda et al. (2015) introduced a multi-sample v…
A new method for discrete data normalizing flows using latent transformations.
problem Challenges in parameterizing bijective transformations for discrete data.
method Predict a distribution over latent transformations to make the marginal likelihood differentiable.
result Discrete-data normalizing flows can be trained using gradient-based learning with unbiased score function estimation.
Stochastic variational inference (SVI) lets us scale up Bayesian computation to massive data. It uses stochastic optimization to fit a variational distribution, following easy-to-compute noisy natural gradients. As with most traditional stochastic optimization methods, SVI takes precautions to use unbiased stochastic g…
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…
This paper analyzes SGD with biased gradients for deep learning models.
problem Analyzing SGD with biased gradients for deep learning models.
method Non-asymptotic analysis of SGD with adaptive steps for non-convex smooth functions.
result Adagrad, RMSProp, and AMSGRAD converge to critical points at a similar rate to unbiased case.
ES-Single uses ES to estimate gradients in unrolled graphs, reducing variance and improving performance.
problem Estimating gradients in unrolled computation graphs with low variance and stability.
method Evolution strategies (ES) applied to unrolled graphs, with a single perturbation per particle.
result ES-Single reduces variance compared to PES, leading to better performance in various tasks.
New HMC framework reduces variance for sampling from log-concave distributions.
problem Efficient sampling from log-concave distributions with high precision.
method Unified formulation of biased and unbiased variance reduction methods for HMC.
result Unbiased and biased gradient estimators achieve different gradient complexities and accuracy.
New estimator reduces variance in discrete random variables.
problem Estimating gradients for discrete random variables with reduced variance.
method Sampling without replacement and Rao-Blackwellization.
result Our estimator is the most consistent gradient estimator across different entropy settings.
Variational Auto-Encoders (VAEs) have become very popular techniques to perform inference and learning in latent variable models as they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (…
We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks. This algorithm works in an online, memoryless setting, thus requiring no backpropagation through time, and is scalable, avoiding the large computational and memory cost of maintaining the full gradie…
vqSGD reduces communication in distributed optimization with convergence guarantees.
problem Reduction of communication cost in distributed optimization.
method Vector quantization schemes based on convex hull of a point set.
result Asymptotic reduction in communication cost with convergence guarantees.
U-Clip clips gradients to reduce bias without discarding them.
problem Reduces bias in stochastic gradient descent.
method Maintains a buffer of clipped gradients to add to next iteration.
result U-Clip updates are unbiased on average.
To address the challenge of backpropagating the gradient through categorical variables, we propose the augment-REINFORCE-swap-merge (ARSM) gradient estimator that is unbiased and has low variance. ARSM first uses variable augmentation, REINFORCE, and Rao-Blackwellization to re-express the gradient as an expectation und…
A new method optimizes diffusion models with recursive likelihood ratios.
problem Efficiently aligning pre-trained diffusion models for specific applications.
method Recursive Likelihood Ratio (RLR) optimizer for Half-Order (HO) fine-tuning.
result The RLR method achieves unbiased and lower-variance gradients, improving model performance.
New estimator for SDEs is shown to be an adjoint state method.
problem Estimating gradients for overparameterized SDEs efficiently.
method Demonstrates generator gradient estimator as an adjoint state method.
result Generator gradient estimator is an adjoint state method for SDEs.
VarGrad reduces variance in ELBO gradient estimation for variational inference.
problem Improving the variance of gradient estimators in variational inference.
method VarGrad uses a new log-variance loss to estimate the ELBO gradient, achieving lower variance than the score function method.
result VarGrad offers a lower variance gradient estimator compared to other methods.
New gradient estimators for discrete variables improve model training.
problem Training models with discrete latent variables is challenging due to high gradient variance.
method Introduced novel gradient estimators based on importance sampling and statistical couplings, extending to categorical variables.
result Proposed gradient estimators outperform previous methods in systematic experiments.
New method reduces training cost by using approximate gradients.
problem Training neural networks is computationally expensive.
method Uses control variates to approximate gradients without full backward pass.
result Efficacy demonstrated on a vision transformer classification task.