The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterizat…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “reparameterization gradients”
Reparameterizes mirror descent as gradient descent for efficient sparse learning.
problem Efficiently training small sparse networks with mirror descent.
method Develops a framework to convert mirror descent updates into gradient descent updates on different parameters.
result Mirror descent can be reparameterized as gradient descent on modified parameters, facilitating standard backpropagation.
By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models. However, it is not applicable to a number of important continuous distributions. We introduce an a…
We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterizati…
Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations. The reparameterization trick is applicable when we can simulate a random variable by applying a differ…
A new method for optimizing models with categorical variables using diffusion.
problem Optimizing models with categorical variables, especially in discrete distributions.
method Introducing ReDGE, a diffusion-based soft reparameterization method for categorical distributions.
result ReDGE consistently matches or outperforms existing gradient-based methods in experiments.
Two new estimators improve VAE training for hierarchical and prior parameters.
problem Efficient gradient estimation for VAEs with hierarchical and prior parameters.
method Developed two generalizations of Doubly-Reparameterized Gradient Estimators (DReGs) for VAEs.
result Improved training of conditional and hierarchical VAEs on image modeling tasks.
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimator…
The reparameterization trick is widely used in variational inference as it yields more accurate estimates of the gradient of the variational objective than alternative approaches such as the score function method. Although there is overwhelming empirical evidence in the literature showing its success, there is relative…
Unified view of LR and RP gradients explained via divergence theorem.
problem Explaining the nature and relationship of LR and RP gradients.
method First principles approach using divergence theorem.
result Characterization of all possible estimators combining LR and RP.
Low-variance gradient estimation is crucial for learning directed graphical models parameterized by neural networks, where the reparameterization trick is widely used for those with continuous variables. While this technique gives low-variance gradient estimates, it has not been directly applicable to discrete variable…
The reparameterization trick has become one of the most useful tools in the field of variational inference. However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution fu…
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the repa…
The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution. After refactoring, the gradients of the loss propag…
New gradient estimators for discrete variables improve model training.
problem Training models with discrete latent variables is challenging due to high gradient variance.
method Introduced novel gradient estimators based on importance sampling and statistical couplings, extending to categorical variables.
result Proposed gradient estimators outperform previous methods in systematic experiments.
Gradient descent implicitly favors group sparsity in neural networks.
problem Understanding implicit regularization in neural networks for structured sparsity.
method Novel neural reparameterization for diagonally grouped linear networks.
result Gradient descent without explicit regularization biases towards group sparsity.
This work improves variational inference by reducing gradient variance.
problem Hard optimization of flexible variational distributions.
method Control variate based on quadratic approximation of the model's mean and covariance.
result Significant improvement in gradient variance and optimization convergence.
Paper improves REINFORCE for VI without restrictive assumptions.
problem Improves REINFORCE for VI without restrictive assumptions.
method Introduces VIMCO- gradient estimator to overcome SNR collapse.
result VIMCO- achieves SNR scaling, superior to existing VIMCO.
Deep latent variable models have become a popular model choice due to the scalable learning algorithms introduced by (Kingma & Welling, 2013; Rezende et al., 2014). These approaches maximize a variational lower bound on the intractable log likelihood of the observed data. Burda et al. (2015) introduced a multi-sample v…
Bayesian optimization tackles expensive discrete and mixed parameter spaces.
problem Optimizing expensive functions with discrete and mixed parameters.
method Probabilistic reparameterization to maximize expectation of AF over continuous parameters.
result Our approach provably converges to a maximizer of the AF and enjoys the same regret bounds as standard BO.
New method for analyzing learning dynamics in singular models.
problem Challenges in analyzing learning of singular models with no one-to-one parameter space.
method Relative reparameterization technique to extract regular sub-models.
result Demonstrated differences in convergence behavior due to algorithmic and intrinsic aspects.
Reparameterization (RP) and likelihood ratio (LR) gradient estimators are used throughout machine and reinforcement learning; however, they are usually explained as simple mathematical tricks without providing any insight into their nature. We use a first principles approach to explain LR and RP, and show a connection …
Natural gradient descent is a robust optimization method for machine learning.
problem Training poorly parameterized networks efficiently.
method Optimization algorithms with natural transformation properties.
result Optimization algorithms with natural transformation properties are more efficient for poorly parameterized networks.
Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters $\gammav$ for expectation-based objectives $\Ebb_{q_{\gammav} (\yv)} [f(\yv)]$. Most existing methods either (i) suffer from high variance, seeking help from (often) complicated variance-…
This paper compares gradient estimators in importance-weighted VI and justifies the superiority of DREP over REP.
problem Understanding the impact of gradient estimators on importance-weighted VI algorithms.
method Unified theoretical comparison of reparameterized and doubly-reparameterized gradient estimators tied to IWAE, VR, and VR-IWAE bounds.
result Formally justifies the superiority of doubly-reparameterized gradient estimators over reparameterized ones in importance-weighted VI.
Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guide the search process. Maximizing acquisition functions yields the best performance; unfortunately, t…
Natural gradient descent avoids the magic of model parametrization, leading to different optimization outcomes.
problem Understanding the impact of model parametrization on optimization and generalization in deep learning.
method Characterization of natural gradient flow in deep linear networks and nonlinear neural networks.
result Natural gradient descent fails to generalize in some cases, while gradient descent with the right architecture performs well.
DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.
problem SNR issues in gradient estimates for DGPs with variational inference.
method Adapted doubly reparameterized gradient estimators for DGP training.
result Fix improves predictive performance of DGP models.
A new variational method for SSMs improves inference efficiency.
problem Hard variational inference for state space models.
method Proposes variational marginal particle filter (VMPF) based on Rao-Blackwellization.
result VMPF provides tighter variational bounds and sometimes benefits from unbiased reparameterization.
We investigate a local reparameterizaton technique for greatly reducing the variance of stochastic gradients for variational Bayesian inference (SGVB) of a posterior over model parameters, while retaining parallelizability. This local reparameterization translates uncertainty about global parameters into local noise th…
Previously, the exploding gradient problem has been explained to be central in deep learning and model-based reinforcement learning, because it causes numerical issues and instability in optimization. Our experiments in model-based reinforcement learning imply that the problem is not just a numerical issue, but it may …
Analytical method approximates ELBO gradient in clutter problem.
problem Clutter problem in Bayesian networks with Gaussian likelihood.
method Reparameterization trick, local approximation of likelihood factors.
result Good accuracy and linear computational complexity compared to classical methods.
Stein's method (Stein, 1973; 1981) is a powerful tool for statistical applications and has significantly impacted machine learning. Stein's lemma plays an essential role in Stein's method. Previous applications of Stein's lemma either required strong technical assumptions or were limited to Gaussian distributions with …
Paper proposes a new estimator for generic discrete distributions.
problem Estimating gradients for stochastic nodes in deep generative models.
method Generalized Gumbel-Softmax estimator using truncation, Gumbel-Softmax trick, and linear transformation.
result Efficacy and practical value demonstrated in synthetic examples and topic models.
SDPG algorithm improves sample efficiency and reward in DRL for continuous action spaces.
problem Improving sample efficiency and reward in distributional reinforcement learning for continuous action spaces.
method SDPG algorithm models return distribution using samples via reparameterization technique.
result SDPG shows better sample efficiency and higher reward in OpenAI Gym environments.
Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Boundsstat.ML
Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
The Black Box Variational Inference (Ranganath et al. (2014)) algorithm provides a universal method for Variational Inference, but taking advantage of special properties of the approximation family or of the target can improve the convergence speed significantly. For example, if the approximation family is a transforma…
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
New method improves performance of Hamiltonian MCMC for log Z estimation.
problem Estimating tight bounds on log Z for unnormalized distributions.
method Uncorrected Hamiltonian Annealing (UHA) using reparameterization gradients.
result Better performance and easier parameter tuning compared to existing methods.
Many machine learning tasks require sampling a subset of items from a collection based on a parameterized distribution. The Gumbel-softmax trick can be used to sample a single item, and allows for low-variance reparameterized gradients with respect to the parameters of the underlying distribution. However, stochastic o…
Improved inference for models with continuous latent variables.
problem Inference accuracy with traditional variational methods is limited.
method Reparameterized Variational Rejection Sampling (RVRS) using a proposal distribution with a reparameterized gradient estimator.
result RVRS offers a better trade-off between computational cost and inference fidelity.
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with generalized divergences as a form of estimating the marginal likelihood via biased import…
Reparameterizable densities are an important way to learn probability distributions in a deep learning setting. For many distributions it is possible to create low-variance gradient estimators by utilizing a `reparameterization trick'. Due to the absence of a general reparameterization trick, much research has recently…
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient …
Recent variational inference methods use stochastic gradient estimators whose variance is not well understood. Theoretical guarantees for these estimators are important to understand when these methods will or will not work. This paper gives bounds for the common "reparameterization" estimators when the target is smoot…
Enhances sample diversity in SGMCMC for better uncertainty estimation in BNNs.
problem Limited sample diversity in SGMCMC affects uncertainty estimation and model performance.
method Reparameterizes neural network weights to produce a more diverse set of samples.
result The proposed approach achieves superior performance in image classification tasks, including OOD robustness.
REP-GAN improves GANs by reparameterizing proposals for better sample quality and efficiency.
problem Poor sample efficiency in GANs due to independent proposal sampling.
method REParameterizing Markov chains into the latent space of the generator to create dependent proposals.
result Empirically shows significant improvement in sample efficiency and quality.
Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations. In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of E…