A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method enhances model fine-tuning with minimal data.
JKO-iFlow uses neural ODEs to improve generative models with reduced memory and training complexity.
A new method for optimizing models with categorical variables using diffusion.
Two new estimators improve VAE training for hierarchical and prior parameters.
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterizat…
Reparameterizes mirror descent as gradient descent for efficient sparse learning.
Recent breakthroughs in computer vision make use of large deep neural networks, utilizing the substantial speedup offered by GPUs. For applications running on limited hardware, however, high precision real-time processing can still be a challenge. One approach to solving this problem is training networks with binary or…
REP-GAN improves GANs by reparameterizing proposals for better sample quality and efficiency.
Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations. The reparameterization trick is applicable when we can simulate a random variable by applying a differ…
The reparameterization trick has become one of the most useful tools in the field of variational inference. However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution fu…
By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models. However, it is not applicable to a number of important continuous distributions. We introduce an a…
We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterizati…
BayesAdapter turns pre-trained NNs into reliable BNNs with minimal overhead.
The reparameterization trick is widely used in variational inference as it yields more accurate estimates of the gradient of the variational objective than alternative approaches such as the score function method. Although there is overwhelming empirical evidence in the literature showing its success, there is relative…
Low-variance gradient estimation is crucial for learning directed graphical models parameterized by neural networks, where the reparameterization trick is widely used for those with continuous variables. While this technique gives low-variance gradient estimates, it has not been directly applicable to discrete variable…
DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.
TFM trains Neural SDEs without backpropagation, improving clinical time series modeling.
A new method for stochastic optimal control improves accuracy over existing techniques.
Geometric correspondence links flow metrics to reparameterizations.
Adaptive-stepsize MCMC sampling inspired by Adam optimizer.
New measure of maximal entropy found for a class of geometrically finite groups.
The paper addresses the invariance issue in Bayesian neural networks using linearized Laplace approximation.
Paper presents a reparameterized DP-DLGMM for clustering.
The performance of deep neural networks is often attributed to their automated, task-related feature construction. It remains an open question, though, why this leads to solutions with good generalization, even in cases where the number of parameters is larger than the number of samples. Back in the 90s, Hochreiter and…
Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guide the search process. Maximizing acquisition functions yields the best performance; unfortunately, t…
Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian distribution. Recent works demonstrated that choice of prior distribution affects learning capacity of VAE models. We propose a general technique (embedding-reparameterization procedure, or ER) for introducing arbitrary manifold-valued variables in VAE…
The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution. After refactoring, the gradients of the loss propag…
We investigate a local reparameterizaton technique for greatly reducing the variance of stochastic gradients for variational Bayesian inference (SGVB) of a posterior over model parameters, while retaining parallelizability. This local reparameterization translates uncertainty about global parameters into local noise th…
This research simplifies verification of machine learning systems using reparameterization.
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the repa…
We stabilize the Kumaraswamy distribution for efficient sampling and differentiation.
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimator…
The Gumbel-Softmax is a continuous distribution over the simplex that is often used as a relaxation of discrete distributions. Because it can be readily interpreted and easily reparameterized, it enjoys widespread use. We propose a modular and more flexible family of reparameterizable distributions where Gaussian noise…
Unified view of LR and RP gradients explained via divergence theorem.
Paper tackles performative prediction without convexity assumptions.
EXPO framework eliminates need for reward model, achieving better optimization.
New method for analyzing learning dynamics in singular models.
New method improves imitation learning from expert observations.
This paper focuses on the study of open curves in a manifold M, and proposes a reparameterization invariant metric on the space of such paths. We use the square root velocity function (SRVF) introduced by Srivastava et al. in [11] to define a reparameterization invariant metric on the space of immersions M' = Imm([0,1]…
Reparameterizable densities are an important way to learn probability distributions in a deep learning setting. For many distributions it is possible to create low-variance gradient estimators by utilizing a `reparameterization trick'. Due to the absence of a general reparameterization trick, much research has recently…
Boosts linear neurons for faster convergence in neural networks.
Gradient descent implicitly favors group sparsity in neural networks.
Online (also called "recursive" or "adaptive") estimation of fixed model parameters in hidden Markov models is a topic of much interest in times series modelling. In this work, we propose an online parameter estimation algorithm that combines two key ideas. The first one, which is deeply rooted in the Expectation-Maxim…
DiMS sampler explores neural network loss minima via dissipative dynamics.
Natural gradient descent is a robust optimization method for machine learning.
RCS-YOLO improves brain tumor detection speed and accuracy.
Improved online learning for hidden-convex losses achieves optimal regret.
New gradient estimators for discrete variables improve model training.