A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new method for stochastic optimal control improves accuracy over existing techniques.
Machine learning improves high-dimensional matrix estimation.
Many matching, tracking, sorting, and ranking problems require probabilistic reasoning about possible permutations, a set that grows factorially with dimension. Combinatorial optimization algorithms may enable efficient point estimation, but fully Bayesian inference poses a severe challenge in this high-dimensional, di…
Generative Adversarial Networks (GANs), though powerful, is hard to train. Several recent works (brock2016neural,miyato2018spectral) suggest that controlling the spectra of weight matrices in the discriminator can significantly improve the training of GANs. Motivated by their discovery, we propose a new framework for t…
Enhances sample diversity in SGMCMC for better uncertainty estimation in BNNs.
Gradient flow on softmax attention minimizes nuclear norm of weight matrices.
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterizat…
Paper generalizes reparameterization trick for broader applicability.
Natural gradient descent avoids the magic of model parametrization, leading to different optimization outcomes.
Reparameterizes mirror descent as gradient descent for efficient sparse learning.
REP-GAN improves GANs by reparameterizing proposals for better sample quality and efficiency.
Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations. The reparameterization trick is applicable when we can simulate a random variable by applying a differ…
Sorting input objects is an important step in many machine learning pipelines. However, the sorting operator is non-differentiable with respect to its inputs, which prohibits end-to-end gradient-based optimization. In this work, we propose NeuralSort, a general-purpose continuous relaxation of the output of the sorting…
By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models. However, it is not applicable to a number of important continuous distributions. We introduce an a…
New flatness measure for neural nets is invariant to reparameterizations.
A new method for optimizing models with categorical variables using diffusion.
New geometric structures defined on SPD matrices for better understanding.
Two new estimators improve VAE training for hierarchical and prior parameters.
We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterizati…
A new method reparameterizes Gaussian noise for better flexibility and performance.
The reparameterization trick is widely used in variational inference as it yields more accurate estimates of the gradient of the variational objective than alternative approaches such as the score function method. Although there is overwhelming empirical evidence in the literature showing its success, there is relative…
Low-variance gradient estimation is crucial for learning directed graphical models parameterized by neural networks, where the reparameterization trick is widely used for those with continuous variables. While this technique gives low-variance gradient estimates, it has not been directly applicable to discrete variable…
Geometric correspondence links flow metrics to reparameterizations.
New measure of maximal entropy found for a class of geometrically finite groups.
The paper addresses the invariance issue in Bayesian neural networks using linearized Laplace approximation.
Paper presents a reparameterized DP-DLGMM for clustering.
Bayesian optimization is a sample-efficient approach to solving global optimization problems. Along with a surrogate model, this approach relies on theoretically motivated value heuristics (acquisition functions) to guide the search process. Maximizing acquisition functions yields the best performance; unfortunately, t…
Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian distribution. Recent works demonstrated that choice of prior distribution affects learning capacity of VAE models. We propose a general technique (embedding-reparameterization procedure, or ER) for introducing arbitrary manifold-valued variables in VAE…
The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution. After refactoring, the gradients of the loss propag…
We investigate a local reparameterizaton technique for greatly reducing the variance of stochastic gradients for variational Bayesian inference (SGVB) of a posterior over model parameters, while retaining parallelizability. This local reparameterization translates uncertainty about global parameters into local noise th…
This research simplifies verification of machine learning systems using reparameterization.
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the repa…
STR reparameterizes DNN weights with soft thresholds for better sparsity and accuracy.
Recent breakthroughs in computer vision make use of large deep neural networks, utilizing the substantial speedup offered by GPUs. For applications running on limited hardware, however, high precision real-time processing can still be a challenge. One approach to solving this problem is training networks with binary or…
We stabilize the Kumaraswamy distribution for efficient sampling and differentiation.
Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimator…
New method enhances model fine-tuning with minimal data.
Unified view of LR and RP gradients with improved importance sampling.
Unified view of LR and RP gradients explained via divergence theorem.
This work presents a novel approach to train invertible linear layers by adding rank-one perturbations.
Paper tackles performative prediction without convexity assumptions.
Investigates how flatness of loss curve relates to generalization in machine learning models.
EXPO framework eliminates need for reward model, achieving better optimization.
New method for analyzing learning dynamics in singular models.
New techniques make fair prediction models easier to train.
New method improves imitation learning from expert observations.
This paper focuses on the study of open curves in a manifold M, and proposes a reparameterization invariant metric on the space of such paths. We use the square root velocity function (SRVF) introduced by Srivastava et al. in [11] to define a reparameterization invariant metric on the space of immersions M' = Imm([0,1]…
Reparameterizable densities are an important way to learn probability distributions in a deep learning setting. For many distributions it is possible to create low-variance gradient estimators by utilizing a `reparameterization trick'. Due to the absence of a general reparameterization trick, much research has recently…