A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Gradient-based MCMC for discrete spaces improves sampling performance.
We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal di…
Statistical inference methods are fundamentally important in machine learning. Most state-of-the-art inference algorithms are variants of Markov chain Monte Carlo (MCMC) or variational inference (VI). However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods …
We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an underdamped form of the Langevin algorithm performs accelerated gradient descent in this metric. To characterize t…
New methods combine MALA and mGRAD for scalable Bayesian inference in high-dimensional state-space models.
New method for MCMC models without perfect or sequential samplers.
Differentiable relaxation for inferring partial orders from noisy linear data.
RLMH improves adaptive MCMC by optimizing contrastive divergence reward.
This work improves VAEs using MCMC methods for better variational bounds.
Study improves Bayesian calibration of mechanical properties using active learning and MCMC.
It is challenging to develop stochastic gradient based scalable inference for deep discrete latent variable models (LVMs), due to the difficulties in not only computing the gradients, but also adapting the step sizes to different latent factors and hidden layers. For the Poisson gamma belief network (PGBN), a recently …
Improved sampling accuracy in SG-MCMC methods via non-uniform gradient subsampling.
Improved text generation with constraints using discrete auto-regressive biasing.
ADVI speeds up Bayesian inference for bridge regression models.
This research accelerates sampling methods using Nesterov's Acceleration.
New PDMP samplers tackle variable selection in models.
SVI and GP surrogates improve calibration of ABMs in epidemiology.
Automatically differentiable estimation for BLP model reduces bias in demand estimation.
Single-site Markov Chain Monte Carlo (MCMC) is a variant of MCMC in which a single coordinate in the state space is modified in each step. Structured relational models are a good candidate for this style of inference. In the single-site context, second order methods become feasible because the typical cubic costs assoc…
QEM uses parallel importance weighting for fast approximate Bayesian inference.
Paper proposes CoopFlow, a two-flow generator for energy-based models.
Unified framework for MCMC algorithms simplifies their design and application.
The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets…
Unified framework for MCMC and machine learning problems.
JaxSGMC simplifies SG-MCMC for Bayesian deep learning.
Paper proposes first unlearning algorithm for MCMC models.
Stochastic EM with biased MCMC improves inference stability.
It is known that the Langevin dynamics used in MCMC is the gradient flow of the KL divergence on the Wasserstein space, which helps convergence analysis and inspires recent particle-based variational inference methods (ParVIs). But no more MCMC dynamics is understood in this way. In this work, by developing novel conce…
Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, s…
Cyclical MCMC tackles high-dimensional multimodal distributions, showing convergence under certain conditions.
Paper compares MCMC-based copula methods for exchange option pricing.
This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform no…
We introduce a new family of MCMC samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalisation over the auxiliary variables yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well…
Nonlinear MCMC improves Bayesian machine learning sampling.
Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new values for the stochastic process, and MCMC moves to propose new values for the parameters. We show how p…
Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling Bayesian inference to large datasets under an assumption of i.i.d data. We instead develop an SG-MCMC algorithm to learn the parameters of hidden Markov models (HMMs) for time-dependent data. There are two challenges to applying SG-MCMC in this…
New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.
Proposes MIVI for efficient posterior estimation and design of MCMC transitions.
This study evaluates different normalizing flow architectures for MCMC.
AutoStep MCMC adapts step size locally for better sampling efficiency.
Markov chain Monte Carlo (MCMC) is a popular and successful general-purpose tool for Bayesian inference. However, MCMC cannot be practically applied to large data sets because of the prohibitive cost of evaluating every likelihood term at every iteration. Here we present Firefly Monte Carlo (FlyMC) an auxiliary variabl…
Stochastic gradient MCMC (SG-MCMC) has played an important role in large-scale Bayesian learning, with well-developed theoretical convergence properties. In such applications of SG-MCMC, it is becoming increasingly popular to employ distributed systems, where stochastic gradients are computed based on some outdated par…
The book covers scalable MCMC methods for Bayesian learning.
We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework where the likelihood function for observations is estimated from a random subset of observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smal…
The modern scale of data has brought new challenges to Bayesian inference. In particular, conventional MCMC algorithms are computationally very expensive for large data sets. A promising approach to solve this problem is embarrassingly parallel MCMC (EP-MCMC), which first partitions the data into multiple subsets and r…
New method reduces computational cost for Bayesian inference.
Sampling from posterior distributions using Markov chain Monte Carlo (MCMC) methods can require an exhaustive number of iterations, particularly when the posterior is multi-modal as the MCMC sampler can become trapped in a local mode for a large number of iterations. In this paper, we introduce the pseudo-extended MCMC…
Bayesian neural networks tutorial via MCMC in Python.