arXiv research

The book covers scalable MCMC methods for Bayesian learning.

HDT improves MCMC on graphs with history-dependent sampling.

New PDMP samplers tackle variable selection in models.

Sampling the parameters of high-dimensional Continuous Time Markov Chains (CTMC) is a challenging problem with important applications in many fields of applied statistics. In this work a recently proposed type of non-reversible rejection-free Markov Chain Monte Carlo (MCMC) sampler, the Bouncy Particle Sampler (BPS), i…

Boundary rigidity proven for non-reversible Finsler metrics.

Blang simplifies Bayesian analysis for non-standard data types.

Characterizes isometries between non-reversible Finsler manifolds.

A new sampler speeds up Bayesian mixture models.

Adaptive algorithm improves convergence rate of Langevin dynamics.

We continue our study of geometric analysis on (possibly non-reversible) Finsler manifolds, based on the Bochner inequality established by the author and Sturm. Following the approach of the $Γ$-calculus a la Bakry et al, we show the dimensional versions of the Poincare--Lichnerowicz inequality, the logarithmic Sobolev…

NSGLD improves SGLD for non-convex optimization problems.

Study on how non-reversible diffusion processes affect homology on manifolds.

Study of parabolas in Funk metric on unit disk.

In the asymmetric setting, Hilbert's fourth problem asks to construct and study all (non-reversible) projective Finsler metrics: Finsler metrics defined on open, convex subsets of real projective $n$-space for which geodesics lie on projective lines. While asymmetric norms and Funk metrics provide many examples of esse…

Langevin dynamics (LD) has been proven to be a powerful technique for optimizing a non-convex objective as an efficient algorithm to find local minima while eventually visiting a global minimum on longer time-scales. LD is based on the first-order Langevin diffusion which is reversible in time. We study two variants th…

A new sampler improves the inference of causal structures from observational data.

We develop the basics of a theory of almost isometries for spaces endowed with a quasi-metric. The case of non-reversible Finsler (more specifically, Randers) metrics is of particular interest, and it is studied in more detail. The main motivation arises from General Relativity, and more specifically in spacetimes endo…

The paper analyzes variance reduction in stochastic gradient Langevin dynamics.

We show the existence of at least two geometrically distinct closed geodesics on an n-dimensional sphere with a bumpy and non-reversible Finsler metric for n>2.

For non-reversible Finsler metrics of positive flag curvature on spheres and projective spaces we present results about the number and the length of closed geodesics and about their stability properties.

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…

New method uses PDMPs with sub-sampling for efficient sampling from posterior distributions.

In this paper two metric properties on geodesic length spaces are introduced by means of the metric projection, studying their validity on Alexandrov and Busemann NPC spaces. In particular, we prove that both properties characterize the non-positivity of the sectional curvature on Riemannian manifolds. Further results …

Estimates Markov chain mixing time from a single trajectory.

We give a lower bound for the length of a non-trivial geodesic loop on a simply-connected and compact manifold of even dimension with a non-reversible Finsler metric of positive flag curvature. Harris and Paternain use this estimate in their recent paper [HP] to give a geometric characterization of dynamically convex F…

The existence of two geometrically distinct closed geodesics on an $n$-dimensional sphere $S^n$ with a non-reversible and bumpy Finsler metric was shown independently by Duan--Long [7] and the author [27]. We simplify the proof of this statement by the following observation: If for some $N \in \mathbb{N}$ all closed ge…

For odd-dimensional spheres, there's always a second short geodesic.

Unified framework for MCMC algorithms simplifies their design and application.

In this article, we show that a Finsler--Laplacian introduced previously can detect changes in the Finsler metric that the marked length spectrum cannot. We also construct examples of non-reversible Finsler metrics in negative curvature such that $4λ_1 > h^2$, where $λ_1$ is the bottom of the $L^2$-spectrum and $h$ the…

The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets…

The Bouncy Particle Sampler is a novel rejection-free non-reversible sampler for differentiable probability distributions over continuous variables. We generalize the algorithm to piecewise differentiable distributions and apply it to generic binary distributions using a piecewise differentiable augmentation. We illust…

Unified framework for MCMC and machine learning problems.

JaxSGMC simplifies SG-MCMC for Bayesian deep learning.

Paper proposes first unlearning algorithm for MCMC models.

TensorFlow Probability MCMC toolkit improves MCMC efficiency for modern hardware.

Stochastic EM with biased MCMC improves inference stability.

It is known that the Langevin dynamics used in MCMC is the gradient flow of the KL divergence on the Wasserstein space, which helps convergence analysis and inspires recent particle-based variational inference methods (ParVIs). But no more MCMC dynamics is understood in this way. In this work, by developing novel conce…

Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, s…

Cyclical MCMC tackles high-dimensional multimodal distributions, showing convergence under certain conditions.

Paper compares MCMC-based copula methods for exchange option pricing.

This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform no…

Nonlinear MCMC improves Bayesian machine learning sampling.

Statistical inference methods are fundamentally important in machine learning. Most state-of-the-art inference algorithms are variants of Markov chain Monte Carlo (MCMC) or variational inference (VI). However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods …

We give the details of the proof of the equality between the critical groups, with respect the H^1 and C^1 topology, at a non-degenerate critical point of the energy functional of a non-reversible Finsler manifold (M,F), defined on the Hilbert manifold of the H^1 curves connecting two given points on M.

Particle MCMC involves using a particle filter within an MCMC algorithm. For inference of a model which involves an unobserved stochastic process, the standard implementation uses the particle filter to propose new values for the stochastic process, and MCMC moves to propose new values for the parameters. We show how p…

Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling Bayesian inference to large datasets under an assumption of i.i.d data. We instead develop an SG-MCMC algorithm to learn the parameters of hidden Markov models (HMMs) for time-dependent data. There are two challenges to applying SG-MCMC in this…

New analysis of SGD with MCMC gradient estimator shows convergence rate and saddle point escape.

Proposes MIVI for efficient posterior estimation and design of MCMC transitions.