arXiv research

A new method simulates a lazy version of a Markov chain for empirical inference.

New concentration inequality for U-statistics of Markov chains.

The paper provides concentration inequalities for Markov chain variance estimators.

Markov Chain Monte Carlo is repeatedly used to analyze the properties of intractable distributions in a convenient way. In this paper we derive conditions for geometric ergodicity of a general class of nonparametric stochastic volatility models with skewness driven by hidden Markov Chain with switching.

New algorithm clusters trajectories from multiple Markov chains with near-optimal error.

Study nonparametric estimator for Markov chain transition matrices in offline setting.

New method estimates convergence bounds for nonlinear Markov chains.

The paper establishes CLTs for Markov chains and improves sampling algorithms for heavy-tailed distributions.

We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors) upper and lower sample complexity bounds for our notion of distance, which is bas…

The Riemannian barycentre is one of the most widely used statistical descriptors for probability distributions on Riemannian manifolds. At present, existing algorithms are able to compute the Riemannian barycentre of a probability distribution, only if i.i.d. samples of this distribution are readily available. However,…

Elliptical slice sampling converges geometrically, providing reliable sampling for Bayesian learning.

Optimizes MCMC chains with neural control variates.

U-turn chains improve sampling from complex distributions.

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…

The paper tackles learning from non-irreducible Markov chains, proving learnability and generalization bounds.

The paper develops new inequalities for Markov chain sums, linking them to mixing time.

New algorithm identifies best policy in MDPs faster.

We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains sequentially to observe their states. The goal of the learner is to sequentially…

The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.

Optimal sequential testing for Markovian data with lower and upper bounds.

Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems. This paper studies the decentralized Markov chain gradient descent (DMGD) algorithm - a variant of the decentralized stochastic gradient methods where the random samples are taken along the trajec…

In this paper we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific str…

In Bayesian statistics, many problems can be expressed as the evaluation of the expectation of a quantity of interest with respect to the posterior distribution. Standard Monte Carlo method is often not applicable because the encountered posterior distributions cannot be sampled directly. In this case, the most popular…

Paper studies CLT rates for dependent data in Wasserstein-p distance.

New MCMC methods map high-dimensional problems to spheres for better mixing.

We analyze the generalization and robustness of the batched weighted average algorithm for V-geometrically ergodic Markov data. This algorithm is a good alternative to the empirical risk minimization algorithm when the latter suffers from overfitting or when optimizing the empirical risk is hard. For the generalization…

New sampling methods improve statistical efficiency for intractable targets.

Approximate inference algorithm is one of the fundamental research fields in machine learning. The two dominant theoretical inference frameworks in machine learning are variational inference (VI) and Markov chain Monte Carlo (MCMC). However, because of the fundamental limitation in the theory, it is very challenging to…

Stochastic approximation algorithms show exponential progress bounds.

We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while…

STANLEY improves sampling for complex data models.

We establish general conditions under which Markov chains produced by the Hamiltonian Monte Carlo method will and will not be geometrically ergodic. We consider implementations with both position-independent and position-dependent integration times. In the former case we find that the conditions for geometric ergodicit…

Study approximates financial market with discrete-time models.

Markov Chain Monte Carlo methods become increasingly popular in applied mathematics as a tool for numerical integration with respect to complex and high-dimensional distributions. However, application of MCMC methods to heavy tailed distributions and distributions with analytically intractable densities turns out to be…

We introduce a multivariate Hawkes process with constraints on its conditional density. It is a multivariate point process with conditional intensity similar to that of a multivariate Hawkes process but certain events are forbidden with respect to boundary conditions on a multidimensional constraint variable, whose evo…

We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter $t_{\rm mix}$. These bounds are the first regret bounds in the general, non-epi…

A new metric based on hitting probabilities for directed graphs and Markov chains.

This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time $t_{\text{mix}}$ of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from a single finite-length sample path from the Markov chain, and does not require t…

The paper analyzes fixed step-size SA schemes on Riemannian manifolds.

The paper analyzes covariate shift in nonparametric regression with Markovian data.

The paper analyzes stability of random matrix products with Markovian noise.

Study on the limits of learning HMM parameters under various conditions.

This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued tha…

The paper advances U-statistics in dependent settings, improving spectral estimation and goodness-of-fit tests.

The paper analyzes learning rates for non-irreducible Markov chains.

Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC) methods are commonly used in learning graphical models. However, MCMC cannot be directly applied in learning topic models because of the quasi-ergodicity problem caused by multimodal distribution of topics. In this paper, we develop an embarra…

Combines local and global samplers for efficient sampling.

We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are defined using features associated with the state-action pairs. The features are kno…