arXiv research

New algorithm improves volatility forecasting using Pairwise Markov Chains.

The Viterbi process can be extended indefinitely in a pairwise Markov model.

A Markov Chain approach for aligning generative models from pairwise human preferences.

As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Cha…

Algorithm learns mixtures of Markov chains and MDPs from short trajectories.

We present a new family of models that is based on graphs that may have undirected, directed and bidirected edges. We name these new models marginal AMP (MAMP) chain graphs because each of them is Markov equivalent to some AMP chain graph under marginalization of some of its nodes. However, MAMP chain graphs do not onl…

A new method estimates protein evolutionary fields and couplings from alignments.

Bayesian method selects interacting regions in Markov models.

PDHAMS improves sampling for discrete distributions with quadratic potential functions.

Pairwise Choice Markov Chains (PCMC) have been recently introduced to overcome limitations of choice models based on traditional axioms unable to express empirical observations from modern behavior economics like context effects occurring when a choice between two options is altered by adding a third alternative. The i…

The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Mark…

The paper advances U-statistics in dependent settings, improving spectral estimation and goodness-of-fit tests.

This paper deals with chain graphs under the Andersson-Madigan-Perlman (AMP) interpretation. In particular, we present a constraint based algorithm for learning an AMP chain graph a given probability distribution is faithful to. Moreover, we show that the extension of Meek's conjecture to AMP chain graphs does not hold…

We introduce a new class of graphical models that generalizes Lauritzen-Wermuth-Frydenberg chain graphs by relaxing the semi-directed acyclity constraint so that only directed cycles are forbidden. Moreover, up to two edges are allowed between any pair of nodes. Specifically, we present local, pairwise and global Marko…

An intervention may have an effect on units other than those to which it was administered. This phenomenon is called interference and it usually goes unmodeled. In this paper, we propose to combine Lauritzen-Wermuth-Frydenberg and Andersson-Madigan-Perlman chain graphs to create a new class of causal models that can re…

We extend Andersson-Madigan-Perlman chain graphs by (i) relaxing the semidirected acyclity constraint so that only directed cycles are forbidden, and (ii) allowing up to two edges between any pair of nodes. We introduce global, and ordered local and pairwise Markov properties for the new models. We show the equivalence…

The paper provides concentration inequalities for Markov chain variance estimators.

A new method simulates a lazy version of a Markov chain for empirical inference.

Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…

Study Markov chain gradient descent in Hilbert spaces for quadratic loss.

Study nonparametric estimator for Markov chain transition matrices in offline setting.

Enhanced Markov chain sampler learns network statistics faster.

Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…

Expands Hidden Markov Model to include Markov chain observations.

In this paper we describe three stochastic models based on a semi-Markov chains approach and its generalizations to study the high frequency price dynamics of traded stocks. The three models are: a simple semi-Markov chain model, an indexed semi-Markov chain model and a weighted indexed semi-Markov chain model. We show…

New insights into Markov chain geometry via positive transition measures.

In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…

New method estimates convergence bounds for nonlinear Markov chains.

Elo ratings learn model parameters quickly using Markov chains.

DCDC calculates convergence rates for Markov chains using neural networks.

The paper studies how quickly samples from Langevin dynamics become independent.

We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains sequentially to observe their states. The goal of the learner is to sequentially…

Method reconstructs hidden Markov chains from insurance data.

This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.

The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.

Identity testing for reversible Markov chains without symmetry assumption.

Unbiased gradient estimation for Markov chains

Generates random persistence diagrams for data analysis.

The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Markov chain. This kind of block selection is neither i.i.d. random nor cyclic. On the other hand, it is…

New framework improves variational inference with Markov chain methods.

This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…

The paper develops new inequalities for Markov chain sums, linking them to mixing time.

Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.

The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior where a distributed ADMM algorithm can be seen as a lifting of Gradient Descent…

Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.

We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix $P$ or $\text{Dist}(P, Q) \geq ε$ where $\text{Dist}$ is suitably defined. In r…

We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state of the underlying chain, the integrand must be of a specific form. This allows u…