arXiv research

New method estimates convergence bounds for nonlinear Markov chains.

Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…

GO-OED maximizes predictive information gain on nonlinear QoIs.

Most previous contributions to BSDEs, and the related theories of nonlinear expectation and dynamic risk measures, have been in the framework of continuous time diffusions or jump diffusions. Using solutions of BSDEs on spaces related to finite state, continuous time Markov chains, we develop a theory of nonlinear expe…

BINDy uses Bayesian methods to identify nonlinear dynamics from data.

New methods solve complex financial equations.

Flexible model for complex relationships using Bayesian nonparametrics.

We provide conditions for the existence and the unicity of strictly stationary solutions of the usual Dynamic Conditional Correlation GARCH models (DCC-GARCH). The proof is based on Tweedie's (1988) criteria, after having rewritten DCC-GARCH models as nonlinear Markov chains. Moreover, we study the existence of their f…

New neural network method simplifies high-dimensional data.

FBMS R package simplifies Bayesian model selection and averaging.

This paper develops a novel stochastic tree ensemble method for nonlinear regression, which we refer to as XBART, short for Accelerated Bayesian Additive Regression Trees. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursive partiti…

We propose dynamical systems trees (DSTs) as a flexible class of models for describing multiple processes that interact via a hierarchy of aggregating parent chains. DSTs extend Kalman filters, hidden Markov models and nonlinear dynamical systems to an interactive group scenario. Various individual processes interact a…

Jump Markov linear models consists of a finite number of linear state space models and a discrete variable encoding the jumps (or switches) between the different linear models. Identifying jump Markov linear models makes for a challenging problem lacking an analytical solution. We derive a new expectation maximization …

The paper provides concentration inequalities for Markov chain variance estimators.

We propose kernel sequential Monte Carlo (KSMC), a framework for sampling from static target densities. KSMC is a family of sequential Monte Carlo algorithms that are based on building emulator models of the current particle system in a reproducing kernel Hilbert space. We here focus on modelling nonlinear covariance s…

A new method simulates a lazy version of a Markov chain for empirical inference.

Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…

Study Markov chain gradient descent in Hilbert spaces for quadratic loss.

Study nonparametric estimator for Markov chain transition matrices in offline setting.

STANLEY improves sampling for complex data models.

A joint conditional autoregressive expectile and Expected Shortfall framework is proposed. The framework is extended through incorporating a measurement equation which models the contemporaneous dependence between the realized measures and the latent conditional expectile. Nonlinear threshold specification is further i…

Enhanced Markov chain sampler learns network statistics faster.

Novel CMG framework improves financial sentiment forecasting.

NoLimits.jl: Flexible and Composable Nonlinear Mixed-Effects Modeling in Julia

Expands Hidden Markov Model to include Markov chain observations.

In this paper we describe three stochastic models based on a semi-Markov chains approach and its generalizations to study the high frequency price dynamics of traded stocks. The three models are: a simple semi-Markov chain model, an indexed semi-Markov chain model and a weighted indexed semi-Markov chain model. We show…

New insights into Markov chain geometry via positive transition measures.

In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…

Elo ratings learn model parameters quickly using Markov chains.

DCDC calculates convergence rates for Markov chains using neural networks.

The paper studies how quickly samples from Langevin dynamics become independent.

We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains sequentially to observe their states. The goal of the learner is to sequentially…

Method reconstructs hidden Markov chains from insurance data.

This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.

The paper extends Hoeffding's inequality for Markov chains using a generalized concentrability condition.

Identity testing for reversible Markov chains without symmetry assumption.

Unbiased gradient estimation for Markov chains

The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Markov chain. This kind of block selection is neither i.i.d. random nor cyclic. On the other hand, it is…

New framework improves variational inference with Markov chain methods.

This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…

The paper develops new inequalities for Markov chain sums, linking them to mixing time.

Matrix Chernoff bound for Markov chains applied to co-occurrence matrices.

The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior where a distributed ADMM algorithm can be seen as a lifting of Gradient Descent…

Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.

Algorithm learns mixtures of Markov chains and MDPs from short trajectories.

We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix $P$ or $\text{Dist}(P, Q) \geq ε$ where $\text{Dist}$ is suitably defined. In r…

We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state of the underlying chain, the integrand must be of a specific form. This allows u…