arXiv research

New assumptions and algorithm solve offline two-player zero-sum Markov games.

Identity testing for reversible Markov chains without symmetry assumption.

New algorithm estimates causal effects for non-Gaussian data.

The paper shows that relaxing assumptions about causal graphs can lead to exponentially large equivalence classes.

The paper defines conditions for learning causal graphs from data with unobserved variables.

Bayesian model selection improves causal discovery in complex datasets.

This work addresses the following question: Under what assumptions on the data generating process can one infer the causal graph from the joint distribution? The approach taken by conditional independence-based causal discovery methods is based on two assumptions: the Markov condition and faithfulness. It has been show…

Algorithm learns Bayesian network structure efficiently from data.

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

We consider the restless Markov bandit problem, in which the state of each arm evolves according to a Markov process independently of the learner's actions. We suggest an algorithm that after $T$ steps achieves $\tilde{O}(\sqrt{T})$ regret with respect to the best policy that knows the distributions of all arms. No ass…

The paper tackles learning from non-irreducible Markov chains, proving learnability and generalization bounds.

Proposes a weaker faithfulness assumption for causal discovery.

New algorithm detects changes in Markov kernels with unknown post-change kernel.

Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We w…

Paper tackles anomaly detection in restless Markov arms with unknown TPMs.

The study revisits portfolio diversification by relaxing assumptions for skewed, multi-regime, and leptokurtic asset returns.

We study the high frequency price dynamics of traded stocks by a model of returns using a semi-Markov approach. More precisely we assume that the intraday return are described by a discrete time homogeneous semi-Markov process and the overnight returns are modeled by a Markov chain. Based on this assumptions we derived…

Study approximates financial market with discrete-time models.

The paper estimates key metrics for linear models with Markov or hidden Markov sources.

Study on gradient descent in Hilbert spaces with Markov chains, focusing on mixing coefficients.

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…

We consider the problem of constructing an appropriate multivariate model for the study of the counterparty credit risk in credit rating migration problem. For this financial problem different multivariate Markov chain models were proposed. However the markovian assumption may be inappropriate for the study of the dyna…

Unified approach to denoising Markov models for efficient sampling.

New Q-learning method achieves optimal sample complexity for average-reward problems.

We have developed a statistical technique to test the model assumption of binary regime switching extension of the geometric Brownian motion (GBM) model by proposing a new discriminating statistics. Given a time series data, we have identified an admissible class of the regime switching candidate models for the statist…

New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.

This paper detects Markov violations in RL with noise, improving policy development.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

New PDMP samplers improve BNN inference with accelerated computation.

Paper introduces DMPMs for efficient discrete data generation with sharp convergence bounds.

Markov models lie at the interface between statistical independence in a probability distribution and graph separation properties. We review model selection and estimation in directed and undirected Markov models with Gaussian parametrization, emphasizing the main similarities and differences. These two model classes a…

Study identifies and validates a method for system identification of Markov jump linear systems.

Paper tackles HMM learning with unknown missing observation locations.

Study improves generalization bounds for equivariant networks on Markov data.

Markov networks are extensively used to model complex sequential, spatial, and relational interactions in a wide range of fields. By learning the structure of independences of a domain, more accurate joint probability distributions can be obtained for inference tasks or, more directly, for interpreting the most signifi…

Risk measures applied to dynamic Markov processes with varying risk aversion.

New method avoids curse of dimensionality in structured density estimation.

We present a new method for nonlinear prediction of discrete random sequences under minimal structural assumptions. We give a mathematical construction for optimal predictors of such processes, in the form of hidden Markov models. We then describe an algorithm, CSSR (Causal-State Splitting Reconstruction), which approx…

The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in parameter space. Partial answers so far available depend on extra assumptions on the pro…

A Semi-Hidden Markov Model (SHMM) for bursty error channels is defined by a state transition probability matrix $A$, a prior probability vector $Π$, and the state dependent output symbol error probability matrix $B$. Several processes are utilized for estimating $A$, $Π$ and $B$ from a given empirically obtained or sim…

We symmetrise neural networks for groups using Markov categories.

We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal prediction, where at each stage the recursion depends on the whole future. The propert…

Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC)…

This paper compares HMM and LSTM for time series forecasting.

Undirected graphical models known as Markov networks are popular for a wide variety of applications ranging from statistical physics to computational biology. Traditionally, learning of the network structure has been done under the assumption of chordality which ensures that efficient scoring methods can be used. In ge…

Algorithm minimizes regret and converges to equilibria in Markov games.

Algorithm improves RL by discovering delayed causal relations.

New empirical PAC-Bayes bound for Markov chains with finite state space.