arXiv research

Direct approach for handling contextual bandits with latent state dynamics.

New model for bandit problem with linear rewards and side information.

Study of multi-armed bandits with state-switching rewards using Markov models.

Expands Hidden Markov Model to include Markov chain observations.

This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian non…

Hidden Markov Neural Networks balance adaptation and forgetting in time-series data.

The paper estimates key metrics for linear models with Markov or hidden Markov sources.

This work speeds up fHMM analysis by tensor algebra.

This paper compares HMM and LSTM for time series forecasting.

Investor selects portfolios based on news attention in a hidden Markov model.

The paper introduces FMCI and hybrid decoding for hidden Markov models.

Modified asymmetric hidden Markov models for time series with autoregressive components.

NoMoPy models noise as HMM/FHMM in Python.

Study symmetric linear bandits with hidden symmetry, achieving improved regret bounds.

The paper develops a Hoeffding inequality for Markov chains and applies it to bandit problems.

Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.

Hierarchical hidden Markov models predict market trends in financial time series.

A new HMM model captures kernel dependencies using context-specific Bayesian networks.

New algorithm for collective Gaussian hidden Markov models inference.

Study evaluates initialization strategies for infinite hidden Markov models.

Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.

Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We w…

Algorithm BGLM-OFU minimizes regret in combinatorial causal bandits with binary models.

Detects anomalies in multiple processes using hidden Markov models.

We define a Hidden Markov Model (HMM) in which each hidden state has time-dependent $\textit{activity levels}$ that drive transitions and emissions, and show how to estimate its parameters. Our construction is motivated by the problem of inferring human mobility on sub-daily time scales from, for example, mobile phone …

The study uses Bayesian Hidden Markov Models to predict cryptocurrency returns.

The partially observable hidden Markov model is an extension of the hidden Markov Model in which the hidden state is conditioned on an independent Markov chain. This structure is motivated by the presence of discrete metadata, such as an event type, that may partially reveal the hidden state but itself emanates from a …

New algorithm processes Riemannian data more efficiently.

We consider the restless Markov bandit problem, in which the state of each arm evolves according to a Markov process independently of the learner's actions. We suggest an algorithm that after $T$ steps achieves $\tilde{O}(\sqrt{T})$ regret with respect to the best policy that knows the distributions of all arms. No ass…

The Viterbi process can be extended indefinitely in a pairwise Markov model.

The generic identification problem is to decide whether a stochastic process $(X_t)$ is a hidden Markov process and if yes to infer its parameters for all but a subset of parametrizations that form a lower-dimensional subvariety in parameter space. Partial answers so far available depend on extra assumptions on the pro…

A new recommendation system model tackles unreliable user behavior.

Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long M…

Hybrid model improves traffic flow prediction accuracy.

A simple linear algebraic explanation of the algorithm in "A Spectral Algorithm for Learning Hidden Markov Models" (COLT 2009). Most of the content is in Figure 2; the text just makes everything precise in four nearly-trivial claims.

New algorithm optimizes complex model selection for non-homogeneous hidden Markov models.

New method infers hidden states in continuous-time phenomena better than traditional models.

Generalizes bits back coding for time-series models with latent Markov structures.

This is a technical report which explores the estimation methodologies on hyper-parameters in Markov Random Field and Gaussian Hidden Markov Random Field. In first section, we briefly investigate a theoretical framework on Metropolis-Hastings algorithm. Next, by using MH algorithm, we simulate the data from Ising model…

Improved online learning for hidden-convex losses achieves optimal regret.

This paper introduces a novel model-based clustering approach for clustering time series which present changes in regime. It consists of a mixture of polynomial regressions governed by hidden Markov chains. The underlying hidden process for each cluster activates successively several polynomial regimes during time. The…

Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of transition dynamics performing inference in the iHMM is difficult. In th…

As one of Bayesian analysis tools, Hidden Markov Model (HMM) has been used to in extensive applications. Most HMMs are solved by Baum-Welch algorithm (BWHMM) to predict the model parameters, which is difficult to find global optimal solutions. This paper proposes an optimized Hidden Markov Model with Particle Swarm Opt…

Continuous Hidden Markov Models for Equity Returns

The paper uses HMM and LSTM for stock market trend analysis.

A new model separates persistence and transition priors in HDP-HMM.

Hidden semi-Markov models (HSMMs) are latent variable models which allow latent state persistence and can be viewed as a generalization of the popular hidden Markov models (HMMs). In this paper, we introduce a novel spectral algorithm to perform inference in HSMMs. Unlike expectation maximization (EM), our approach cor…

Hidden Markov Model helps track asymptomatic carriers in pandemic.