arXiv research

New framework interprets deep nets via finite-state distributions.

Extends effect variable concept to finite states for web search evaluation.

In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…

A framework solves parametric families of MFGs efficiently.

Two neural network methods solve the master equation for MFGs.

The paper simplifies multi-agent RL dynamics in finite-state Markov games using homogenization.

A new layer learns abstract relations from graph structure using finite-state automata.

In this paper, we first establish the reflected backward stochastic difference equations with finite state (FS-RBSDEs for short). Then we explore the Existence and Uniqueness Theorem as well as the Comparison Theorem by "one step" method. The connections between FS-RBSDEs and optimal stopping time problems are investig…

We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…

The paper explores geometric calculations on probability manifolds derived from master equations.

The method approximates stationary distributions of Markov models by truncating irrelevant states.

New approach to understand recurrent policies as FSMs without minimization.

Framework for differentiating WFSTs for structured loss functions.

Study on natural actor-critic for POMDPs with finite memory.

We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as t…

Epsilon-machines are minimal, unifilar presentations of stationary stochastic processes. They were originally defined in the history machine sense, as hidden Markov models whose states are the equivalence classes of infinite pasts with the same probability distribution over futures. In analyzing synchronization, though…

Transformers simulate finite-state automata with fewer layers.

State-regularized RNNs improve interpretability and performance on long-term memory tasks.

Paper modifies automata learning to improve interpretability.

A novel method reduces dimensionality for filtering SRNs with observed variables.

A new method approximates Laplacian eigenvectors for RL efficiently.

Modeling and learning turn-taking behaviors in multi-agent systems.

Researchers found a regular language for maximal lexicographic representatives in braid monoids.

Reservoir computers and RNNs fall short of optimal prediction for stochastic PDFA.

We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …

A new method uses Mean Field Games to optimize mixture models of Bernoulli and categorical distributions.

The paper develops a Hoeffding inequality for Markov chains and applies it to bandit problems.

New model estimates unknown number of users in asynchronous multiuser communication.

A simplified model shows how wealth distribution can be derived from random exchanges.

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Montúfar and Ay,…

The potential approach is a general and simple method for modelling interest rates, foreign exchange rates, and in principle other types of financial assets. This paper takes data on some liquid interest rate derivatives, and fits potential models using a small finite-state Markov chain as the base Markov process.

Develops STC for sequential data with missing labels.

Study of multi-armed bandits with state-switching rewards using Markov models.

New method improves convergence of gradient descent for non-convex, non-reversible Markov chains.

Deep Galerkin Method estimates value function for mean-field control problem.

The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.

We consider the estimation of the policy gradient in partially observable Markov decision processes (POMDP) with a special class of structured policies that are finite-state controllers. We show that the gradient estimation can be done in the Actor-Critic framework, by making the critic compute a "value" function that …

Study shows challenges in converting RNNs to FSMs due to computational complexity.

Study provides convergence guarantees for discrete diffusion models on finite and infinite state spaces.

Recurrent neural networks trained on regular languages exhibit stable states that can recover from noise.

In this paper, a finite-state mean-reverting model for the short-rate, based on the continuous time Ehrenfest process, will be examined. Two explicit pricing formulae for zero-coupon bonds will be derived in the general and the special symmetric cases. Its limiting relationship to the Vasicek model will be examined wit…

This work optimizes MCMC algorithms for modern accelerators without synchronization overheads.

User-specific KWS system learns new keywords on-device.

Study optimal portfolio allocation in credit markets with default contagion.

In this work, we study stability of distributed filtering of Markov chains with finite state space, partially observed in conditionally Gaussian noise. We consider a nonlinear filtering scheme over a Distributed Network of Agents (DNA), which relies on the distributed evaluation of the likelihood part of the centralize…

Sparse regularization reduces NLP model complexity without sacrificing accuracy.

Algorithm solves word problem for 3-manifold groups.

We solve POMDPs by approximating them as finite-state MDPs.