A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Extends effect variable concept to finite states for web search evaluation.
The paper simplifies multi-agent RL dynamics in finite-state Markov games using homogenization.
A new layer learns abstract relations from graph structure using finite-state automata.
In this paper, we first establish the reflected backward stochastic difference equations with finite state (FS-RBSDEs for short). Then we explore the Existence and Uniqueness Theorem as well as the Comparison Theorem by "one step" method. The connections between FS-RBSDEs and optimal stopping time problems are investig…
In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show the converse to the above by showing that if the relative law of a Markov chain…
We extend a recent synchronization analysis of exact finite-state sources to nonexact sources for which synchronization occurs only asymptotically. Although the proof methods are quite different, the primary results remain the same. We find that an observer's average uncertainty in the source state vanishes exponential…
The paper explores geometric calculations on probability manifolds derived from master equations.
New approach to understand recurrent policies as FSMs without minimization.
Framework for differentiating WFSTs for structured loss functions.
Two neural network methods solve the master equation for MFGs.
A framework solves parametric families of MFGs efficiently.
Transformers simulate finite-state automata with fewer layers.
The smallest eigenvectors of the graph Laplacian are well-known to provide a succinct representation of the geometry of a weighted graph. In reinforcement learning (RL), where the weighted graph may be interpreted as the state transition process induced by a behavior policy acting on the environment, approximating the …
We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a variant of stochastic gradient descent where the random samples are taken on the t…
The paper develops a Hoeffding inequality for Markov chains and applies it to bandit problems.
The potential approach is a general and simple method for modelling interest rates, foreign exchange rates, and in principle other types of financial assets. This paper takes data on some liquid interest rate derivatives, and fits potential models using a small finite-state Markov chain as the base Markov process.
Develops STC for sequential data with missing labels.
The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.
We study an open problem of risk-sensitive portfolio allocation in a regime-switching credit market with default contagion. The state space of the Markovian regime-switching process is assumed to be a countably infinite set. To characterize the value function, we investigate the corresponding recursive infinite-dimensi…
We consider the estimation of the policy gradient in partially observable Markov decision processes (POMDP) with a special class of structured policies that are finite-state controllers. We show that the gradient estimation can be done in the Actor-Critic framework, by making the critic compute a "value" function that …
Recurrent neural networks trained on regular languages exhibit stable states that can recover from noise.
Study shows challenges in converting RNNs to FSMs due to computational complexity.
In this paper, a finite-state mean-reverting model for the short-rate, based on the continuous time Ehrenfest process, will be examined. Two explicit pricing formulae for zero-coupon bonds will be derived in the general and the special symmetric cases. Its limiting relationship to the Vasicek model will be examined wit…
Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. We aim to address both…
This work optimizes MCMC algorithms for modern accelerators without synchronization overheads.
We propose a new way of thinking about deep neural networks, in which the linear and non-linear components of the network are naturally derived and justified in terms of principles in probability theory. In particular, the models constructed in our framework assign probabilities to uncertain realizations, leading to Ku…
Reservoir computers (RCs) and recurrent neural networks (RNNs) can mimic any finite-state automaton in theory, and some workers demonstrated that this can hold in practice. We test the capability of generalized linear models, RCs, and Long Short-Term Memory (LSTM) RNN architectures to predict the stochastic processes g…
Study on natural actor-critic for POMDPs with finite memory.
We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as Markov cubature rules. The polynomial property allows us to study such rules using …
We solve POMDPs by approximating them as finite-state MDPs.
Study growth patterns in random networks using i.i.d. perturbations.
We investigate the problem of reliable communication between two legitimate parties over deletion channels under an active eavesdropping (aka jamming) adversarial model. To this goal, we develop a theoretical framework based on probabilistic finite-state automata to define novel encoding and decoding schemes that ensur…
We consider a self-exciting counting process, the parameters of which depend on a hidden finite-state Markov chain. We derive the optimal filter and smoother for the hidden chain based on observation of the jump process. This filter is in closed form and is finite dimensional. We demonstrate the performance of this fil…
Reciprocal processes are acausal generalizations of Markov processes introduced by Bernstein in 1932. In the literature, a significant amount of attention has been focused on developing dynamical models for reciprocal processes. In this paper, we provide a probabilistic graphical model for reciprocal processes. This le…
We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state of the underlying chain, the integrand must be of a specific form. This allows u…
New communication standards need to deal with machine-to-machine communications, in which users may start or stop transmitting at any time in an asynchronous manner. Thus, the number of users is an unknown and time-varying parameter that needs to be accurately estimated in order to properly recover the symbols transmit…
In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive equations for the higher order moments of the discount factor and we describe a Monte Ca…
Multi-dimensional state-integrals of products of Faddeev's quantum dilogarithms arise frequently in Quantum Topology, quantum Teichmüller theory and complex Chern--Simons theory. Using the quasi-periodicity property of the quantum dilogarithm, we evaluate 1-dimensional state-integrals at rational points and express the…
Epsilon-machines are minimal, unifilar presentations of stationary stochastic processes. They were originally defined in the history machine sense, as hidden Markov models whose states are the equivalence classes of infinite pasts with the same probability distribution over futures. In analyzing synchronization, though…
We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of the nature of its shape: a general polytope (Aigner et al., 2010). To demonstrate this result, we exhibit several properties of the structu…
We obtain an index of the complexity of a random sequence by allowing the role of the measure in classical probability theory to be played by a function we call the generating mechanism. Typically, this generating mechanism will be a finite automata. We generate a set of biased sequences by applying a finite state auto…
We axiomatically introduce risk-consistent conditional systemic risk measures defined on multidimensional risks. This class consists of those conditional systemic risk measures which can be decomposed into a state-wise conditional aggregation and a univariate conditional risk measure. Our studies extend known results f…
The language of maximal lexicographic representatives of elements in the positive braid monoid with generators is a regular language. We describe with great detail the smallest Finite State Automaton accepting such language, and study the proportion of elements of length whose maximal lexicographic repres…
We present an interactive version of an evidence-driven state-merging (EDSM) algorithm for learning variants of finite state automata. Learning these automata often amounts to recovering or reverse engineering the model generating the data despite noisy, incomplete, or imperfectly sampled data sources rather than optim…
Automaton models are often seen as interpretable models. Interpretability itself is not well defined: it remains unclear what interpretability means without first explicitly specifying objectives or desired attributes. In this paper, we identify the key properties used to interpret automata and propose a modification o…
A novel method reduces dimensionality for filtering SRNs with observed variables.
Study on indexability of restless multi-armed bandits and rollout policy performance.