arXiv research

This paper analyzes stability and generalization of Markov chain stochastic gradient methods.

We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning (RL) algorithms can be cast as SAs driven by a `controlled Markov' process. In t…

Study identifies and validates a method for system identification of Markov jump linear systems.

The paper analyzes stability of random matrix products with Markovian noise.

We introduce a new braid-theoretic framework with which to understand the Legendrian and transversal classification of knots, namely a Legendrian Markov Theorem without Stabilization which induces an associated transversal Markov Theorem without Stabilization. We establish the existence of a nontrivial knot-type specif…

We give a new proof of Markov's classical theorem relating any two closed braid representations of the same knot or link. The proof is based upon ideas in a forthcoming paper by the authors, "Stabilization in the braid groups". The new proof of the classical Markov theorem is used by Nancy Wrinkle in her forthcoming ma…

Paper improves TD learning algorithm bounds with linear approx.

Algorithm identifies bilinear dynamical systems from noisy data.

TensorFlow Probability MCMC toolkit improves MCMC efficiency for modern hardware.

RILA learns HQMMs robustly against adversarial corruption.

We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighboring replicas. The use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics thus providing a practical approach to difficult anisotr…

Regularization improves generalization in Bayesian RL, shown through algorithmic stability.

Graph clustering uses multiscale community detection for improved performance.

Policy gradient algorithm with variable learning rates achieves near-optimal performance in multi-arm bandit problems.

Study analyzes price change patterns across different market capitalizations using Markov chains.

We describe random walk boundaries (in particular, the Poisson--Furstenberg, or PF-boundary) for a vast family of groups in terms of the hyperbolic boundary of a special free subgroup. We prove that almost all trajectories of the random walk (with respect to an arbitrary nondegenerate measure on the group) converge to …

Hidden Markov Models analyze mobile health data to identify APNS states.

Mathematical methods of population genetics and framework of exchangeability provide a Markov chain model for analysis and interpretation of stochastic behaviour of equity markets, explaining, in particular, market shape formation, statistical equilibrium and temporal stability of market weights.

Paper analyzes stability and forgetting in score-based generative models.

Developed algebraic theory of bonded braids, proving Markov theorem.

A new algorithm speeds up elliptical slice sampling for truncated multivariate normals.

We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate …

In this work, we study stability of distributed filtering of Markov chains with finite state space, partially observed in conditionally Gaussian noise. We consider a nonlinear filtering scheme over a Distributed Network of Agents (DNA), which relies on the distributed evaluation of the likelihood part of the centralize…

Study on financial systems using perturbed unimodal maps with heteroscedastic noise.

New algorithms help machines forget old data efficiently.

We consider the problem of learning a realization for a linear time-invariant (LTI) dynamical system from input/output data. Given a single input/output trajectory, we provide finite time analysis for learning the system's Markov parameters, from which a balanced realization is obtained using the classical Ho-Kalman al…

We build a sequence of empirical measures on the space D(R_+,R^d) of R^d-valued càdlàg functions on R_+ in order to approximate the law of a stationary R^d-valued Markov and Feller process (X_t). We obtain some general results of convergence of this sequence. Then, we apply them to Brownian diffusions and solutions to …

Style Miner generates stable and significant style factors for time series analysis.

Novel framework proves fast RL convergence in continuous spaces.

In order to obtain a Markov theorem without stabilization, Birman and Menasco introduced the notion of exchange related braids. In this paper I study the way the Fiedler polynomial distinguishes conjugacy classes of some particular braided knots. I introduce the Kauffman bracket in the solid torus. Its Taylor expansion…

This paper presents a technique for reduced-order Markov modeling for compact representation of time-series data. In this work, symbolic dynamics-based tools have been used to infer an approximate generative Markov model. The time-series data are first symbolized by partitioning the continuous measurement space of the …

New approach to concentration inequalities for unbounded state space dynamical systems.

Survey of RL methods for control systems with time delays.

This study analyzes convergence and stability of reinforcement learning algorithms.

A number of problems in statistical physics and computer science can be expressed as the computation of marginal probabilities over a Markov random field. Belief propagation, an iterative message-passing algorithm, computes exactly such marginals when the underlying graph is a tree. But it has gained its popularity as …

This paper extends stable blanket theory to models with hidden variables and causal cycles.

HOPE improves SSMs for long-memory tasks with robust initialization and training.

We define a homology $\mathcal{H}_N$ for closed braids by applying Khovanov and Rozansky's matrix factorization construction with potential $ax^{N+1}$. Up to a grading shift, $\mathcal{H}_0$ is the HOMFLYPT homology defined in arXiv:math/0505056. We demonstrate that, for $N \geq 1$, $\mathcal{H}_N$ is a $\mathbb{Z}_2\o…

The Markov Theorem Without Stabilization (MTWS) established the existence of a calculus of braid isotopies that can be used to move between closed braid representatives of a given oriented link type without having to increase the braid index by stabilization. Although the calculus is extensive there are three key isoto…

The Markov Theorem Without Stabilization (MTWS) (see math.GT/0310279) established the existence of a calculus of braid isotopies that can be used to move between closed braid representatives of a given oriented link type without having to increase the braid index by stabilization. Although the calculus is extensive the…

Coregulation of the expression of groups of genes has been extensively demonstrated empirically in bacterial and eukaryotic systems. Such coregulation can arise through the use of shared regulatory motifs, which allow the coordinated expression of modules (and module groups) of functionally related genes across the gen…

TD(0) with Polyak-Ruppert averaging achieves robust and fast convergence rates

This paper tackles adaptive control of unknown Markov jump systems with sample complexity and regret bounds.

Batch normalization prevents rank collapse in deep networks, improving training stability.

This paper improves MARL for networked systems through new protocols and discount factors.

PQ-learning improves Q-learning by periodically updating target estimates.

Simplified matrix generator resolves credit migration model calibration issues.

New algorithm learns linear dynamical systems from measurements.