arXiv research

New algorithm reduces reinforcement learning regret for linear MDPs with unknown transitions.

This research improves dynamical systems understanding by identifying latent states and their nonlinear transitions.

This paper studies the dynamic generator model for spatial-temporal processes such as dynamic textures and action sequences in video data. In this model, each time frame of the video sequence is generated by a generator model, which is a non-linear transformation of a latent state vector, where the non-linear transform…

A new algorithm for deep Q-learning with robustness to state transition uncertainty.

New algorithms achieve logarithmic regret in learning linear quadratic control systems.

We give a polynomial-time algorithm for learning latent-state linear dynamical systems without system identification, and without assumptions on the spectral radius of the system's transition matrix. The algorithm extends the recently introduced technique of spectral filtering, previously applied only to systems with a…

Linear recurrent networks explain reinforcement learning performance in partially observable settings.

Proposes a new model for time series that considers smooth transitions between states.

Linear Dynamical System (LDS) is an elegant mathematical framework for modeling and learning multivariate time series. However, in general, it is difficult to set the dimension of its hidden state space. A small number of hidden states may not be able to model the complexities of a time series, while a large number of …

Study shows optimal RL with transition look-ahead is NP-hard for $\ell \geq 2$.

Neural models learn continuous-time Markov chain transition rates from data.

The paper tackles reward-relevance in offline RL with sparse decision dynamics.

Study learns linear system dynamics from noisy bilinear data.

Efficient RL for linear MDPs with unknown transitions.

Proposes a new model to analyze mortgage delinquency transitions.

Neural Physicist learns physical dynamics from images.

Model shows phase transitions in asset pricing with market maker incentives.

State aggregation is a popular model reduction method rooted in optimal control. It reduces the complexity of engineering systems by mapping the system's states into a small number of meta-states. The choice of aggregation map often depends on the data analysts' knowledge and is largely ad hoc. In this paper, we propos…

Neural networks with DAGs show linearity as width increases.

Efficient RL algorithm for multinomial logistic MDPs with provable guarantees.

This paper develops a low-nonnegative-rank approximation method to identify the state aggregation structure of a finite-state Markov chain under an assumption that the state space can be mapped into a handful of meta-states. The number of meta-states is characterized by the nonnegative rank of the Markov transition mat…

A new method learns robust policies from offline data with latent structures.

A nonparametric Bayesian sparse graph linear dynamical system (SGLDS) is proposed to model sequentially observed multivariate data. SGLDS uses the Bernoulli-Poisson link together with a gamma process to generate an infinite dimensional sparse random graph to model state transitions. Depending on the sparsity pattern of…

A new method learns state and proposal dynamics in state-space models using neural networks.

We study online reinforcement learning for finite-horizon deterministic control systems with {\it arbitrary} state and action spaces. Suppose that the transition dynamics and reward function is unknown, but the state and action space is endowed with a metric that characterizes the proximity between different states and…

New filters for non-linear systems achieve closed-form solutions.

An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…

Probabilistic programming languages can simplify the development of machine learning techniques, but only if inference is sufficiently scalable. Unfortunately, Bayesian parameter estimation for highly coupled models such as regressions and state-space models still scales poorly; each MCMC transition takes linear time i…

Proves error bounds for state representation in RL using graph spectral features.

Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of transition dynamics performing inference in the iHMM is difficult. In th…

Modeling dynamical systems is important in many disciplines, e.g., control, robotics, or neurotechnology. Commonly the state of these systems is not directly observed, but only available through noisy and potentially high-dimensional observations. In these cases, system identification, i.e., finding the measurement map…

Study non-asymptotic bounds on correlation in high-dimensional linear systems, revealing invariant subspaces and bottlenecks.

Trajectory data suffices for efficient RL in linear MDPs.

A Longitudinal Attribute-Conditioned Neural Network (LANTERN) framework for modeling health-state transition probabilities in irregular longitudinal data.

New method links covariates to CTMCs using RKHS, improving state transitions modeling.

Quantum theory reinterprets financial pricing by focusing on observable price transitions.

The paper proposes a survival model to optimize mobile notification delivery times.

New model of vague knowledge without strict partitions or transitivity.

The paper investigates non-linear and heavy-tailed predictability in transition-energy financial markets.

Study identifies transitions between traffic modes on Cologne motorways.

Adaptive populations such as those in financial markets and distributed control can be modeled by the Minority Game. We consider how their dynamics depends on the agents' initial preferences of strategies, when the agents use linear or quadratic payoff functions to evaluate their strategies. We find that the fluctuatio…

New algorithms solve robust MDPs efficiently, significantly faster than existing methods.

We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions. We develop variants of the UCRL and posterior sampling algorithms that employ nonparametric Gaussian process priors to generalize across the state and action spaces. When the trans…

A framework combining HSMM and survival analysis for lifecycle-oriented mobility analysis.

Develops methods to simulate rare transitions in molecular systems.

New STH distance finds patterns in event timeseries without resampling.

Algorithm minimizes regret in dueling bandits with contextualized utilities.

A new method for ILO with transition model disparity using an intermediary policy.