arXiv research

This study bridges discrete and continuous state spaces using the Ehrenfest process and diffusion models.

New IRL algorithm for continuous state spaces with formal guarantees.

This primer explains diffusion models in general state spaces.

Novel framework proves fast RL convergence in continuous spaces.

Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a…

HiPPO-Prophecy models can learn dynamical systems without fine-tuning.

Paper solves POMDPs in continuous time and discrete spaces.

New model for insurance states using Markov jump processes with non-countable state space.

NCDSSM models irregularly sampled time series with improved imputation and forecasting.

CADD improves generative quality by augmenting discrete diffusion with continuous latent space.

The paper tackles finding optimal treatment sequences in continuous state spaces.

RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.

We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision Processes (BAMDPs) in continuous state and action spaces, to the best of our knowledge. The BAMDP framework elegantly addresses model uncertainty by incorporating Bayesian belief updates into long-term expected return. However, computing an e…

Paper develops unbiased gradient estimator for continuous-time models.

ZoomRL learns efficient strategies for large state-action spaces using a metric.

Algorithm learns diffusion processes with high-dimensional state spaces.

This paper introduces a new neural ODE model for continuous-time sequence generation.

Study of filtering and smoothing in submanifolds of Euclidean space.

We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those …

New method infers hidden states in continuous-time phenomena better than traditional models.

Study geometric flows with varying parameters and prove continuous dependence.

Method infers causal structure from system behaviors using RKHS and kernel $ε$-machines.

The study explores how Matrix Product States can represent boolean and continuous functions.

GP-PSRL achieves sublinear regret for continuous control with unbounded state space.

MO2 learns useful behaviours from past experience for new tasks.

We consider differentially private algorithms for reinforcement learning in continuous spaces, such that neighboring reward functions are indistinguishable. This protects the reward information from being exploited by methods such as inverse reinforcement learning. Existing studies that guarantee differential privacy a…

ACSSM models irregular time series with continuous dynamics.

New Q-learning methods improve sample complexity for MDPs with continuous states.

A classical result in Riemannian geometry states that the absolutely continuous curves into a (finite-dimensional) Riemannian manifold form an infinite-dimensional manifold. In the present paper this construction and related results are generalised to absolutely continuous curves with values in a strong Riemannian mani…

Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be…

A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints. It relies on a notion of risk implemented in the shape of a cost signal constrained to lie below an - adjustable - threshold. So far, BMDPs could only be solved in the case of fi…

LS4 models time-series with latent states, outperforming previous methods.

Theory broadens GFlowNets to handle continuous spaces.

New algorithm for aggregate inference in HMMs with continuous observations.

We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel $Q$-learning policy with adaptive data-driven discretization. The central idea is to maintain a finer partition of the state-action space in regions w…

Study robust control for systems with continuous states using adversarial perturbations.

In this paper we prove the probabilistic continuous complexity conjecture. In continuous complexity theory, this states that the complexity of solving a continuous problem with probability approaching 1 converges (in this limit) to the complexity of solving the same problem in its worst case. We prove the conjecture ho…

In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time…

We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous tim…

DFMs enable flow-based models for multimodal discrete and continuous data.

Continuous state spaces and stochastic, switching dynamics characterize a number of rich, realworld domains, such as robot navigation across varying terrain. We describe a reinforcementlearning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct wit…

Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown un…

S2P2 model improves predictive likelihoods for MTPPs.

We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models. It is efficient because (1) local models are estimated only when the planner requires them; (2) the planner focuses on the most relevant states to the current planni…

Paper explores using LLMs for zero-shot reinforcement learning in continuous spaces.

Study uses actor-critic method for continuous-time mean-field control with entropy regularisation.

We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning (RL) algorithms can be cast as SAs driven by a `controlled Markov' process. In t…

Entropy regularization is used to get improved optimization performance in reinforcement learning tasks. A common form of regularization is to maximize policy entropy to avoid premature convergence and lead to more stochastic policies for exploration through action space. However, this does not ensure exploration in th…