On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “continuous state space”

Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion modelsstat.ML

This study bridges discrete and continuous state spaces using the Ehrenfest process and diffusion models.

problem Understanding the relationship between discrete and continuous state spaces in stochastic processes.

method Investigates time-continuous Markov jump processes on discrete state spaces and their correspondence to state-continuous diffusion processes.

result The time-reversal of the Ehrenfest process converges to the time-reversed Ornstein-Uhlenbeck process, bridging discrete and continuous state spaces.

Ehrenfest process Ornstein-Uhlenbeck process Markov jump processes Diffusion models Time-reversal

2024-05-06abs ↗pdf ↗

Budgeted Reinforcement Learning in Continuous State Spacecs.LG

Extends reinforcement learning to continuous state spaces with safety constraints.

problem Safety-critical reinforcement learning in continuous state spaces with unknown dynamics.

method Introduces a novel Budgeted Bellman Optimality operator and applies it to continuous state spaces.

result Validated on spoken dialogue and autonomous driving applications.

Reinforcement Learning Safety Constraints Continuous State Space Budgeted Markov Decision Process Deep Reinforcement Learning

2019-03-03abs ↗pdf ↗

Inverse Reinforcement Learning in a Continuous State Space with Formal Guaranteescs.LG

New IRL algorithm for continuous state spaces with formal guarantees.

problem Finding a reward function for expert behavior in continuous state spaces.

method Modeling the system using orthonormal functions and providing correctness proofs.

result Proof of correctness and formal guarantees on sample and time complexity.

Inverse Reinforcement Learning Continuous State Space Formal Guarantees Orthonormal Functions Correctness Proofs

2021-02-16abs ↗pdf ↗

Foundations of Diffusion Models in General State Spaces: A Self-Contained Introductionstat.ML

This primer explains diffusion models in general state spaces.

problem Diffusion models in general state spaces are not well-introduced.

method Develops discrete-time and continuous-time views of diffusion models, deriving Fokker-Planck and master equations.

result Unified understanding of diffusion models across continuous and discrete domains.

Diffusion models General state spaces Continuous domains Discrete sequences Fokker-Planck equations

2025-12-04abs ↗pdf ↗

Privacy-preserving Q-Learning with Functional Noise in Continuous State Spacesstat.ML

We protect value function approximators in continuous state spaces from differential privacy attacks.

problem Protecting reward information from inverse reinforcement learning attacks in continuous state spaces.

method Adding functional noise to the value function iteratively to ensure differential privacy.

result Rigorous privacy guarantees and approximate optimality of the algorithm in discrete state spaces.

differential privacy reinforcement learning continuous state space inverse reinforcement learning functional noise

2019-01-30abs ↗pdf ↗

Taming "data-hungry" reinforcement learning? Stability in continuous state-action spacescs.LG

Novel framework proves fast RL convergence in continuous spaces.

problem Analyzing stability in continuous state-action RL.

method Introduces a novel framework to analyze stability properties of RL.

result Highlights two key stability properties and demonstrates their satisfaction in RL.

reinforcement learning continuous state-action stability fast convergence Bellman operator

2024-01-10abs ↗pdf ↗

Geometric fluid approximation for general continuous-time Markov chainseess.SY

New method for fluid approximation of CTMCs without population structure.

problem Approximating the macro-scale behavior of large CTMCs.

method Spectral analysis of CTMC transition matrix, diffusion maps, Gaussian process regression.

result Construct an ODE approximating CTMC mean in continuous space.

fluid approximation continuous-time Markov chain spectral analysis manifold learning Gaussian process regression

2019-01-31abs ↗pdf ↗

HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Contextcs.LG

HiPPO-Prophecy models can learn dynamical systems without fine-tuning.

problem Learning dynamical systems in context without fine-tuning parameters.

method Introduced a novel weight construction for SSMs that approximates derivatives of input signals.

result Discrete SSMs can predict the next state of any dynamical system after observing previous states.

State Space Models Dynamical Systems Continuous SSMs Discrete SSMs Derivative Approximation

2024-07-12abs ↗pdf ↗

DeepMDP: Learning Continuous Latent Space Models for Representation Learningcs.LG

DeepMDP simplifies complex observations into continuous latent states.

problem Learning from high-dimensional observations in reinforcement learning.

method Trains a DeepMDP model that predicts rewards and next latent states.

result Optimization of DeepMDP objectives ensures quality of latent space and environment model.

reinforcement learning latent space continuous states rewards prediction environment modeling

2019-06-06abs ↗pdf ↗

Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processescs.LG

Bayes-CPACE optimally explores continuous BAMDPs.

problem Model uncertainty in continuous state and action spaces.

method Covering state-belief-action space with samples, exploiting Lipschitz continuity.

result Near-optimal value function computed efficiently.

Bayesian Markov Decision Processes Continuous Space PAC Optimal Value Function

2018-10-06abs ↗pdf ↗

POMDPs in Continuous Time and Discrete Spacescs.LG

Paper solves POMDPs in continuous time and discrete spaces.

problem Optimal decision making in discrete state and action space systems under partial observability.

method Combining optimal filtering theory and deep learning to solve a Hamilton-Jacobi-Bellman equation.

result Derives a mathematical description and solution approach for continuous-time POMDPs.

POMDPs Continuous time Discrete space Optimal decision making Partial observability

2020-10-02abs ↗pdf ↗

Thiele's Differential Equation Based on Markov Jump Processes with Non-countable State Spaceq-fin.RM

New model for insurance states using Markov jump processes with non-countable state space.

problem Modeling insurance states with non-countable state spaces.

method Developed a new Thiele's differential equation for continuous time rehabilitation rates.

result Allows for consistent calculation of reserves in disability insurance.

insurance Markov processes Thiele's equation disability insurance rehabilitation rates

2021-02-19abs ↗pdf ↗

On Applications of Bootstrap in Continuous Space Reinforcement Learningcs.LG

Bootstrap policies improve regret in continuous state-action reinforcement learning.

problem Improving regret in reinforcement learning for continuous state and action spaces.

method Bootstrap-based policies for stochastic linear systems with quadratic cost functions.

result Bootstrap policies achieve a square root scaling of regret with respect to time.

reinforcement learning continuous state-action bootstrap regret stochastic linear systems

2019-03-14abs ↗pdf ↗

Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Seriescs.LG

NCDSSM models irregularly sampled time series with improved imputation and forecasting.

problem Accurate modeling of irregularly sampled time series with missing observations.

method Neural Continuous-Discrete State Space Model (NCDSSM) with amortized inference for auxiliary variables and flexible dynamic state parameterizations.

result Improved imputation and forecasting performance on multiple benchmark datasets.

time series irregular sampling missing data state space models neural networks

2023-01-26abs ↗pdf ↗

Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingstat.ML

CADD improves generative quality by augmenting discrete diffusion with continuous latent space.

problem Loss of semantic information between denoising steps in discrete diffusion models.

method Introduces a framework that augments discrete state space with a continuous latent space, allowing for graded, informative masked tokens.

result CADD improves generative quality across text generation, image synthesis, and code modeling.

discrete diffusion semantic information continuous latent space mode-coverage mode-seeking

2025-10-01abs ↗pdf ↗

Learning State Abstractions for Transfer in Continuous Controlcs.LG

Tabular Q-Learning with learned state abstractions solves continuous control tasks.

problem Challenging reinforcement learning problems in continuous control.

method Learned state abstraction to transform continuous state-space into discrete.

result Tabular Q-Learning with learned abstractions achieves efficient learning in unseen tasks.

reinforcement learning continuous control state abstraction tabular Q-Learning

2020-02-08abs ↗pdf ↗

Efficient Model-Free Reinforcement Learning Using Gaussian Processcs.LG

GPPSTD uses Gaussian Processes for efficient RL in continuous states.

problem Efficient reinforcement learning in continuous state spaces.

method Gaussian Process Posterior Sampling Reinforcement Learning (GPPSTD) algorithm.

result Combining demonstration and exploration improves reinforcement learning efficiency.

Gaussian Process Reinforcement Learning Posterior Sampling Efficiency Continuous State Space

2018-12-11abs ↗pdf ↗

Finding Counterfactually Optimal Action Sequences in Continuous State Spacescs.LG

The paper tackles finding optimal treatment sequences in continuous state spaces.

problem Finding counterfactually optimal action sequences in continuous state spaces.

method Formalizes the problem using finite horizon Markov decision processes and structural causal models. Develops a search method based on the A* algorithm.

result The method can find optimal action sequences in polynomial time under certain conditions.

reinforcement learning causal inference Markov decision processes continuous state spaces sequential decision making

2023-06-06abs ↗pdf ↗

Randomized Policy Learning for Continuous State and Action MDPscs.LG

RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.

problem Efficient reinforcement learning in environments with continuous state and action spaces.

method RANDPOL uses randomized function approximation to represent policy and value functions, providing finite time guarantees and improved numerical performance.

result RANDPOL achieves better numerical performance and provides finite time guarantees compared to deep neural network based algorithms.

reinforcement learning randomized networks continuous state action spaces policy iteration

2020-06-08abs ↗pdf ↗

Unbiased Estimation of the Gradient of the Log-Likelihood for a Class of Continuous-Time State-Space Modelsstat.ML

Paper develops unbiased gradient estimator for continuous-time models.

problem Estimating unbiased gradient of log-likelihood for continuous-time models.

method Doubly randomized scheme with coupled conditional particle filter (CCPF).

result Unbiased gradient estimate facilitates gradient-based algorithms.

gradient estimation log-likelihood state-space models continuous-time particle filter

2021-05-24abs ↗pdf ↗

Zooming for Efficient Model-Free Reinforcement Learning in Metric Spacescs.LG

ZoomRL learns efficient strategies for large state-action spaces using a metric.

problem Handling large state-action spaces in reinforcement learning.

method ZoomRL leverages continuous bandits to adaptively discretize the joint space.

result Achieves worst-case regret of $ ilde{O}(H^{rac{5}{2}} K^{rac{d+1}{d+2}})$.

Reinforcement Learning Metric Spaces Continuous Bandits Adaptive Discretization Regret Analysis

2020-03-09abs ↗pdf ↗

Adaptive Partitioning and Learning for Stochastic Control of Diffusion Processescs.LG

Algorithm learns diffusion processes with high-dimensional state spaces.

problem Stochastic control of unbounded diffusion processes with high-dimensional state spaces.

method Adaptive partitioning and learning algorithm that refines discretization based on estimation bias and statistical confidence.

result Established regret bounds that depend on problem parameters, extending to unbounded diffusion processes.

Reinforcement Learning Stochastic Control Diffusion Processes High-Dimensional State Spaces Adaptive Partitioning

2025-12-17abs ↗pdf ↗

Latent Space Energy-based Neural ODEscs.LG

This paper introduces a new neural ODE model for continuous-time sequence generation.

problem Representing and predicting continuous-time sequences with high accuracy.

method A neural emission model and neural ODE define the latent state evolution, with an Energy-based model for prior distribution.

result The model outperforms existing methods in various tasks, including long-horizon predictions.

neural ODE energy-based model latent space continuous-time sequence dynamical models

2024-09-05abs ↗pdf ↗

Continuous-Discrete Filtering and Smoothing on Submanifolds of Euclidean Spacemath.OC

Study of filtering and smoothing in submanifolds of Euclidean space.

problem Filtering and smoothing in continuous-discrete time on submanifolds.

method Formal expressions and projection approach for prediction and smoothing.

result Agreement with classical results for prediction, differences for smoothing.

submanifolds Euclidean space continuous-discrete filtering smoothing

2020-04-17abs ↗pdf ↗

Explicit Explore-Exploit Algorithms in Continuous State Spacescs.LG

New RL algorithm for large state spaces with explicit exploration and exploitation phases.

problem Reinforcement learning in large or infinite state spaces.

method Model-based approach with explicit exploration and exploitation phases.

result Proves near-optimal policy with polynomial sample complexity under certain assumptions.

reinforcement learning exploration-exploitation continuous state spaces neural networks sample efficiency

2019-11-01abs ↗pdf ↗

Approximate Kalman Filter Q-Learning for Continuous State-Space MDPscs.LG

We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those …

2013-09-26abs ↗pdf ↗

Forward-Backward Latent State Inference for Hidden Continuous-Time semi-Markov Chainsstat.ML

New method infers hidden states in continuous-time phenomena better than traditional models.

problem Traditional HSMM's are limited to discrete time grids and cannot handle irregularly spaced data.

method Formulated integro-differential forward and backward equations for CTSMC's, introduced scalable Viterbi-type algorithm.

result Efficiently solved equations for posterior marginals and path estimates.

Hidden semi-Markov Models Continuous-Time semi-Markov Chains Latent State Inference Integro-differential Equations Bayesian Posterior Marginals

2022-10-17abs ↗pdf ↗

Discovering Causal Structure with Reproducing-Kernel Hilbert Space

ε

-Machinescs.LG

Method infers causal structure from system behaviors using RKHS and kernel $ε$ -machines.

problem Discovering causal structure in systems with varying external and measurement noise.

method Combines causal states and RKHS for efficient representation and inference of causal structure.

result Robustly estimates causal structure in high-dimensional data with varying noise.

causal inference reproducing-kernel Hilbert space kernel machines system dynamics hidden Markov models

2020-11-23abs ↗pdf ↗

Topologies for geometric flows and continuous dependence on parametersmath.GT

Study geometric flows with varying parameters and prove continuous dependence.

problem Continuous dependence of flows on parameters in geometric settings.

method Derived suitable topologies for vector fields and flows, proved new continuous dependence.

result Proved continuous dependence of flows on parameters in a general topological space.

geometric flows continuous dependence vector fields flows parameters

2023-10-18abs ↗pdf ↗

Representation Theorem for Matrix Product Statesstat.ML

The study explores how Matrix Product States can represent boolean and continuous functions.

problem Representing arbitrary boolean and continuous functions using Matrix Product States.

method Developed a construction method for MPS to represent boolean gates and proved density in continuous function space.

result MPS can accurately represent arbitrary boolean functions and continuous functions densely.

Matrix Product States Boolean Functions Continuous Functions Neural Networks Gaussian Processes

2021-03-15abs ↗pdf ↗

Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spacesstat.ML

GP-PSRL achieves sublinear regret for continuous control with unbounded state space.

problem Analyzing regret bounds for GP-PSRL in continuous control with unbounded state space.

method Recursive application of Borell-Tsirelson-Ibragimov-Sudakov inequality and chaining method.

result Sublinear regret bound of

\widetilde{\mathcal{O}}(H\sqrt{γ_TT})

for GP-PSRL.

reinforcement learning Gaussian processes posterior sampling continuous control sublinear regret

2026-03-09abs ↗pdf ↗

Efficient Model-free Reinforcement Learning in Metric Spacescs.LG

Efficient Q-learning for metric spaces without planning oracles.

problem Efficient reinforcement learning in continuous state-action spaces.

method Model-free Q-learning adapted for metric spaces without a planning oracle.

result Sample efficiency in continuous state-action spaces.

Reinforcement Learning Q-learning Metric Spaces Model-Free Efficient Algorithms

2019-05-01abs ↗pdf ↗

MO2: Model-Based Offline Optionscs.LG

MO2 learns useful behaviours from past experience for new tasks.

problem Discovering useful behaviours from past experience and transferring them to new tasks.

method Model-Based Offline Options (MO2) framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces.

result MO2 outperforms recent option learning methods on complex long-horizon continuous control tasks.

Model-Based Offline Options bottleneck state discovery sample-efficiency continuous state-action spaces bottleneck options

2022-09-05abs ↗pdf ↗

Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Seriesstat.ML

ACSSM models irregular time series with continuous dynamics.

problem Modeling irregular time series data.

method ACSSM uses a multi-marginal Doob's h-transform and variational inference with stochastic optimal control.

result ACSSM outperforms in tasks like classification, regression, interpolation, and extrapolation.

time series irregular data continuous dynamics variational inference stochastic optimal control

2024-10-08abs ↗pdf ↗

Minimax Optimal Q Learning with Nearest Neighborscs.LG

New Q-learning methods improve sample complexity for MDPs with continuous states.

problem Challenges in MDPs with continuous state spaces.

method Two nearest neighbor Q-learning methods with improved sample complexities.

result Minimax optimal sample complexities for offline and online settings.

Q learning Markov decision process Nearest neighbor Sample complexity Minimax optimal

2023-08-03abs ↗pdf ↗

Sparse tree search optimality guarantees in POMDPs with continuous observation spacescs.LG

POWSS simplifies Q-value estimation in POMDPs with continuous observations.

problem Lack of theoretical justification for online sampling-based algorithms in POMDPs with continuous observation spaces.

method Developed POWSS, a simplified algorithm that estimates Q-values accurately with high probability and can approach optimality with increased computational power.

result POWSS provides formal theoretical guarantees for Q-value estimation in POMDPs with continuous observations.

POMDPs Continuous observation spaces Observation likelihood weighting Sparse sampling Q-value estimation

2019-10-10abs ↗pdf ↗

Manifolds of absolutely continuous curves and the square root velocity frameworkmath.DG

A classical result in Riemannian geometry states that the absolutely continuous curves into a (finite-dimensional) Riemannian manifold form an infinite-dimensional manifold. In the present paper this construction and related results are generalised to absolutely continuous curves with values in a strong Riemannian mani…

2016-12-08abs ↗pdf ↗

Adaptive Discretization for Episodic Reinforcement Learning in Metric Spacescs.LG

Efficient algorithm for reinforcement learning in large state-action spaces with adaptive discretization.

problem Efficient reinforcement learning in large, potentially continuous state-action spaces.

method Adaptive

Q

-learning policy with data-driven adaptive discretization.

result Demonstrates improved performance compared to existing methods, especially in adapting to the problem's structure.

reinforcement learning adaptive discretization metric spaces episodic learning Q-learning

2019-10-17abs ↗pdf ↗

Deep Latent State Space Models for Time-Series Generationstat.ML

LS4 models time-series with latent states, outperforming previous methods.

problem Learning sharp transitions in time-series data.

method State space ODE with convolutional representation to bypass hidden states.

result LS4 significantly outperforms previous models in various metrics.

time-series latent space state space convolutional sharp transitions

2022-12-24abs ↗pdf ↗

A theory of continuous generative flow networkscs.LG

Theory broadens GFlowNets to handle continuous spaces.

problem Limitation of GFlowNets to discrete spaces.

method Developed a theory for generalized GFlowNets.

result Empirical results show strong performance in continuous cases.

GFlowNets amortized variational inference continuous spaces discrete spaces probabilistic inference

2023-01-30abs ↗pdf ↗

Filtering for Aggregate Hidden Markov Models with Continuous Observationsstat.ML

New algorithm for aggregate inference in HMMs with continuous observations.

problem Inference in large populations with indistinguishable individuals and continuous measurements.

method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.

result Efficacy demonstrated through numerical experiments.

Hidden Markov Models Continuous Observations Aggregate Inference Collective Forward-Backward Algorithm

2020-11-04abs ↗pdf ↗

Learning Optimal Distributionally Robust Stochastic Control in Continuous State Spacesstat.ML

Study robust control for systems with continuous states using adversarial perturbations.

problem Fragile policies in Markov control models under internal or external perturbations.

method Distributionally robust stochastic control with adaptive adversarial perturbations.

result Optimal robust policies for continuous state systems with uniform learning guarantees.

robust control distributionally robust stochastic processes continuous state adversarial perturbations

2024-06-17abs ↗pdf ↗

On the probabilistic continuous complexity conjecturestat.ML

In this paper we prove the probabilistic continuous complexity conjecture. In continuous complexity theory, this states that the complexity of solving a continuous problem with probability approaching 1 converges (in this limit) to the complexity of solving the same problem in its worst case. We prove the conjecture ho…

2012-12-06abs ↗pdf ↗

Irregular-Time Bayesian Networkscs.AI

In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time…

2012-03-15abs ↗pdf ↗

Unbiased Bayesian Inference for Population Markov Jump Processes via Random Truncationsstat.ML

We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous tim…

2015-09-28abs ↗pdf ↗

Distributionally Robust Reinforcement Learningstat.ML

Enhances reinforcement learning safety through risk-averse exploration.

problem Safety concerns in reinforcement learning due to sub-optimal actions.

method Distributionally robust policy iteration scheme with lower bound guarantees.

result Efficient algorithm that prevents poor decisions and converges to optimal policy.

reinforcement learning risk-averse exploration state-values policy iteration

2019-02-23abs ↗pdf ↗

Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designstat.ML

DFMs enable flow-based models for multimodal discrete and continuous data.

problem Combining discrete and continuous data for generative models.

method Discrete Flow Models (DFMs) using Continuous Time Markov Chains.

result DFMs achieve state-of-the-art co-design performance for protein structure and sequence generation.

Generative Models Discrete Data Continuous Data Multimodal Protein Co-Design

2024-02-07abs ↗pdf ↗

CORL: A Continuous-state Offset-dynamics Reinforcement Learnercs.LG

Continuous state spaces and stochastic, switching dynamics characterize a number of rich, realworld domains, such as robot navigation across varying terrain. We describe a reinforcementlearning algorithm for learning in these domains and prove for certain environments the algorithm is probably approximately correct wit…

2012-06-13abs ↗pdf ↗