Tabular Q-Learning with learned state abstractions solves continuous control tasks.
problem Challenging reinforcement learning problems in continuous control.
method Learned state abstraction to transform continuous state-space into discrete.
result Tabular Q-Learning with learned abstractions achieves efficient learning in unseen tasks.
This study bridges discrete and continuous state spaces using the Ehrenfest process and diffusion models.
problem Understanding the relationship between discrete and continuous state spaces in stochastic processes.
method Investigates time-continuous Markov jump processes on discrete state spaces and their correspondence to state-continuous diffusion processes.
result The time-reversal of the Ehrenfest process converges to the time-reversed Ornstein-Uhlenbeck process, bridging discrete and continuous state spaces.
Continual state learning model using generative replay for RL.
problem Efficiently learn and adapt state representations as the environment changes.
method Variational Auto-Encoders for state representation and Generative Replay for past knowledge.
result Automatic environment change detection and efficient state representation.
S-TRIGGER learns state representations for continual learning.
problem Efficiently compress and maintain past knowledge in changing environments.
method Generative Replay with self-triggered environment change detection.
result S-TRIGGER enables fast and high-performing Reinforcement Learning without catastrophic forgetting.
Novel framework proves fast RL convergence in continuous spaces.
problem Analyzing stability in continuous state-action RL.
method Introduces a novel framework to analyze stability properties of RL.
result Highlights two key stability properties and demonstrates their satisfaction in RL.
Paper finds unique viscosity solution to complex control problems.
problem Complex stochastic control problems with singular terminal state constraints.
method Establishes existence of unique nonnegative continuous viscosity solution using novel comparison principle.
result Unique viscosity solution to HJB equation for linear-quadratic control problems.
Complex contagion model explains financial fire sales through continuous asset prices.
problem Modeling financial fire sales with a continuum of asset prices.
method Developed a threshold model of continuous-state cascades using real values for asset prices.
result Discretization approach accurately replicates the distribution of defaulted banks and asset prices.
New ODE-Block handles stateful layers with continuous-in-depth functions using basis functions.
problem Handling stateful layers in ODE-Nets.
method Formulate ODE-Block using continuous-in-depth functions with basis function expansions.
result Enables state-of-the-art performance and reduces memory footprint.
New method infers hidden states in continuous-time phenomena better than traditional models.
problem Traditional HSMM's are limited to discrete time grids and cannot handle irregularly spaced data.
method Formulated integro-differential forward and backward equations for CTSMC's, introduced scalable Viterbi-type algorithm.
result Efficiently solved equations for posterior marginals and path estimates.
Extends reinforcement learning to continuous state spaces with safety constraints.
problem Safety-critical reinforcement learning in continuous state spaces with unknown dynamics.
method Introduces a novel Budgeted Bellman Optimality operator and applies it to continuous state spaces.
result Validated on spoken dialogue and autonomous driving applications.
New IRL algorithm for continuous state spaces with formal guarantees.
problem Finding a reward function for expert behavior in continuous state spaces.
method Modeling the system using orthonormal functions and providing correctness proofs.
result Proof of correctness and formal guarantees on sample and time complexity.
This primer explains diffusion models in general state spaces.
problem Diffusion models in general state spaces are not well-introduced.
method Develops discrete-time and continuous-time views of diffusion models, deriving Fokker-Planck and master equations.
result Unified understanding of diffusion models across continuous and discrete domains.
New method for fluid approximation of CTMCs without population structure.
problem Approximating the macro-scale behavior of large CTMCs.
method Spectral analysis of CTMC transition matrix, diffusion maps, Gaussian process regression.
result Construct an ODE approximating CTMC mean in continuous space.
HiPPO-Prophecy models can learn dynamical systems without fine-tuning.
problem Learning dynamical systems in context without fine-tuning parameters.
method Introduced a novel weight construction for SSMs that approximates derivatives of input signals.
result Discrete SSMs can predict the next state of any dynamical system after observing previous states.
Bootstrap policies improve regret in continuous state-action reinforcement learning.
problem Improving regret in reinforcement learning for continuous state and action spaces.
method Bootstrap-based policies for stochastic linear systems with quadratic cost functions.
result Bootstrap policies achieve a square root scaling of regret with respect to time.
The paper tackles finding optimal treatment sequences in continuous state spaces.
problem Finding counterfactually optimal action sequences in continuous state spaces.
method Formalizes the problem using finite horizon Markov decision processes and structural causal models. Develops a search method based on the A* algorithm.
result The method can find optimal action sequences in polynomial time under certain conditions.
RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.
problem Efficient reinforcement learning in environments with continuous state and action spaces.
method RANDPOL uses randomized function approximation to represent policy and value functions, providing finite time guarantees and improved numerical performance.
result RANDPOL achieves better numerical performance and provides finite time guarantees compared to deep neural network based algorithms.
Introduces a Boltzmann machine with Riemann-Theta functions for continuous and discrete states.
problem Modeling continuous and discrete states in neural networks.
method Develops a Boltzmann machine with continuous visible and discrete hidden states, solving probability density and conditional expectation analytically.
result Derives a novel parametric density function involving Riemann-Theta functions and uses it as an activation function in a feedforward neural network.
We protect value function approximators in continuous state spaces from differential privacy attacks.
problem Protecting reward information from inverse reinforcement learning attacks in continuous state spaces.
method Adding functional noise to the value function iteratively to ensure differential privacy.
result Rigorous privacy guarantees and approximate optimality of the algorithm in discrete state spaces.
NCDSSM models irregularly sampled time series with improved imputation and forecasting.
problem Accurate modeling of irregularly sampled time series with missing observations.
method Neural Continuous-Discrete State Space Model (NCDSSM) with amortized inference for auxiliary variables and flexible dynamic state parameterizations.
result Improved imputation and forecasting performance on multiple benchmark datasets.
Echo state networks with random weights can approximate any continuous system.
problem Approximating continuous dynamical systems using echo state networks.
method Randomly generated internal weights and a sampling procedure for activation functions.
result Echo state networks with random weights can approximate any continuous casual time-invariant operators with high probability.
We first analyze the integrated density of states (IDS) of periodic Schrödinger operators on an amenable covering manifold. A criterion for the continuity of the IDS at a prescribed energy is given along with examples of operators with both continuous and discontinuous IDS'. Subsequently, alloy-type perturbations of th…
GPPSTD uses Gaussian Processes for efficient RL in continuous states.
problem Efficient reinforcement learning in continuous state spaces.
method Gaussian Process Posterior Sampling Reinforcement Learning (GPPSTD) algorithm.
result Combining demonstration and exploration improves reinforcement learning efficiency.
Bayes-CPACE optimally explores continuous BAMDPs.
problem Model uncertainty in continuous state and action spaces.
method Covering state-belief-action space with samples, exploiting Lipschitz continuity.
result Near-optimal value function computed efficiently.
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those …
Survey tackles continual learning in classification tasks to prevent forgetting.
problem Prevent catastrophic forgetting in neural networks across multiple tasks.
method Taxonomy, stability-plasticity trade-off framework, comprehensive comparison of methods.
result Empirical evaluation of continual learning methods on various benchmarks.
DeepMDP simplifies complex observations into continuous latent states.
problem Learning from high-dimensional observations in reinforcement learning.
method Trains a DeepMDP model that predicts rewards and next latent states.
result Optimization of DeepMDP objectives ensures quality of latent space and environment model.
CADD improves generative quality by augmenting discrete diffusion with continuous latent space.
problem Loss of semantic information between denoising steps in discrete diffusion models.
method Introduces a framework that augments discrete state space with a continuous latent space, allowing for graded, informative masked tokens.
result CADD improves generative quality across text generation, image synthesis, and code modeling.
Improves continual learning with theoretical guarantees and a new algorithm.
problem Learning incremental tasks with dynamic data distributions.
method Contrastive and distillation losses with theoretical performance guarantees.
result Theoretical performance bounds and improved continual learning performance.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
DisCoRL combines state learning and policy distillation for continual reinforcement learning.
problem Learning multiple policies with a single model and inferring the correct policy at test time.
method Combining state representation learning and policy distillation.
result The policy can solve all tasks and automatically infer the correct one to run.
CRUs model irregular time series with continuous hidden states.
problem Handling irregular time intervals in sequential data.
method Continuous Recurrent Units (CRUs) that integrate hidden states via a linear stochastic differential equation.
result CRUs outperform methods based on neural ordinary differential equations in irregular time series interpolation.
Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets…
Proposes a novel approach for RUL estimation of aero-engines.
problem Lack of prior knowledge for defining exact failure thresholds in dynamic environments.
method Simultaneous and dynamic prediction of continuous and discrete states within a single learning framework.
result Improves RUL estimation for aero-engines by reducing complexity.
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
New model for insurance states using Markov jump processes with non-countable state space.
problem Modeling insurance states with non-countable state spaces.
method Developed a new Thiele's differential equation for continuous time rehabilitation rates.
result Allows for consistent calculation of reserves in disability insurance.
New LTC RNNs can approximate any continuous system with fewer units.
problem Approximating continuous dynamical systems with neural networks.
method Introducing LTC RNNs with variable time-constant synaptic transmission.
result LTC RNNs can approximate any n-dimensional continuous dynamical system. ACSSM models irregular time series with continuous dynamics.
problem Modeling irregular time series data.
method ACSSM uses a multi-marginal Doob's h-transform and variational inference with stochastic optimal control.
result ACSSM outperforms in tasks like classification, regression, interpolation, and extrapolation.
Paper solves POMDPs in continuous time and discrete spaces.
problem Optimal decision making in discrete state and action space systems under partial observability.
method Combining optimal filtering theory and deep learning to solve a Hamilton-Jacobi-Bellman equation.
result Derives a mathematical description and solution approach for continuous-time POMDPs.
Develops DPG methods for continuous-time RL with deterministic policies.
problem High variance and slow convergence in stochastic policy RL methods.
method Derives continuous-time policy gradient formula and proposes CT-DDPG algorithm.
result CT-DDPG achieves superior stability and faster convergence in continuous-time RL.
MO2 learns useful behaviours from past experience for new tasks.
problem Discovering useful behaviours from past experience and transferring them to new tasks.
method Model-Based Offline Options (MO2) framework supporting sample-efficient bottleneck option discovery over continuous state-action spaces.
result MO2 outperforms recent option learning methods on complex long-horizon continuous control tasks.
GoTube verifies neural networks over time, scaling to large horizons.
problem Verifying the robustness of time-continuous neural networks.
method Solves Go problems to construct a conservative execution set.
result Substantially outperforms existing tools in size, speed, and scalability.
Neural Laplace Control tackles offline RL for continuous-time delayed systems with irregular observations.
problem Offline reinforcement learning problems involving continuous-time environments with delays and irregular observations.
method Combines a Neural Laplace dynamics model with a model predictive control (MPC) planner.
result Achieves near expert policy performance on continuous-time delayed environments.
Study robust control for systems with continuous states using adversarial perturbations.
problem Fragile policies in Markov control models under internal or external perturbations.
method Distributionally robust stochastic control with adaptive adversarial perturbations.
result Optimal robust policies for continuous state systems with uniform learning guarantees.
A new model minimizes investment risk at multiple time points.
problem Minimizing risk in investment portfolios with multiple stopping points.
method Developed a multi-time state mean-variance model using Riccati equations.
result Optimal investment strategies can be derived from a sequence of Riccati equations.
Study Transformer layers under cross-entropy training using mean field control.
problem Understanding the behavior of Transformer layers in cross-entropy training.
method Continuous-depth mean field control analysis, treating depth as time and layer parameters as controls.
result Derivation of a Pontryagin condition for the limiting population problem, involving the softmax residual.
This paper introduces a new neural ODE model for continuous-time sequence generation.
problem Representing and predicting continuous-time sequences with high accuracy.
method A neural emission model and neural ODE define the latent state evolution, with an Energy-based model for prior distribution.
result The model outperforms existing methods in various tasks, including long-horizon predictions.