Faster policy learning via continuous-time gradients.
problem Efficiently estimating policy gradients for continuous-time systems.
method Approximating continuous-time gradients directly, using adaptive discretization.
result More efficient policy gradient estimator leads to faster learning.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
Study shows certainty equivalent policy minimizes regret in continuous-time systems.
problem Minimizing regret in continuous-time stochastic linear-quadratic systems.
method Theoretical analysis of randomized certainty equivalent policy.
result Establishes square-root of time regret bounds and linear scaling with parameters.
New algorithm for continuous-time switching systems using variational inference.
problem Inference in time-series data with continuous-time switching systems.
method Developed a variational inference algorithm combining Gaussian process approximation and posterior inference for Markov jump processes.
result Bayesian latent state estimates and point estimates of unknown parameters for arbitrary points on the real axis.
Bayesian algorithm stabilizes unknown continuous-time systems from unstable data.
problem Learning and stabilizing unknown continuous-time systems with uncertain dynamics.
method Bayesian learning algorithm that learns from unstable data to stabilize the system in finite time.
result The algorithm stabilizes unknown continuous-time stochastic linear systems effectively after a short time period.
Proposes exact inference for continuous-time Gaussian process dynamics.
problem Inexact inference methods for continuous-time Gaussian process dynamics are impractical for irregularly-sampled data.
method Uses higher-order numerical integrators to discretize dynamics with arbitrary accuracy and proposes multistep and Taylor integrators for exact inference.
result Demonstrates accurate representation of continuous-time systems through exact GP inference.
Neural models learn continuous-time Markov chain transition rates from data.
problem Learning transition rates for complex stochastic systems.
method Neural networks to model nonlinear transition rates from observed data.
result Neural models outperform traditional methods in accuracy.
Neural Laplace Control tackles offline RL for continuous-time delayed systems with irregular observations.
problem Offline reinforcement learning problems involving continuous-time environments with delays and irregular observations.
method Combines a Neural Laplace dynamics model with a model predictive control (MPC) planner.
result Achieves near expert policy performance on continuous-time delayed environments.
We solve a continuous-time game-theoretic problem for Kihlstrom-Mirman preferences.
problem Dynamic inconsistency in preferences due to multiattribute utility theory.
method Formalized an equilibrium control theory for continuous-time Markov processes.
result Equilibrium strategy and value function as solution to extended HJB system.
Graph neural networks learn PDEs from sparse, irregular data.
problem Learning PDEs from irregularly spaced data.
method Continuous-time differential model with graph neural networks for arbitrary discretizations.
result Efficient inference with continuous-time adjoint method.
VSDN models sporadic time series with neural SDEs.
problem Modeling irregular and sparse time series data.
method Variational Bayesian method and neural SDEs.
result VSDNs outperform state-of-the-art models in prediction and interpolation.
In this paper, two Q-learning (QL) methods are proposed and their convergence theories are established for addressing the model-free optimal control problem of general nonlinear continuous-time systems. By introducing the Q-function for continuous-time systems, policy iteration based QL (PIQL) and value iteration based…
Paper solves POMDPs in continuous time and discrete spaces.
problem Optimal decision making in discrete state and action space systems under partial observability.
method Combining optimal filtering theory and deep learning to solve a Hamilton-Jacobi-Bellman equation.
result Derives a mathematical description and solution approach for continuous-time POMDPs.
FHRN uses continuous-time dynamics to stabilize reentrant neural computation.
problem Stabilizing reentrant neural computation.
method Formulated as a continuous-time neural-ODE system, revealing norm-regulated reentry.
result Achieves stable oscillatory trajectories through population-level gain modulation.
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In particular, we establish general sufficient conditions for universal approximation using continuous-time deep residual networks, which can …
Study on estimating unstable open-loop matrices from state trajectories.
problem System identification for stochastic continuous-time dynamics.
method Employing randomized control inputs to estimate unstable open-loop matrix.
result Estimation error decays with trajectory length, signal-to-noise ratio, and excitability.
Efficiently infers coupled hidden Markov models with noisy discrete observations.
problem Intractable inference for coupled continuous-time Markov chains with discrete observations.
method Latent Interacting Particle Systems, look-ahead functions, twisted Sequential Monte Carlo sampling.
result Demonstrated effectiveness on latent SIRS model and wildfire spread dynamics.
In this paper, we propose two discontinuous dynamical systems in continuous time with guaranteed prescribed finite-time local convergence to strict local minima of a given cost function. Our approach consists of exploiting a Lyapunov-based differential inequality for differential inclusions, which leads to finite-time …
Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets…
This paper tackles Bayesian system identification with probabilistic numerical methods.
problem Accurately modeling nonlinear dynamic systems from noisy data.
method Probabilistic Sequential Monte Carlo (SMC) combined with probabilistic numerical integration.
result Efficient identification of latent states and system parameters from noisy measurements.
Modeling cascading behavior in complex systems using CTBNs.
problem Understanding which states trigger cascading events in complex systems.
method Continuous-time Bayesian networks (CTBNs) for modeling and identifying likely sentry states.
result Identification of likely sentry states that may lead to cascading behavior.
Paper develops a consistent algorithm for learning graph structure from continuous-time stochastic differential equations.
problem Learning structure from continuous-time stochastic differential equations.
method Score-based structure learning using Neural Ordinary Differential Equations with adaptive regularization.
result The method consistently recovers directed graphs of local independencies in systems of stochastic differential equations.
We solve continuous-time latent SDE identifiability using diffusion shifts.
problem Identifiability of latent SDEs in continuous-time time series.
method Environment-induced shifts in diffusion covariance for additive-noise latent SDEs.
result Two diagonal diffusion regimes with distinct variance ratios identify latent coordinates up to permutation and scaling.
We develop an approach to learn an interpretable semi-parametric model of a latent continuous-time stochastic dynamical system, assuming noisy high-dimensional outputs sampled at uneven times. The dynamics are described by a nonlinear stochastic differential equation (SDE) driven by a Wiener process, with a drift evolu…
Recommender systems play an essential role in the modern business world. They recommend favorable items like books, movies, and search queries to users based on their past preferences. Applying similar ideas and techniques to Monte Carlo simulations of physical systems boosts their efficiency without sacrificing accura…
Solves steering problem with continuous time, Hilbert-Schmidt cost, and matrix ODEs.
problem Fixed horizon linear quadratic covariance steering in continuous time with a specific terminal cost.
method Formulates necessary conditions as a coupled matrix ODE two-point boundary value problem, designs a matricial recursive algorithm, and proves convergence.
result Proposes and proves the convergence of a matricial recursive algorithm for solving the steering problem.
New method solves optimization problems on manifolds using symplectic integrators.
problem Optimization tasks on manifolds with nonlinear constraints.
method Dissipative extension of Dirac's theory of constrained Hamiltonian systems and geometric/symplectic numerical integrators.
result Developed algorithms achieve optimal convergence rates locally.
Continuous-time PCD for MLE with explicit error bounds.
problem Maximum likelihood estimation of unnormalised densities.
method Continuous-time formulation as coupled SDEs, deriving UiT bounds.
result Explicit error bounds between PCD iterates and MLE solution.
New method learns policies from offline data using operator models.
problem Limited understanding of approximation errors in offline reinforcement learning.
method Linking reinforcement learning to Hamilton-Jacobi-Bellman equation, proposing operator-theoretic algorithm.
result Global convergence of the value function and finite-sample guarantees derived.
The paper develops RL methods for optimal switching between multiple states.
problem Optimal switching between multiple states in continuous time.
method Entropy-regularized exploration, HJB equations, policy improvement, value function convergence.
result The RL algorithm converges to optimal policies as temperature parameter vanishes.
We prove that every flat nonlinear discrete-time system can be decomposed by coordinate transformations into a smaller-dimensional subsystem and an endogenous dynamic feedback. For flat continuous-time systems, no comparable result is available. The advantage of such a decomposition is that the complete system is flat …
DQNs can approximate optimal Q-functions with high accuracy on compact sets.
problem Approximating optimal Q-functions in continuous-time Markov Decision Processes.
method Stochastic control, FBSDEs, residual network approximation theorems, large deviation bounds, viscosity solutions.
result DQNs can approximate optimal Q-functions on compact sets with arbitrary accuracy and high probability.
Extended flatness approach for discrete-time systems considers forward and backward shifts.
problem Defining flatness for discrete-time systems with forward-shifts.
method Introducing backward-shifts to extend flatness definition.
result Extended flat systems maintain key properties like reachability and controllability.
New framework for fair online allocation in continuous time with deadlines.
problem Fair allocation under deadlines in continuous-time online learning.
method Continuous-time utility maximization, dual ascent optimization for time averages.
result Achieves ildeO(B−1/2) regret bound in the absence of statistical knowledge. Continuous-time Bayesian networks (CTBNs) constitute a general and powerful framework for modeling continuous-time stochastic processes on networks. This makes them particularly attractive for learning the directed structures among interacting entities. However, if the available data is incomplete, one needs to simulat…
SIMPOL solves complex economic models using numerical methods.
problem Optimizing consumption and savings under uncertainty.
method SIMPOL uses a modular numerical framework combining policy iteration and finite difference schemes.
result SIMPOL produces solutions consistent with economic and mathematical theory.
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empirically demonstrated that the gradient noise in several deep learning settings admits a non-Gaussian, heavy-tailed behavior. This suggests tha…
We find a normal form for two-input flat discrete-time systems.
problem No comparable normal form exists for flat continuous-time systems.
method State- and input transformations to achieve a triangular structure.
result A systematic parameterization of system variables by the flat output and its shifts.
We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous tim…
Unified framework for PE and TD methods in continuous time and space.
problem Policy evaluation and TD learning in continuous settings.
method Martingale characterization for designing PE algorithms.
result Convergent time-discretized algorithms converge to continuous-time counterparts.
Study policy gradient for large-agent mean-field control and game in continuous time.
problem Optimal policy learning for large number of agents in continuous-time mean-field systems.
method Policy gradient method applied to linear-quadratic mean-field control and game models.
result Policy gradient converges to optimal solution at a linear rate for both mean-field control and game.
New analysis shows FM learns underlying dynamical structure, not just trajectory replay.
problem Understanding whether flow matching models learn transferable dynamical structure or merely replay trajectories.
method Derived velocity field implied by FM objective, characterized as a continuous-time dynamical system.
result FM models can be seen as parametric surrogates of nonparametric solutions, providing strong probabilistic forecasts.
Defines hybrid systems on principal bundles and studies impact effects.
problem Understanding impact effects in hybrid mechanical systems.
method Defines hybrid systems on principal bundles, studies underlying geometry, and finds conditions for impact preservation.
result Conditions for preservation of both exterior and interior impacts by mechanical connections.
Lipschitz RNNs improve stability and performance in various tasks.
problem Improving stability and performance of RNNs.
method Introduced a Lipschitz recurrent unit with a linear and Lipschitz nonlinear component for stability analysis.
result Lipschitz RNNs outperform existing units on benchmark tasks.
Neural dynamical systems are dynamical systems that are described at least in part by neural networks. The class of continuous-time neural dynamical systems must, however, be numerically integrated for simulation and learning. Here, we present a compact neural circuit for two common numerical integrators: the explicit …
Continuous time framework for discrete data denoising models.
problem Efficient training and sampling for discrete data denoising models.
method Formulated as Continuous Time Markov Chains (CTMCs), efficient training using continuous time ELBO, high-dimensional CTMC simulation, novel theoretical error bound.
result Continuous time treatment enables novel theoretical error bound between generated and true data distributions.
We establish causal semantics for SDEs and develop methods to reason about them.
problem Understanding causal relationships in systems modeled by stochastic differential equations.
method We introduce a causal graph framework, Markov properties, and do-calculus for SDEs.
result We prove the σ-separation Markov property and do-calculus for causal SDEs. This paper introduces a new neural ODE model for continuous-time sequence generation.
problem Representing and predicting continuous-time sequences with high accuracy.
method A neural emission model and neural ODE define the latent state evolution, with an Energy-based model for prior distribution.
result The model outperforms existing methods in various tasks, including long-horizon predictions.