Study analyzes a new algorithm for complex optimization problems.
problem Stochastic bilevel optimisation problems in continuous-time models.
method Continuous-time, two-timescale stochastic approximation algorithm.
result Obtained weak convergence rate using central limit theorem.
Continuous-time SGD converges under certain conditions, useful for deep learning.
problem Minimizing population expected loss in learning problems.
method Continuous-time approximation of stochastic gradient descent.
result Establishes sufficient conditions for convergence, applicable to overparametrized neural networks.
DQNs can approximate optimal Q-functions with high accuracy on compact sets.
problem Approximating optimal Q-functions in continuous-time Markov Decision Processes.
method Stochastic control, FBSDEs, residual network approximation theorems, large deviation bounds, viscosity solutions.
result DQNs can approximate optimal Q-functions on compact sets with arbitrary accuracy and high probability.
New algorithm learns value and advantage functions for continuous-time Markov processes without structural assumptions.
problem Learning value and advantage functions for continuous-time Markov processes without structural assumptions.
method Proposes Sobolev-prox fitted q-learning algorithm based on Hilbert-space positive definiteness and boundedness properties of Bellman operators. result Identifies ellipticity as a key structural property enabling reinforcement learning for Markov diffusions.
Faster policy learning via continuous-time gradients.
problem Efficiently estimating policy gradients for continuous-time systems.
method Approximating continuous-time gradients directly, using adaptive discretization.
result More efficient policy gradient estimator leads to faster learning.
Continuous-time Bayesian networks (CTBNs) constitute a general and powerful framework for modeling continuous-time stochastic processes on networks. This makes them particularly attractive for learning the directed structures among interacting entities. However, if the available data is incomplete, one needs to simulat…
This work develops efficient methods for continuous-time distributional reinforcement learning.
problem Continuous-time reinforcement learning with return distributions.
method Parameterizing return distributions using quantile representation and showing topological properties.
result Efficient approximation algorithm for continuous-time distributional reinforcement learning.
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In particular, we establish general sufficient conditions for universal approximation using continuous-time deep residual networks, which can …
We consider the inverse problem of reconstructing the posterior measure over the trajec- tories of a diffusion process from discrete time observations and continuous time constraints. We cast the problem in a Bayesian framework and derive approximations to the posterior distributions of single time marginals using vari…
New method decomposes profits and losses continuously, avoiding discrete reporting issues.
problem Analyzing profits and losses at discrete dates ignores detailed paths.
method Constructs a large class of continuous-time decompositions using extended Itô's formula.
result Identifies a preferred decomposition from exactness, symmetry, and normalization axioms.
Paper solves POMDPs in continuous time and discrete spaces.
problem Optimal decision making in discrete state and action space systems under partial observability.
method Combining optimal filtering theory and deep learning to solve a Hamilton-Jacobi-Bellman equation.
result Derives a mathematical description and solution approach for continuous-time POMDPs.
We solve continuous-time reinforcement learning using distributional Hamilton-Jacobi-Bellman equations.
problem Predicting the distribution of returns in continuous-time, stochastic environments.
method We derive a distributional Hamilton-Jacobi-Bellman equation for Itô diffusions and Feller-Dynkin processes, and propose an algorithm based on a JKO scheme.
result We propose an online control algorithm that can be used to approximately solve the distributional HJB equation.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman equation and link it with the impulse control problem. The optimal strategy for the und…
Revisits superhedging under proportional costs in continuous time markets.
problem Superhedging in markets with proportional transaction costs.
method Set-valued stochastic analysis, continuous trading schemes, dynamic risk measure.
result Dynamic set-valued risk measure with multi-portfolio time-consistency.
Unified framework for PE and TD methods in continuous time and space.
problem Policy evaluation and TD learning in continuous settings.
method Martingale characterization for designing PE algorithms.
result Convergent time-discretized algorithms converge to continuous-time counterparts.
Generalizes PCA and ICA for continuous-time signals using neural networks.
problem Low-rank decomposition of continuous-time vector-valued signals.
method Implicit neural network framework to learn numerical approximations of PCA and ICA.
result Unified approach to PCA and ICA in continuous domain, enforcing decorrelation and independence.
New methods for inferring, predicting, and estimating continuous-time, discrete-event processes.
problem Inferring, predicting, and estimating entropy rate of continuous-time, discrete-event processes.
method Bayesian structural inference extended with neural networks.
result Methods are competitive for prediction and entropy-rate estimation with state-of-the-art.
Gradient descent-based optimization methods underpin the parameter training of neural networks, and hence comprise a significant component in the impressive test results found in a number of applications. Introducing stochasticity is key to their success in practical problems, and there is some understanding of the rol…
Study approximates financial market with discrete-time models.
problem Approximating continuous-time financial market models with discrete-time.
method Constructs discrete-time market models with Markov switching and proves convergence.
result Discrete-time models converge to continuous-time Black-Scholes model with Markov switching.
We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however i…
Paper approximates rough stochastic local volatility models for efficient computation.
problem No unified method for rough stochastic local volatility models.
method Semimartingale and continuous-time Markov chain approximation.
result Fast CTMC algorithm with weak convergence proved.
Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a…
Transformers can solve complex filtering problems for non-Gaussian signals.
problem Non-linear and non-Markovian filtering problems for conditionally Gaussian signals.
method Continuous-time transformer models called filterformers.
result Filterformers can approximate the conditional law of non-Markovian and conditionally Gaussian signal processes.
The discrete-time GARCH methodology which has had such a profound influence on the modelling of heteroscedasticity in time series is intuitively well motivated in capturing many `stylized facts' concerning financial series, and is now almost routinely used in a wide range of situations, often including some where the d…
New algorithm for continuous-time switching systems using variational inference.
problem Inference in time-series data with continuous-time switching systems.
method Developed a variational inference algorithm combining Gaussian process approximation and posterior inference for Markov jump processes.
result Bayesian latent state estimates and point estimates of unknown parameters for arbitrary points on the real axis.
The hyperfinite G-expectation is a nonstandard discrete analogue of G-expectation (in the sense of Robinsonian nonstandard analysis). A lifting of a continuous-time G-expectation operator is defined as a hyperfinite G-expectation which is infinitely close, in the sense of nonstandard topology, to the continuous…
Efficiently infers coupled hidden Markov models with noisy discrete observations.
problem Intractable inference for coupled continuous-time Markov chains with discrete observations.
method Latent Interacting Particle Systems, look-ahead functions, twisted Sequential Monte Carlo sampling.
result Demonstrated effectiveness on latent SIRS model and wildfire spread dynamics.
Deep learning solves complex stochastic control with jumps.
problem Solving high-dimensional stochastic control tasks with jumps.
method Model-based approach using two neural networks, iteratively trained with objectives derived from the Hamilton-Jacobi-Bellman equation.
result Demonstrates effectiveness in solving complex high-dimensional stochastic control tasks.
Study optimal growth strategies in a continuous-time asset market.
problem Guaranteeing that individual agent strategies cannot outperform the market.
method Mean-field approximation of an infinite number of infinitesimal agents, focusing on optimal strategy distribution among assets.
result Optimal strategy for market agents is to invest proportionally to discounted expected relative dividend intensities.
Continuous-time Q-learning theory developed for reinforcement learning.
problem Continuous-time reinforcement learning challenges.
method Entropy-regularized, exploratory diffusion process formulation; first-order approximation of Q-function; martingale conditions.
result Developed a q-learning theory independent of time discretization.
Regime-switching models, in particular Hidden Markov Models (HMMs) where the switching is driven by an unobservable Markov chain, are widely-used in financial applications, due to their tractability and good econometric properties. In this work we consider HMMs in continuous time with both constant and switching volati…
Graph neural networks learn PDEs from sparse, irregular data.
problem Learning PDEs from irregularly spaced data.
method Continuous-time differential model with graph neural networks for arbitrary discretizations.
result Efficient inference with continuous-time adjoint method.
QSurv models survival data without discretization, achieving high accuracy.
problem Intractable likelihood estimation for continuous-time survival models.
method QSurv uses numerical quadrature for cumulative hazard approximation and time-conditioned low-rank adaptation.
result QSurv achieves competitive predictive performance and interpretable hazard patterns.
Optimal fund deployment strategy under uncertain deal arrivals.
problem Deciding when to invest in deals with uncertain future arrivals.
method Formulated as CTMDP, solved via ADP with QMC sampling.
result Developed interpretable acceptance policy outperforming baseline.
Continuous-time optimal stopping solved with deep reinforcement learning
problem Optimal stopping problems in continuous time
method CARLOS (Continuous-time Adaptive Reinforcement Learning for Optimal Stopping)
result Higher prices than existing Bermudan solvers, approaching American upper bound
Study on memory effects in RNNs learning temporal data.
problem Understanding memory effects in RNNs for temporal data learning.
method Mathematical analysis of continuous-time linear RNNs, focusing on approximation and optimization dynamics.
result Long-term memory requires a large number of neurons and slows down training.
Proposes exact inference for continuous-time Gaussian process dynamics.
problem Inexact inference methods for continuous-time Gaussian process dynamics are impractical for irregularly-sampled data.
method Uses higher-order numerical integrators to discretize dynamics with arbitrary accuracy and proposes multistep and Taylor integrators for exact inference.
result Demonstrates accurate representation of continuous-time systems through exact GP inference.
Novel approach to learn CTBNs from data with minimal interventions.
problem Learning CTBNs from time-course data with limited resources.
method Variational approximation of expected information gain for experimental design.
result Semi-analytical expression for structure and parameter learning.
New method uses tensor trains for efficient PDE approximation.
problem High-dimensional PDEs and the curse of dimensionality.
method Tensor trains and backward stochastic differential equations for parabolic PDEs.
result Achieves a favorable trade-off between accuracy and computational efficiency.
We demonstrate that a number of sociology models for social network dynamics can be viewed as continuous time Bayesian networks (CTBNs). A sampling-based approximate inference method for CTBNs can be used as the basis of an expectation-maximization procedure that achieves better accuracy in estimating the parameters of…
Study policy gradient for large-agent mean-field control and game in continuous time.
problem Optimal policy learning for large number of agents in continuous-time mean-field systems.
method Policy gradient method applied to linear-quadratic mean-field control and game models.
result Policy gradient converges to optimal solution at a linear rate for both mean-field control and game.
This paper considers binomial approximation of continuous time stochastic processes. It is shown that, under some mild integrability conditions, a process can be approximated in mean square sense and in other strong metrics by binomial processes, i.e., by processes with fixed size binary increments at sampling points. …
Paper solves Bayesian bandit problem with continuous-time limit and approximate policy.
problem Finding optimal policy in Bayesian bandit problems with large horizons.
method Reformulates Bayesian bandit problem as continuous Hamilton-Jacobi-Bellman (HJB) equation and proposes approximate Bayes-optimal policy.
result Approximate Bayes-optimal policy for large horizons with constant computational cost.
Study policy gradient and actor-critic methods for continuous-time reinforcement learning.
problem Continuous-time reinforcement learning with policy gradient and actor-critic approaches.
method Regularized exploratory formulation, martingale approach, simultaneous policy and value function updates.
result Proposed two types of actor-critic algorithms for online and offline learning.
Study on policy gradient for stochastic bandits using diffusion approximation.
problem Improving policy gradient methods for stochastic bandits with optimal regret bounds.
method Continuous-time diffusion approximation of policy gradient with learning rate analysis.
result Proved optimal regret bound of O(klog(k)log(n)/η) for η=O(Δ2/log(n)). The paper analyzes SGD and its continuous counterpart, improving convergence rates and approximation results.
problem Theoretical analysis of convergence rates and approximation results for SGD and its continuous-time counterpart.
method Provable approximation of SGD recursion by solutions of a time inhomogeneous SDE, using Stein's method for batch noise, and new comparison techniques.
result Improved non-asymptotic bounds for SGD under weaker assumptions and finite-time convergence results.
We derive the ODE of MAML and propose a new BI-MAML algorithm.
problem Training efficiency and computational burden in MAML.
method Continuous-time limit view of MAML, ODE derivation, and BI-MAML algorithm.
result MAML ODE shows linear convergence rate for strongly convex task losses.