New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
A new Fourier model improves ODE prediction.
problem Improving the accuracy of ODE solutions, especially for periodic functions.
method Constructing a Fourier state space model and a hybrid model combining Taylor and Fourier methods.
result The hybrid model can predict ODE solutions more accurately, especially for periodic functions.
Neural Jump ODEs improve online filtering and classification with robust performance.
problem Online filtering and classification in settings with irregular and partial observations.
method Modeling conditional expectation using Neural Jump ODEs, with theoretical convergence guarantees.
result Demonstrated superior performance over classical methods, especially in complex scenarios.
New method uses Gaussian ODE filtering to approximate likelihoods for fast ODE inverse problems.
problem Intractable forward models in likelihood-free inference, especially for ODEs.
method Gaussian ODE filtering to construct local Gaussian likelihood approximations.
result New solvers outperform standard likelihood-free approaches on benchmark systems.
Neural Jump ODE improves continuous-time prediction and filtering of irregularly sampled time series.
problem Theoretical guarantees for continuous-time prediction and filtering of irregularly observed time series.
method Introducing Neural Jump ODE (NJ-ODE) that models conditional expectation between observations with neural ODEs and jumps.
result Theoretical guarantees for the L2-optimal prediction are provided, showing convergence of model output to optimal prediction. There is resurging interest, in statistics and machine learning, in solvers for ordinary differential equations (ODEs) that return probability measures instead of point estimates. Recently, Conrad et al. introduced a sampling-based class of methods that are 'well-calibrated' in a specific sense. But the computational c…
New method combines ODE solvers with Bayesian inference for efficient model training.
problem Combining ODE solvers with Bayesian inference for efficient model training.
method Probabilistic state space model using extended Kalman filter for joint inference from differential equations and data.
result Efficient approximate Bayesian inference on latent force and ODE solution.
We formulate probabilistic numerical approximations to solutions of ordinary differential equations (ODEs) as problems in Gaussian process (GP) regression with non-linear measurement functions. This is achieved by defining the measurement sequence to consist of the observations of the difference between the derivative …
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, an…
A recently-introduced class of probabilistic (uncertainty-aware) solvers for ordinary differential equations (ODEs) applies Gaussian (Kalman) filtering to initial value problems. These methods model the true solution x and its first q derivatives \emph{a priori} as a Gauss--Markov process X, which is…
This paper develops an asymptotic expansion technique in momentum space for stochastic filtering. It is shown that Fourier transformation combined with a polynomial-function approximation of the nonlinear terms gives a closed recursive system of ordinary differential equations (ODEs) for the relevant conditional distri…
Paper improves neural ODEs for forecasting non-Markovian processes.
problem Forecasting irregularly observed time series with incomplete data.
method Path-dependent Neural Jump ODEs with signature transform.
result Path-dependent NJ-ODE outperforms original framework in non-Markovian data.
This paper presents, evaluates, and discusses a new software tool to automatically build Dynamic Bayesian Networks (DBNs) from ordinary differential equations (ODEs) entered by the user. The DBNs generated from ODE models can handle both data uncertainty and model uncertainty in a principled manner. The application, na…
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
Mathematical modeling with Ordinary Differential Equations (ODEs) has proven to be extremely successful in a variety of fields, including biology. However, these models are completely deterministic given a certain set of initial conditions. We convert mathematical ODE models of three benchmark biological systems to Dyn…
A new method for estimating uncertainties in neural ODEs without numerical integration.
problem Accurate estimation of predictive uncertainties in neural ODEs.
method Distributional Gradient Matching (DGM) algorithm that jointly trains a smoother and a dynamics model.
result Significantly more accurate predictions compared to traditional methods.
Extends nonlinear filtering to predictable jump times.
problem Filtering with jumps in both signal and observation, especially when jump times are known.
method Derive Kushner-Stratonovich and Zakai equations for predictable discontinuities.
result Extends classical nonlinear filtering results to a setting with predictable discontinuities.
Bayesian approach improves ODE solution accuracy.
problem Improving numerical solutions of ordinary differential equations.
method Bayesian inference with Gaussian filtering and smoothing.
result Maximum a posteriori estimate converges to true solution at polynomial rate.
Probabilistic method combines space and time uncertainties in PDEs.
problem Separate treatment of space and time in PDE solvers obscures interactions and error quantification.
method Gaussian process interpretation of finite difference methods interacting with probabilistic ODE solvers.
result Joint quantification of space- and time-uncertainty possible without sacrificing ODE solver performance.
Like many numerical methods, solvers for initial value problems (IVPs) on ordinary differential equations estimate an analytically intractable quantity, using the results of tractable computations as inputs. This structure is closely connected to the notion of inference on latent variables in statistics. We describe a …
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
Generative models need per-sample confidence scores to improve quality and stability.
problem Generative models produce unreliable outputs and lack confidence measures.
method Flow Matching with Confidence (FMwC) injects noise and integrates it through the network, providing per-sample confidence scores.
result The confidence score correlates with the velocity field's divergence, offering insights into generative processes.
Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace t…
Minimal surfaces in third-order ODEs identified for linear second-order ODEs.
problem Characterizing minimal surfaces in third-order ODEs.
method Analyzing submanifolds of third-order ODEs as Riemannian manifolds.
result Linear second-order ODEs with y′′=±y+β(x) are the only minimal surfaces and totally geodesic. DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
Faster training of neural ODEs using Gauß-Legendre quadrature.
problem Training neural ODEs is slow due to solving ODEs numerically.
method Use Gauß-Legendre quadrature to solve integrals faster than ODE-based methods.
result Faster training of neural ODEs, especially for large models.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
In this paper we propose the use of continuous residual modules for graph kernels in Graph Neural Networks. We show how both discrete and continuous residual layers allow for more robust training, being that continuous residual layers are those which are applied by integrating through an Ordinary Differential Equation …
Sequential data often originates from diverse domains across which statistical regularities and domain specifics exist. To specifically learn cross-domain sequence representations, we introduce disentangled state space models (DSSM) -- a class of SSM in which domain-invariant state dynamics is explicitly disentangled f…
Paper analyzes convergence of ODE samplers in Wasserstein distances.
problem Limited theoretical understanding of convergence properties of probability flow ODEs.
method Convergence analysis for general probability flow ODEs in 2-Wasserstein distance.
result First non-asymptotic convergence analysis for probability flow ODE samplers.
First-order ODEs linked to flat surfaces, leading to integrability.
problem Integrating first-order ODEs.
method Defined Riemannian metrics on variable spaces, studied surface properties, and established connections between Jacobi fields and Lie point symmetries.
result Flat associated surfaces lead to integrable first-order ODEs.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
We present Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
The study identifies exceptions to fiber-preserving symmetry in ODEs and systems.
problem Identifying exceptions to fiber-preserving symmetry in ODEs and systems.
method Lie's classification of Lie algebras of vector fields, absolute and relative scalar differential invariants, conditional and vector-valued relative invariants, prolongations of actions.
result Examples of scalar ODEs and systems with symmetry groups not fiber-preserving.
New symmetry dimensions for higher order ODEs are identified.
problem Determining the maximal and submaximal symmetry dimensions for higher order ODEs.
method Cartan-geometric approach to classify symmetry dimensions.
result Next largest realizable symmetry dimensions for scalar ODEs of order ≥ 4 and vector ODEs of order ≥ 3 are determined.
The aim of this paper is to construct a Riemann-Lagrange geometry on 1-jet spaces, in the sense of d-connections, d-torsions, d-curvatures, electromagnetic d-field and geometric electromagnetic Yang-Mills energy, starting from a given linear ODEs system or a given superior order ODE. The case of a non-homogenous linear…
Enhanced Neural ODEs outperform traditional models in image classification and video prediction.
problem Efficiently modeling time-varying dynamics in neural networks.
method Proposed a novel family of non-autonomous Neural ODEs with time-varying weights.
result Outperformed previous Neural ODE variants in speed and representational capacity.
This paper explores normalization in neural ODEs, achieving high accuracy in CIFAR-10.
problem Understanding the role of normalization in neural ODEs.
method Investigated different normalization techniques and their impact on neural ODEs performance.
result Achieved 93% accuracy in CIFAR-10 classification task.
Derives PF-ODE for infinite-dimensional functions, improving function generation tasks.
problem Efficient inference in infinite-dimensional diffusion models.
method Derives PF-ODE in infinite-dimensional function spaces.
result Reduces function evaluations while maintaining sample quality.
Neural ODEs' performance varies with numerical method, requiring adaptive step size control.
problem Neural ODEs' performance depends on the numerical method used during training.
method Proposes an adaptive step size control algorithm to ensure a valid ODE without increasing computational cost.
result Valid Neural ODEs require careful numerical method selection and step size adaptation.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
New vector fields integrate first-order ODEs.
problem Integrating first-order ODEs.
method Relation between Riemannian manifolds and ODEs integration.
result Integration procedure for first-order ODEs.
We compute the characteristic Cartan connection associated with a system of third order ODEs. Our connection is different from Tanaka normal one, but still is uniquely associated with the system of third order ODEs. This allows us to find all fundamental invariants of a system of third order ODEs and, in particular, de…
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.