Faster training of neural ODEs using Gauß-Legendre quadrature.
problem Training neural ODEs is slow due to solving ODEs numerically.
method Use Gauß-Legendre quadrature to solve integrals faster than ODE-based methods.
result Faster training of neural ODEs, especially for large models.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
New symmetry dimensions for higher order ODEs are identified.
problem Determining the maximal and submaximal symmetry dimensions for higher order ODEs.
method Cartan-geometric approach to classify symmetry dimensions.
result Next largest realizable symmetry dimensions for scalar ODEs of order ≥ 4 and vector ODEs of order ≥ 3 are determined.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
Statistical approach uses ODEs for modeling individual health trajectories.
problem Challenges in applying ODEs to longitudinal cohort data, especially noise and parameter sensitivity.
method Combines ODEs with neural networks to model individual health trajectories using each observation as initial value.
result Demonstrates improved modeling of individual health trajectories compared to global regression.
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is…
We present Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
New method uses Gaussian ODE filtering to approximate likelihoods for fast ODE inverse problems.
problem Intractable forward models in likelihood-free inference, especially for ODEs.
method Gaussian ODE filtering to construct local Gaussian likelihood approximations.
result New solvers outperform standard likelihood-free approaches on benchmark systems.
Neural Jump ODE improves continuous-time prediction and filtering of irregularly sampled time series.
problem Theoretical guarantees for continuous-time prediction and filtering of irregularly observed time series.
method Introducing Neural Jump ODE (NJ-ODE) that models conditional expectation between observations with neural ODEs and jumps.
result Theoretical guarantees for the L2-optimal prediction are provided, showing convergence of model output to optimal prediction. New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
New method trains neural ODEs faster with fewer layers.
problem Training neural ODEs on large datasets is computationally expensive.
method Combines optimal transport and stability regularizations.
result Significant reductions in training time with no performance loss.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
Neural Jump ODEs improve online filtering and classification with robust performance.
problem Online filtering and classification in settings with irregular and partial observations.
method Modeling conditional expectation using Neural Jump ODEs, with theoretical convergence guarantees.
result Demonstrated superior performance over classical methods, especially in complex scenarios.
A new method for estimating uncertainties in neural ODEs without numerical integration.
problem Accurate estimation of predictive uncertainties in neural ODEs.
method Distributional Gradient Matching (DGM) algorithm that jointly trains a smoother and a dynamics model.
result Significantly more accurate predictions compared to traditional methods.
New approach connects stochastic gradient descent to ODE splitting schemes.
problem Improving convergence in stochastic optimization.
method Connection between stochastic gradient descent and ODE splitting schemes.
result Derive a new upper bound on global splitting error.
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
This paper compares two methods for training neural ODEs in time-series regression and CNFs.
problem Training neural ODEs for time-series regression and CNFs efficiently.
method Discretize-Optimize (Disc-Opt) vs. Optimize-Discretize (Opt-Disc) approaches.
result Disc-Opt methods can achieve similar performance as Opt-Disc at inference with drastically reduced training costs.
The study focuses on estimating and predicting time-varying origin to destination (OD) trip tables for a dynamic traffic assignment (DTA) model. A bi-level optimisation problem is formulated and solved to estimate OD flows from pre-existent demand matrix and historical traffic flow counts. The estimated demand is then …
Extends Neural ODEs to model discrete changes in continuous systems.
problem Lack of explicit termination time in existing Neural ODE formulations.
method Introduces neural event functions to implicitly define termination criteria.
result Models discrete changes in continuous systems without prior knowledge.
LHM integrates expert ODEs with neural ODEs for disease progression prediction.
problem Predicting disease progression under medications using limited data.
method Integrating expert-designed ODEs with machine-learned Neural ODEs.
result LHM consistently outperforms previous methods, especially in small sample regimes.
New method calibrates complex ODEs from noisy data using neural networks.
problem Calibrating multi-dimensional complex ODEs from noisy data.
method Two-stage approach: de-noising and higher-order derivatives followed by a deep neural network.
result Consistent recovery of ODE system without curse of dimensionality.
Researchers dissect Neural ODEs to understand their dynamics.
problem Understanding the inner workings of Neural ODEs.
method Developing continuous-depth formulation to clarify design choices.
result Clarified the influence of design choices on Neural ODE dynamics.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
A new method combines ANN and Laplace for fast Bayesian inference in ODE models.
problem Bayesian inference for ODE systems with non-analytical solutions is computationally expensive.
method Hybrid approach using ANN for tractable likelihood and Laplace approximation.
result Effective posterior inference with improved computational cost compared to traditional methods.
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
The equivalence problem for second order ODEs given modulo point transformations is solved in full analogy with the equivalence problem of nondegenerate 3-dimensional CR structures. This approach enables an analog of the Feffereman metrics to be defined. The conformal class of these (split signature) metrics is well de…
The paper introduces a new ODE approach to improve Wasserstein GANs.
problem Improving Wasserstein GANs for better training results.
method Derives an ODE representing the gradient flow of Wasserstein-1 loss and proposes a new model W1-FE.
result W1-FE outperforms WGAN in training experiments across various dimensions.
WENDy now estimates nonlinear ODEs with noisy data.
problem Estimating parameters of nonlinear ODEs with noisy data.
method WENDy-MLE algorithm for maximum likelihood estimation of nonlinear-in-parameters ODEs.
result WENDy-MLE outperforms other methods in accuracy, speed, and domain of convergence.
New error bounds for flow matching methods using deterministic sampling.
problem Improving the accuracy of flow matching methods for generating probability distributions.
method Derived error bounds for flow matching methods under deterministic sampling conditions.
result Presented error bounds for flow matching methods using L2 loss and regularity conditions. New method constructs confidence bands for ODE models with unknown regulatory effects.
problem Building confidence bands for ODE models with unknown regulatory relations is challenging.
method Localized kernel learning approach combined with de-biasing method.
result The constructed confidence band has the desired asymptotic coverage probability and accurately recovers the regulatory network.
Balanced Neural ODEs combine VAEs and Neural ODEs for efficient time series modeling.
problem Efficiently modeling systems with time-varying inputs and varying complexity.
method Combines VAEs for dimensionality reduction and Neural ODEs for dynamics, using variational parameters to adaptively learn.
result Balanced Neural ODEs (B-NODE) efficiently approximate Koopman operator without predefined dimensionality.
New RL framework models continuous-time dynamics using neural ODEs.
problem Modeling continuous-time dynamics in semi-Markov decision processes.
method Model-based reinforcement learning with neural ODEs.
result High-performing policies developed with minimal data.
Gradient matching with Gaussian processes is a promising tool for learning parameters of ordinary differential equations (ODE's). The essence of gradient matching is to model the prior over state variables as a Gaussian process which implies that the joint distribution given the ODE's and GP kernels is also Gaussian di…
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
Neural ODEs extended to manifolds for flexible sampling.
problem Sampling from complex multimodal distributions on non-trivial topologies.
method Extending Neural ODEs to smooth manifolds using vector fields.
result A general methodology for building normalizing flows on manifolds.
Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace t…
In 1896 Tresse gave a complete description of relative differential invariants for the pseudogroup action of point transformations on the 2nd order ODEs. The purpose of this paper is to review, in light of modern geometric approach to PDEs, this classification and also discuss the role of absolute invariants and the eq…
Bayesian Gaussian Process ODEs enhanced with normalizing flows for improved flexibility and accuracy.
problem Limitations of standard Gaussian Process ODEs in modeling complex scenarios.
method Introducing normalizing flows to reparameterize the ODE vector field, developing a data-driven variational learning algorithm.
result Improved accuracy and uncertainty estimates for Bayesian Gaussian Process ODEs.
High-probability bound for distributed stochastic approximation tracking error.
problem Analyzing the convergence of distributed stochastic approximation schemes.
method Analysis using ODE approach to stochastic approximation.
result High probability bound for tracking error between iterates and limiting differential equation.
Minimal surfaces in third-order ODEs identified for linear second-order ODEs.
problem Characterizing minimal surfaces in third-order ODEs.
method Analyzing submanifolds of third-order ODEs as Riemannian manifolds.
result Linear second-order ODEs with y′′=±y+β(x) are the only minimal surfaces and totally geodesic. DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.
The paper provides convergence guarantees for ODE-based generative models using transformers.
problem Theoretical guarantees for ODE-based generative models.
method A pre-trained autoencoder maps inputs to a latent space, and a transformer predicts the velocity field.
result The distribution of samples generated via estimated ODE flow converges to the target distribution in Wasserstein-2 distance.
CODE learns ODE dynamics from sparse data, outperforming neural and kernel methods.
problem Learning ODE dynamics from sparse and noisy data.
method CODE uses Polynomial Chaos Expansion (aPCE) for the ODE's RHS, enabling global orthonormal polynomial representation.
result CODE exhibits remarkable extrapolation capabilities even under novel initial conditions and measurement noise.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
Develops a new method for learning ODEs from sparse data.
problem Learning systems of ODEs from scarce, partial, and noisy data.
method Combines sparse recovery and RKHS techniques.
result Significant gains in accuracy, sample efficiency, and robustness to noise.
Paper analyzes convergence of ODE samplers in Wasserstein distances.
problem Limited theoretical understanding of convergence properties of probability flow ODEs.
method Convergence analysis for general probability flow ODEs in 2-Wasserstein distance.
result First non-asymptotic convergence analysis for probability flow ODE samplers.