New model handles uneven time intervals better than traditional methods.
problem Irregularly-sampled time series data.
method Generalizes RNNs to ODE-RNNs, explicitly modeling observation gaps.
result ODE-RNNs outperform traditional models on irregular data.
Augmented Neural ODEs improve stability and expressiveness of ODEs.
problem Limitations of Neural ODEs in representing complex functions.
method Introducing Augmented Neural ODEs that are more expressive and stable.
result Augmented Neural ODEs outperform Neural ODEs in stability, generalization, and computational efficiency.
ODE2VAE learns latent dynamics for sequential data.
problem Learning latent dynamics for high-dimensional sequential data.
method Deep generative second order ODE model with Bayesian neural networks.
result State-of-the-art performance in long-term motion prediction and imputation.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
Faster training of neural ODEs using Gauß-Legendre quadrature.
problem Training neural ODEs is slow due to solving ODEs numerically.
method Use Gauß-Legendre quadrature to solve integrals faster than ODE-based methods.
result Faster training of neural ODEs, especially for large models.
A new Fourier model improves ODE prediction.
problem Improving the accuracy of ODE solutions, especially for periodic functions.
method Constructing a Fourier state space model and a hybrid model combining Taylor and Fourier methods.
result The hybrid model can predict ODE solutions more accurately, especially for periodic functions.
This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
Gaussian processes learn unknown ODE dynamics from sparse data.
problem Learning unknown ODE models with limited data.
method Nonparametric ODE modelling using Gaussian process vector fields.
result Model infers dynamics from sparse data and simulates future states.
New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
Paper analyzes convergence of ODE samplers in Wasserstein distances.
problem Limited theoretical understanding of convergence properties of probability flow ODEs.
method Convergence analysis for general probability flow ODEs in 2-Wasserstein distance.
result First non-asymptotic convergence analysis for probability flow ODE samplers.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
Derives PF-ODE for infinite-dimensional functions, improving function generation tasks.
problem Efficient inference in infinite-dimensional diffusion models.
method Derives PF-ODE in infinite-dimensional function spaces.
result Reduces function evaluations while maintaining sample quality.
ANODEV2 extends Neural ODEs to include evolving parameters.
problem Training and accuracy of neural networks.
method Coupled ODE-based framework for evolving neural network parameters.
result ANODEV2 achieves higher accuracy than baseline models and Neural ODEs.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Study estimates and predicts dynamic traffic OD flows for improved DTA models.
problem Estimating and predicting time-varying OD trip tables for dynamic traffic assignment.
method Bi-level optimisation for OD flow estimation and time series prediction for OD demand.
result High capability of proposed OD demand estimation method to reduce DTA model error.
DBNs improve accuracy of biological ODE models with missing data.
problem Uncertainty in biological ODE models with missing data.
method Converted ODE models to DBNs and used Particle Filtering for parameter estimation.
result DBNs can accurately infer model variables with missing data.
Enhanced Neural ODEs outperform traditional models in image classification and video prediction.
problem Efficiently modeling time-varying dynamics in neural networks.
method Proposed a novel family of non-autonomous Neural ODEs with time-varying weights.
result Outperformed previous Neural ODE variants in speed and representational capacity.
Generative ODE model learns unknown variables in medical systems.
problem Estimating unknown variables in complex medical systems.
method Variational autoencoder incorporating known ODE functions.
result Modeling known-unknowns improves system parameter discovery and extrapolation.
Neural ODEs and i-ResNets can't approximate all continuous invertible functions.
problem Neural ODEs and i-ResNets' limitations in approximating continuous invertible functions.
method Proving the approximation capabilities of Neural ODEs and i-ResNets.
result Neural ODEs and i-ResNets can approximate homeomorphisms on a p-dimensional Euclidean space.
DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.
LHM integrates expert ODEs with neural ODEs for disease progression prediction.
problem Predicting disease progression under medications using limited data.
method Integrating expert-designed ODEs with machine-learned Neural ODEs.
result LHM consistently outperforms previous methods, especially in small sample regimes.
The paper explores solving inverse problems for ODEs with and without constraints.
problem Understanding when second order ODEs can represent Lagrangian models with or without constraints.
method Geometric techniques to address the inverse problem for both constrained and unconstrained systems of second order ODEs.
result The constrained case presents more ambiguities and complexities than the unconstrained one.
Statistical approach uses ODEs for modeling individual health trajectories.
problem Challenges in applying ODEs to longitudinal cohort data, especially noise and parameter sensitivity.
method Combines ODEs with neural networks to model individual health trajectories using each observation as initial value.
result Demonstrates improved modeling of individual health trajectories compared to global regression.
New method uses Gaussian ODE filtering to approximate likelihoods for fast ODE inverse problems.
problem Intractable forward models in likelihood-free inference, especially for ODEs.
method Gaussian ODE filtering to construct local Gaussian likelihood approximations.
result New solvers outperform standard likelihood-free approaches on benchmark systems.
HomoODE connects DEQs and Neural ODEs via homotopy continuation, improving accuracy and memory efficiency.
problem Connecting DEQs and Neural ODEs for better model performance and efficiency.
method Established a connection between DEQs and Neural ODEs using homotopy continuation, proposing HomoODE.
result HomoODE outperforms existing implicit models in accuracy and memory consumption.
Neural ODEs' performance varies with numerical method, requiring adaptive step size control.
problem Neural ODEs' performance depends on the numerical method used during training.
method Proposes an adaptive step size control algorithm to ensure a valid ODE without increasing computational cost.
result Valid Neural ODEs require careful numerical method selection and step size adaptation.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
This paper bridges the gap between ODE and SDE in diffusion models using Fokker-Planck equations.
problem Empirical evidence shows that ODE-based samples from score-based diffusion models are inferior to SDE-based samples.
method The paper rigorously describes dynamics and approximations in training score-based diffusion models, linking them to Fokker-Planck equations.
result Adding a regularisation term based on the Fokker-Planck residual can close the gap between ODE- and SDE-induced distributions.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
Neural ODEs simplified using Chen-Fliess series for Rademacher complexity analysis.
problem Analyzing the complexity of neural ODE models.
method Using Chen-Fliess series to frame neural ODEs as infinite-width nets, where weights are signature of control input and features are Lie derivatives.
result Derived compact expressions for the Rademacher complexity of ODE models.
Extends Neural ODEs to model discrete changes in continuous systems.
problem Lack of explicit termination time in existing Neural ODE formulations.
method Introduces neural event functions to implicitly define termination criteria.
result Models discrete changes in continuous systems without prior knowledge.
Neural Jump ODE improves continuous-time prediction and filtering of irregularly sampled time series.
problem Theoretical guarantees for continuous-time prediction and filtering of irregularly observed time series.
method Introducing Neural Jump ODE (NJ-ODE) that models conditional expectation between observations with neural ODEs and jumps.
result Theoretical guarantees for the L2-optimal prediction are provided, showing convergence of model output to optimal prediction. New method trains neural ODEs faster with fewer layers.
problem Training neural ODEs on large datasets is computationally expensive.
method Combines optimal transport and stability regularizations.
result Significant reductions in training time with no performance loss.
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
The paper improves the probability flow ODE sampler for faster sampling of natural images.
problem Improving the convergence rate of the probability flow ODE sampler.
method Adapting the probability flow ODE sampler to exploit intrinsic low-dimensional structures in natural image data.
result Achieves a dimension-free convergence rate of O(k/T) in total variation distance, improving upon existing results. Stochastic neural ODEs outperform deterministic ones on image classification tasks.
problem Improving generalization in continuous-time models like neural ODEs.
method Empirical study of stochastically regularized neural ODEs using SDEs.
result Data augmentation negates the benefits of stochastic regularization, making neural ODEs and SDEs nearly equivalent.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
Hypersolvers enable fast continuous-depth models for practical applications.
problem Infinite-depth models like Neural ODEs are computationally infeasible for large problems.
method Introducing hypersolvers, neural networks that solve ODEs efficiently with theoretical guarantees.
result Hypersolvers achieve comparable inference time to traditional discrete networks, making continuous-depth models practical.
A new method for estimating uncertainties in neural ODEs without numerical integration.
problem Accurate estimation of predictive uncertainties in neural ODEs.
method Distributional Gradient Matching (DGM) algorithm that jointly trains a smoother and a dynamics model.
result Significantly more accurate predictions compared to traditional methods.
Balanced Neural ODEs combine VAEs and Neural ODEs for efficient time series modeling.
problem Efficiently modeling systems with time-varying inputs and varying complexity.
method Combines VAEs for dimensionality reduction and Neural ODEs for dynamics, using variational parameters to adaptively learn.
result Balanced Neural ODEs (B-NODE) efficiently approximate Koopman operator without predefined dimensionality.
NP-ODE models FEA simulations with uncertainty, improving accuracy and efficiency.
problem Limitations of FEA in terms of computational cost and uncertainty quantification.
method Physics-informed neural process aided ordinary differential equations (NP-ODE).
result NP-ODE outperforms benchmark methods in uncertainty quantification and prediction accuracy.
A neural RNN model adapts time steps for non-stationary time series data.
problem Modeling and forecasting non-stationary time series with sharp changes.
method RNN-ODE-Adap model using neural ODE and adaptive time steps.
result Consistent estimation of intensity function for Hawkes-type data.
ODS improves adversarial attacks by maximizing output diversity.
problem Efficiency and effectiveness of adversarial attacks, especially black-box attacks.
method Output Diversified Sampling (ODS) that maximizes diversity in model outputs.
result ODS reduces the number of queries needed for black-box attacks on ImageNet by a factor of two.
ODENets are more robust to perturbations and adversarial attacks compared to CNNs.
problem Robustness of neural ODEs in the face of perturbations and adversarial attacks.
method Empirical study and theoretical analysis of ODENets' robustness properties.
result ODENets are more robust to random Gaussian perturbations and adversarial attacks compared to CNNs.
Rodent identifies ODEs from trajectories without needing basis functions.
problem Identifying the generating ODE from observed system trajectories.
method Uses Neural Arithmetic Units and sparsification techniques (VAE and ARD) to minimize state size and non-zero parameters.
result Learned models represent a manifold of ODEs including harmonic signals and Lotka-Volterra systems.