We present Ordinary Differential Equation Variational Auto-Encoder (ODEVAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODEVAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “Bayesian Neural ODEs”
Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Predictionstat.ML
Bayesian Neural ODEs improve vessel trajectory prediction with better uncertainty estimates.
problem Challenges in predicting vessel trajectories from irregular AIS data.
method Adopted a Gaussian process (GP) kernel-based prior on the vector field evaluated at measurement points, combined with probabilistic multiple shooting for long trajectories.
result Improved accuracy and uncertainty quantification in vessel trajectory predictions.
A new method combines ANN and Laplace for fast Bayesian inference in ODE models.
problem Bayesian inference for ODE systems with non-analytical solutions is computationally expensive.
method Hybrid approach using ANN for tractable likelihood and Laplace approximation.
result Effective posterior inference with improved computational cost compared to traditional methods.
Neural Ordinary Differential Equations (N-ODEs) are a powerful building block for learning systems, which extend residual networks to a continuous-time dynamical system. We propose a Bayesian version of N-ODEs that enables well-calibrated quantification of prediction uncertainty, while maintaining the expressive power …
Continuous-time MBRL framework tackles control systems with Bayesian ODEs.
problem Discretization of continuous-time systems in MBRL.
method Novel actor-critic method with Bayesian ODEs for state inference.
result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
Extracting Signal out of Chaos: Advancements on MAGI for Bayesian Analysis of Dynamical Systemsstat.CO
Novel methods improve Bayesian analysis of chaotic dynamical systems.
problem Bayesian parameter inference and trajectory reconstruction of chaotic systems with sparse and noisy data.
method Pilot MAGI (pMAGI) and Pilot MAGI Sequential Prediction (PMSP) methods.
result pMAGI and PMSP significantly outperform existing methods in accuracy and computational efficiency.
A new Fourier model improves ODE prediction.
problem Improving the accuracy of ODE solutions, especially for periodic functions.
method Constructing a Fourier state space model and a hybrid model combining Taylor and Fourier methods.
result The hybrid model can predict ODE solutions more accurately, especially for periodic functions.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
We present a particle flow realization of Bayes' rule, where an ODE-based neural operator is used to transport particles from a prior to its posterior after a new observation. We prove that such an ODE operator exists. Its neural parameterization can be trained in a meta-learning framework, allowing this operator to re…
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
New method uses neural ODEs to approximate complex distributions efficiently.
problem Approximating complex probability distributions efficiently.
method Neural ODEs with minimum energy regularization for distribution approximation.
result Deep neural network representations can achieve accurate distribution approximation.
Bayesian Gaussian Process ODEs enhanced with normalizing flows for improved flexibility and accuracy.
problem Limitations of standard Gaussian Process ODEs in modeling complex scenarios.
method Introducing normalizing flows to reparameterize the ODE vector field, developing a data-driven variational learning algorithm.
result Improved accuracy and uncertainty estimates for Bayesian Gaussian Process ODEs.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
New method combines ODE solvers with Bayesian inference for efficient model training.
problem Combining ODE solvers with Bayesian inference for efficient model training.
method Probabilistic state space model using extended Kalman filter for joint inference from differential equations and data.
result Efficient approximate Bayesian inference on latent force and ODE solution.
Augmented Neural ODEsstat.ML
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
There is resurging interest, in statistics and machine learning, in solvers for ordinary differential equations (ODEs) that return probability measures instead of point estimates. Recently, Conrad et al. introduced a sampling-based class of methods that are 'well-calibrated' in a specific sense. But the computational c…
This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
This paper explores normalization in neural ODEs, achieving high accuracy in CIFAR-10.
problem Understanding the role of normalization in neural ODEs.
method Investigated different normalization techniques and their impact on neural ODEs performance.
result Achieved 93% accuracy in CIFAR-10 classification task.
Faster training of neural ODEs using Gauß-Legendre quadrature.
problem Training neural ODEs is slow due to solving ODEs numerically.
method Use Gauß-Legendre quadrature to solve integrals faster than ODE-based methods.
result Faster training of neural ODEs, especially for large models.
Enhanced Neural ODEs outperform traditional models in image classification and video prediction.
problem Efficiently modeling time-varying dynamics in neural networks.
method Proposed a novel family of non-autonomous Neural ODEs with time-varying weights.
result Outperformed previous Neural ODE variants in speed and representational capacity.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
Nonlocal Bayesian modeling for continuous spatio-temporal dynamics
problem Handling irregular time points, sparse observations, and nonlocal interactions in spatio-temporal forecasting
method Hierarchical Bayesian framework with coordinate-based spatial basis expansion and continuous-time ODE
result Strong forecasting and uncertainty calibration
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Neural ODEs' performance varies with numerical method, requiring adaptive step size control.
problem Neural ODEs' performance depends on the numerical method used during training.
method Proposes an adaptive step size control algorithm to ensure a valid ODE without increasing computational cost.
result Valid Neural ODEs require careful numerical method selection and step size adaptation.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
Neural ordinary differential equations (ODEs) have been attracting increasing attention in various research domains recently. There have been some works studying optimization issues and approximation capabilities of neural ODEs, but their robustness is still yet unclear. In this work, we fill this important gap by expl…
Bayesian ANN method predicts chaotic systems with uncertainty.
problem Estimating chaotic dynamical systems from noisy data.
method Bayesian Artificial Neural Networks for ODE inverse problems.
result Accurate time predictions and uncertainty bounds.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
Extends Neural ODEs to model discrete changes in continuous systems.
problem Lack of explicit termination time in existing Neural ODE formulations.
method Introduces neural event functions to implicitly define termination criteria.
result Models discrete changes in continuous systems without prior knowledge.
Paper improves neural ODEs for forecasting non-Markovian processes.
problem Forecasting irregularly observed time series with incomplete data.
method Path-dependent Neural Jump ODEs with signature transform.
result Path-dependent NJ-ODE outperforms original framework in non-Markovian data.
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
HomoODE connects DEQs and Neural ODEs via homotopy continuation, improving accuracy and memory efficiency.
problem Connecting DEQs and Neural ODEs for better model performance and efficiency.
method Established a connection between DEQs and Neural ODEs using homotopy continuation, proposing HomoODE.
result HomoODE outperforms existing implicit models in accuracy and memory consumption.
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is…
The paper investigates how activation functions impact the training of Neural ODEs, leading to global convergence.
problem Challenges in training Neural ODEs, particularly gradient computation accuracy and convergence analysis.
method Investigates the impact of activation functions on the training dynamics of Neural ODEs.
result Establishes global convergence of Neural ODEs under gradient descent in overparameterized regimes.
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty b…
Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous inverti…
Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filteringstat.ML
Neural Jump ODE improves continuous-time prediction and filtering of irregularly sampled time series.
problem Theoretical guarantees for continuous-time prediction and filtering of irregularly observed time series.
method Introducing Neural Jump ODE (NJ-ODE) that models conditional expectation between observations with neural ODEs and jumps.
result Theoretical guarantees for the -optimal prediction are provided, showing convergence of model output to optimal prediction.
Deep residual networks implicitly converge to neural ODEs.
problem Link between discrete and continuous deep learning models.
method Establishing implicit regularization for residual networks towards neural ODEs.
result Deep residual networks initialized as discretizations of neural ODEs converge to such ODEs during training.
The paper develops a method to infer model parameters and shared dynamics from related physical systems using data.
problem Calibrating models to match data when detailed system properties and laws are unknown.
method Hierarchical Bayesian framework, adaptive surrogate models, bilevel optimization.
result Joint estimation of individual model parameters and shared dynamics using data from related systems.
Theory of learning of high-dimensional controlled non-linear dynamical systems (I): models and methodscond-mat.dis-nn
Neural ODEs provide a framework for studying the training dynamics of neural networks.
problem Training dynamics of neural networks
method Dynamical mean field theory
result Derive learning curves in the high-dimensional limit
We introduce a flexible, scalable Bayesian inference framework for nonlinear dynamical systems characterised by distinct and hierarchical variability at the individual, group, and population levels. Our model class is a generalisation of nonlinear mixed-effects (NLME) dynamical systems, the statistical workhorse for ma…
A new method for estimating uncertainties in neural ODEs without numerical integration.
problem Accurate estimation of predictive uncertainties in neural ODEs.
method Distributional Gradient Matching (DGM) algorithm that jointly trains a smoother and a dynamics model.
result Significantly more accurate predictions compared to traditional methods.
Bayesian framework discovers interpretable Lagrangian from data.
problem Discovering physical laws from limited data.
method Sparse Bayesian approach for learning interpretable Lagrangian.
result Automates Hamiltonian discovery from Lagrangian and provides ODE/PDE descriptions.
Balanced Neural ODEs combine VAEs and Neural ODEs for efficient time series modeling.
problem Efficiently modeling systems with time-varying inputs and varying complexity.
method Combines VAEs for dimensionality reduction and Neural ODEs for dynamics, using variational parameters to adaptively learn.
result Balanced Neural ODEs (B-NODE) efficiently approximate Koopman operator without predefined dimensionality.