Study on Hitchin index for cohomogeneity one nearly Kähler structures.
problem Characterizing Hitchin index for cohomogeneity one nearly Kähler structures.
method Variational characterization, Morse-like index, cohomogeneity one symmetry, ODE eigenvalue problem.
result Obtained non-trivial lower bounds on Hitchin index for specific structure.
Exact risk and learning rate curves derived for adaptive SGD on high-dimensional problems.
problem Analyzing risk and learning rate dynamics in high-dimensional optimization problems.
method Developed a framework to give exact expressions for risk and learning rate curves using ODEs.
result Exact expressions for risk and learning rate curves, with detailed analysis of two adaptive learning rates.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs). We generalize RNNs to have continuous-time hidden dynamics defined by ordinary differential equations (ODEs), a model we call ODE-RNNs. Furthermore, we use ODE-RNNs to replace t…
Minimal surfaces in third-order ODEs identified for linear second-order ODEs.
problem Characterizing minimal surfaces in third-order ODEs.
method Analyzing submanifolds of third-order ODEs as Riemannian manifolds.
result Linear second-order ODEs with y′′=±y+β(x) are the only minimal surfaces and totally geodesic. DALTON improves ODE parameter estimation by learning from noisy data.
problem High sensitivity to parameters in ODEs produces unreliable parameter estimates.
method Data-adaptive probabilistic likelihood approximation for ODEs.
result DALTON produces more accurate parameter estimates than existing methods.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
Faster training of neural ODEs using Gauß-Legendre quadrature.
problem Training neural ODEs is slow due to solving ODEs numerically.
method Use Gauß-Legendre quadrature to solve integrals faster than ODE-based methods.
result Faster training of neural ODEs, especially for large models.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
Paper analyzes convergence of ODE samplers in Wasserstein distances.
problem Limited theoretical understanding of convergence properties of probability flow ODEs.
method Convergence analysis for general probability flow ODEs in 2-Wasserstein distance.
result First non-asymptotic convergence analysis for probability flow ODE samplers.
First-order ODEs linked to flat surfaces, leading to integrability.
problem Integrating first-order ODEs.
method Defined Riemannian metrics on variable spaces, studied surface properties, and established connections between Jacobi fields and Lie point symmetries.
result Flat associated surfaces lead to integrable first-order ODEs.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
A new Fourier model improves ODE prediction.
problem Improving the accuracy of ODE solutions, especially for periodic functions.
method Constructing a Fourier state space model and a hybrid model combining Taylor and Fourier methods.
result The hybrid model can predict ODE solutions more accurately, especially for periodic functions.
This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
We present Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
The study identifies exceptions to fiber-preserving symmetry in ODEs and systems.
problem Identifying exceptions to fiber-preserving symmetry in ODEs and systems.
method Lie's classification of Lie algebras of vector fields, absolute and relative scalar differential invariants, conditional and vector-valued relative invariants, prolongations of actions.
result Examples of scalar ODEs and systems with symmetry groups not fiber-preserving.
New symmetry dimensions for higher order ODEs are identified.
problem Determining the maximal and submaximal symmetry dimensions for higher order ODEs.
method Cartan-geometric approach to classify symmetry dimensions.
result Next largest realizable symmetry dimensions for scalar ODEs of order ≥ 4 and vector ODEs of order ≥ 3 are determined.
The aim of this paper is to construct a Riemann-Lagrange geometry on 1-jet spaces, in the sense of d-connections, d-torsions, d-curvatures, electromagnetic d-field and geometric electromagnetic Yang-Mills energy, starting from a given linear ODEs system or a given superior order ODE. The case of a non-homogenous linear…
New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
Enhanced Neural ODEs outperform traditional models in image classification and video prediction.
problem Efficiently modeling time-varying dynamics in neural networks.
method Proposed a novel family of non-autonomous Neural ODEs with time-varying weights.
result Outperformed previous Neural ODE variants in speed and representational capacity.
Paper improves neural ODEs for forecasting non-Markovian processes.
problem Forecasting irregularly observed time series with incomplete data.
method Path-dependent Neural Jump ODEs with signature transform.
result Path-dependent NJ-ODE outperforms original framework in non-Markovian data.
This paper explores normalization in neural ODEs, achieving high accuracy in CIFAR-10.
problem Understanding the role of normalization in neural ODEs.
method Investigated different normalization techniques and their impact on neural ODEs performance.
result Achieved 93% accuracy in CIFAR-10 classification task.
Derives PF-ODE for infinite-dimensional functions, improving function generation tasks.
problem Efficient inference in infinite-dimensional diffusion models.
method Derives PF-ODE in infinite-dimensional function spaces.
result Reduces function evaluations while maintaining sample quality.
Neural ODEs' performance varies with numerical method, requiring adaptive step size control.
problem Neural ODEs' performance depends on the numerical method used during training.
method Proposes an adaptive step size control algorithm to ensure a valid ODE without increasing computational cost.
result Valid Neural ODEs require careful numerical method selection and step size adaptation.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
New vector fields integrate first-order ODEs.
problem Integrating first-order ODEs.
method Relation between Riemannian manifolds and ODEs integration.
result Integration procedure for first-order ODEs.
We compute the characteristic Cartan connection associated with a system of third order ODEs. Our connection is different from Tanaka normal one, but still is uniquely associated with the system of third order ODEs. This allows us to find all fundamental invariants of a system of third order ODEs and, in particular, de…
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
ODE trajectories become abnormal curves in Carnot groups.
problem Understanding abnormal curves in Carnot groups.
method Explicit construction of covectors for abnormal curves.
result Polynomial ODE trajectories lift to abnormal curves in Carnot groups.
Generative ODE model learns unknown variables in medical systems.
problem Estimating unknown variables in complex medical systems.
method Variational autoencoder incorporating known ODE functions.
result Modeling known-unknowns improves system parameter discovery and extrapolation.
Solutions of an implicit ODE form a web. Already for cubic ODEs the 3-web of solutions has a nontrivial local invariant, namely the curvature form. Thus any local classification of implicit ODEs necessarily has functional moduli if no restriction on the class of ODEs is imposed. Here the most symmetric case of hexagona…
A systematic algorithm for building integrating factors of the form mu(x,y') or mu(y,y') for non-linear second order ODEs is presented. When such an integrating factor exists, the algorithm determines it without solving any differential equations. Examples of ODEs not having point symmetries are shown to be solvable us…
The paper explores solving inverse problems for ODEs with and without constraints.
problem Understanding when second order ODEs can represent Lagrangian models with or without constraints.
method Geometric techniques to address the inverse problem for both constrained and unconstrained systems of second order ODEs.
result The constrained case presents more ambiguities and complexities than the unconstrained one.
Parallel-in-time solver reduces ODE simulation time from linear to logarithmic.
problem Efficiently solving ordinary differential equations (ODEs) with reduced computational cost.
method Formulated a parallel-in-time probabilistic numerical ODE solver using time-parallel formulation of iterated extended Kalman smoothers.
result Reduces span cost from linear to logarithmic in the number of time steps.
Signature tensors uniquely identify ODE solutions.
problem Identifying ODE solutions from signature tensors.
method Geometric theory of nonlinear systems of ODEs.
result Necessary and sufficient algebraic conditions for signature tensors to represent ODE solutions.
LHM integrates expert ODEs with neural ODEs for disease progression prediction.
problem Predicting disease progression under medications using limited data.
method Integrating expert-designed ODEs with machine-learned Neural ODEs.
result LHM consistently outperforms previous methods, especially in small sample regimes.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
Bayesian ODEs with Gaussian processes infer unknown dynamics from data.
problem Estimating unknown continuous-time system dynamics from data.
method Bayesian nonparametric model using Gaussian processes, sparse variational inference, probabilistic shooting.
result Posterior predictive uncertainty scores outperform alternative methods on multiple ODE learning tasks.
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is…
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
We describe 4th order ODEs satisfying two contact invariant conditions of Bryant in terms of the Ricci tensor of a certain gl(2,R) valued connection. We also provide nonhomogeneous examples of such ODEs.
New method calibrates complex ODEs from noisy data using neural networks.
problem Calibrating multi-dimensional complex ODEs from noisy data.
method Two-stage approach: de-noising and higher-order derivatives followed by a deep neural network.
result Consistent recovery of ODE system without curse of dimensionality.
It is demonstrated that point symmetry algebras of general analytic second order ODEs, not necessary of principal type, can have all dimensions between 0 and 8 except for 7. For the symmetry dimension 8 the ODE must be locally trivializable.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
Efficiently solves high-dimensional ODEs with probabilistic methods.
problem Solving high-dimensional ODEs with uncertainty quantification.
method Probabilistic numerical algorithm based on independence assumptions or Kronecker structure.
result Efficient probabilistic solutions for ODEs with millions of dimensions.
Neural ordinary differential equations (ODEs) have been attracting increasing attention in various research domains recently. There have been some works studying optimization issues and approximation capabilities of neural ODEs, but their robustness is still yet unclear. In this work, we fill this important gap by expl…
Neural ODEs simplified using Chen-Fliess series for Rademacher complexity analysis.
problem Analyzing the complexity of neural ODE models.
method Using Chen-Fliess series to frame neural ODEs as infinite-width nets, where weights are signature of control input and features are Lie derivatives.
result Derived compact expressions for the Rademacher complexity of ODE models.