Steerable neural ODEs on homogeneous spaces for equivariant feature dynamics.
problem Learning continuous-time equivariant dynamics of vector-valued features on homogeneous spaces.
method Introduces steerable neural ordinary differential equations on homogeneous spaces, interpreting features as sections of associated vector bundles over M. result Steerable NODEs are G-equivariant when the flow and connection are G-invariant, and they incorporate existing models. We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations.…
Neural controlled DEs model irregular time series by adjusting based on observations.
problem Modeling irregularly sampled multivariate time series with memory-efficient adjoint-based backpropagation.
method Neural controlled differential equations (CDEs) that adjust based on subsequent observations.
result Achieves state-of-the-art performance on various datasets.
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation functio…
Elvet solves differential equations and variational problems with neural networks.
problem Solving complex differential and variational equations with arbitrary conditions.
method Machine learning, specifically neural networks, to represent and solve equations.
result Elvet can solve a wide range of differential and variational problems.
Reconstructing signature features from randomized vector fields in differential equations.
problem Reconstructing signature features from controlled differential equations with random vector fields.
method Using controlled ordinary differential equations driven by continuous bounded variation curves, the study explores the extent to which signature features can be reconstructed from the non-linear flow of these equations.
result The number of signature features that can be reconstructed from the non-linear flow of controlled ordinary differential equations with random vector fields is exponential in the hidden dimension, under certain conditions.
ICODEN models survival data with interval-censored times using neural networks and ODEs.
problem Predicting time-to-event outcomes with interval-censored data, especially when models require strong assumptions or cannot handle high-dimensional predictors.
method ICODEN uses ordinary differential equations and deep neural networks to model the hazard function and cumulative hazard without proportional hazards assumption.
result ICODEN achieves satisfactory predictive accuracy across various simulation and real-world applications, handling high-dimensional predictors robustly.
NODEs can approximate a wide range of diffeomorphisms with strong guarantees.
problem The approximation power of NODEs under certain conditions.
method Leveraging a structure theorem of the diffeomorphism group.
result NODEs can approximate a large class of diffeomorphisms with a stronger guarantee.
NP-ODE models FEA simulations with uncertainty, improving accuracy and efficiency.
problem Limitations of FEA in terms of computational cost and uncertainty quantification.
method Physics-informed neural process aided ordinary differential equations (NP-ODE).
result NP-ODE outperforms benchmark methods in uncertainty quantification and prediction accuracy.
This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
We propose a neural network based approach for extracting models from dynamic data using ordinary and partial differential equations. In particular, given a time-series or spatio-temporal dataset, we seek to identify an accurate governing system which respects the intrinsic differential structure. The unknown governing…
New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
New sampling scheme improves ML accuracy in physics simulations.
problem Improving accuracy of ML models in physics simulations.
method Taylor-based data sampling scheme for DNNs.
result Reduces error in DNN solutions of ODE systems.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
Proposes a new framework to disentangle event influences in MTPP.
problem Underexplored how individual events influence overall dynamics over time.
method Decoupled MTPP framework using Neural Ordinary Differential Equations (Neural ODEs).
result Significantly improves performance on real-life datasets compared to state-of-the-art methods.
Proposes a method to model uncertainty in neural ordinary differential equations.
problem Lack of uncertainty modeling and robustness in neural ordinary differential equations.
method Introduces a novel approach to model uncertainty by considering a distribution over the end-time of the ODE solver.
result Demonstrates the effectiveness of the proposed approaches in modelling uncertainty and robustness through experiments.
SODEN uses neural networks and ODEs for scalable survival analysis.
problem Survival analysis with censored data and strong structural assumptions.
method Modeling survival distribution as an ODE, using adjoint sensitivity analysis for efficient optimization.
result Efficient estimation of survival models in large-scale applications.
PINNs struggle with increasingly complex ODEs, especially when parameters control their complexity.
problem Evaluating physics-informed neural networks on complex coupled ODEs.
method Tuned benchmarks of partial differential equations and harmonic oscillators; varying network architecture and training method.
result PINNs fail to solve complex ODEs, revealing issues like insufficient capacity, poor conditioning, and high local curvature.
DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEqua…
Continuous-depth Evoformer reduces protein folding prediction time and resource usage.
problem Efficient protein structure prediction with reduced computational costs.
method Continuous-depth formulation of Evoformer using Neural Ordinary Differential Equations (Neural ODEs).
result The continuous-time Evoformer achieves constant memory cost and improved efficiency.
The theory of Lie remarkable equations, i.e. differential equations characterized by their Lie point symmetries, is reviewed and applied to ordinary differential equations. In particular, we consider some relevant Lie algebras of vector fields on Rk and characterize Lie remarkable equations admitted by the …
MSLs use parallelizable root-finding for efficient ODE and PDE solutions.
problem Efficiently solving initial value problems for ODEs and PDEs.
method Leveraging time-parallel methods, MSLs use parallelizable root-finding algorithms.
result MSLs offer significant speedups in NFEs and inference time.
The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differen…
Neural Manifold ODEs improve manifold data modeling.
problem Adapting deep generative models to non-Euclidean spaces.
method Introducing Neural Manifold ODEs for manifold generalization and continuous probability computation.
result Improves density estimation and downstream tasks on arbitrary manifolds.
We use E. Cartan's method to solve the problem of equivalence of the second order ordinary differential equations with respect to the pseudogroup of point transformations.
We prove that a single-layer neural network trained with the Q-learning algorithm converges in distribution to a random ordinary differential equation as the size of the model and the number of training steps become large. Analysis of the limit differential equation shows that it has a unique stationary solution which …
The systems of complex analytic second order ordinary differential equations whose solutions close up to become rational curves (after analytic continuation) are characterized by the vanishing of an explicit differential invariant, and turn out to provide an infinite dimensional family of integrable systems.
Neural ODEs control graph dynamics with low energy feedback.
problem Controlling complex dynamical systems on graphs.
method Neural Ordinary Differential Equation Control (NODEC) framework.
result NODEC learns low-energy control signals for graph dynamical systems.
Neural differential equations combine deep learning and differential equations for modeling complex systems.
problem Modeling complex systems with high capacity and efficiency.
method Combining neural networks and differential equations, focusing on neural ordinary, controlled, and stochastic differential equations.
result NDEs offer high-capacity function approximation, strong priors, and handle irregular data efficiently.
Neural Laplace models diverse DEs in the Laplace domain for better dynamics.
problem Inadequate ODEs for long-range dependencies and discontinuities.
method Unified framework in Laplace domain, using stereographic map for smoothness.
result Superior performance in diverse DEs, including complex history dependency and abrupt changes.
To a system of second order ordinary differential equations (SODE) one can assign a canonical nonlinear connection that describes the geometry of the system. In this work we develop a geometric setting that allows us to assign a canonical nonlinear connection also to a system of higher order ordinary differential equat…
Paper solves a class of differential equations with specific solutions.
problem Identifying solutions to a class of nonlinear ODEs.
method Solves using a proposed side condition involving a third-order linear ODE.
result New closed and integral-form solutions for the Tzitzeica curve equation.
ANCDEs improve time-series forecasting and classification using attention in NCDEs.
problem Improving time-series forecasting and classification using neural controlled differential equations.
method Integrating attention into neural controlled differential equations (ANCDEs).
result ANCDEs consistently show the best accuracy in time-series classification and forecasting.
We show that classical Wilczynski--Se-ashi invariants of linear systems of ordinary differential equations are generalized in a natural way to contact invariants of non-linear ODEs. We explore geometric structures associated with equations that have vanishing generalized Wilczynski invariants and establish relationship…
ICON learns differential equation operators from prompts, reducing retraining and improving few-shot learning.
problem Training neural networks to solve differential equations without retraining for new problems.
method In-Context Operator Networks (ICON) that learns operators from prompted data and applies them to new problems.
result ICON can generalize to new operators beyond the training distribution and requires only a few demos.
Investigates the relationship between ResNets and Neural ODEs, quantifying their closeness and providing training methods.
problem Quantifying the distance between ResNet dynamics and Neural ODE solutions.
method Bounding the distance between hidden state trajectories and Neural ODE solutions, using gradient descent and Heun's method.
result Gradient descent and Heun's method can implicitly regularize ResNets towards Neural ODEs, especially for smooth residual functions.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation functio…
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
We shall study the equivalence problem for ordinary differential equations with respect to the affine transformations group.
DyNODE uses neural ODEs to model system dynamics in continuous control tasks.
problem Modeling the dynamics of systems in continuous control tasks.
method Neural Ordinary Differential Equations (ODEs) combined with actor-critic RL.
result DyNODE outperforms standard neural networks in sample efficiency and predictive performance.
Introduces geometric control theory for students.
problem No specific problem addressed in the abstract.
method Expository presentation of geometric control theory.
result Suitable for advanced students with solid math background.
ExNODE uses ODE to model sets with permutation equivariance.
problem Capturing intra-set dependencies in unordered sets.
method Exchangeable Neural ODE (ExNODE) using ODE.
result ExNODE achieves permutation equivariance for set modeling.
Novel method for solving ODEs on k-polysymplectic manifolds.
problem Solving ordinary differential equations on k-polysymplectic manifolds.
method k-polysymplectic energy-momentum method.
result Novel stability analysis techniques applied to Hamiltonian systems.
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs). Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabl…
A Gaussian Process Ordinary Differential Equation framework for large continuous dynamical systems
problem Forecasting complex dynamical systems
method Kernel autonomous ODE approach based on Gaussian Processes and Quadratic Order Model Reduction
result Full model outperforms ROM methods in terms of accuracy or computational costs
Deep ResNets exhibit distinct scaling properties with depth, challenging neural ODE models.
problem Understanding the scaling properties of deep ResNets and their relation to neural ODEs.
method Detailed numerical experiments on weights trained by stochastic gradient descent.
result Deep ResNets can exhibit different scaling regimes, including stochastic differential equations or neither, challenging the neural ODE model.
Paper classifies minimal graph transformations into new families of surfaces.
problem Classifying minimal graph transformations into new families of surfaces.
method Formulated and solved a coupled system of partial differential equations, reduced to solving an ordinary differential equation.
result Established rigorous equivalence to a modified problem for a harmonic function, yielding new families of minimal surfaces.