We present Ordinary Differential Equation Variational Auto-Encoder (ODEVAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODEVAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex conti…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Second Order Neural ODEs”
SONODEs and ANODEs improve learning of second order dynamics.
problem Learning dynamics governed by second order laws.
method Extended adjoint sensitivity method and theoretical analysis of ANODEs.
result SONODEs and ANODEs can learn higher order dynamics efficiently.
Minimal surfaces in third-order ODEs identified for linear second-order ODEs.
problem Characterizing minimal surfaces in third-order ODEs.
method Analyzing submanifolds of third-order ODEs as Riemannian manifolds.
result Linear second-order ODEs with are the only minimal surfaces and totally geodesic.
The paper explores solving inverse problems for ODEs with and without constraints.
problem Understanding when second order ODEs can represent Lagrangian models with or without constraints.
method Geometric techniques to address the inverse problem for both constrained and unconstrained systems of second order ODEs.
result The constrained case presents more ambiguities and complexities than the unconstrained one.
We apply the Cartan equivalence method to the study of real analytic second order ODEs under the local real analytic diffeomorphism of $\C^2$ which are area-preserving. This enables us to give a characterization of the second order ODEs which are equivalent to under such transformations. Moreover w…
It is demonstrated that point symmetry algebras of general analytic second order ODEs, not necessary of principal type, can have all dimensions between 0 and 8 except for 7. For the symmetry dimension 8 the ODE must be locally trivializable.
In the present paper we consider the problem of local equivalence of second order ODEs which are cubic in second derivative under the action of the pseudogroup of contact transformations. We show how it may be reduced to the equivalence problem of 2-webs in under the action of finite-dimensional group, a…
We show that the local equivalence problem for second-order ordinary differential equations under point transformations is completely characterized by differential invariants of order at most 10 and that this upper bound is sharp. We also show that, modulo Cartan duality and point transformations, the Painlevé-I equati…
We use the solution space of a pair of ODEs of at least second order to construct a smooth surface in Euclidean space. We describe when this surface is a proper embedding which is geodesically complete with finite total Gauss curvature. If the associated roots of the ODEs are real and distinct, we give a universal uppe…
The equivalence problem for second order ODEs given modulo point transformations is solved in full analogy with the equivalence problem of nondegenerate 3-dimensional CR structures. This approach enables an analog of the Feffereman metrics to be defined. The conformal class of these (split signature) metrics is well de…
We solve the local equivalence problem for second order (smooth or analytic) ordinary differential equations. We do so by presenting a {\em complete convergent normal form} for this class of ODEs. The normal form is optimal in the sense that it is defined up to the automorphism group of the model (flat) ODE . For…
This is an review on the point classification of second order ODE's by Ruslan Sharipov. His works were published in 1997-1998 at the Electronic Archive at LANL and undeservedly forgotten. Last chapter is an application of this classification to the investigation of Painleve equations.
These are lecture notes of the Summer school on the geometry of differential equations held in Nordfjordeid, Norway in 1996. They cover geometric structures related to scalar second order ODEs, the construction of the associated Cartan connection, techniques for computing invariants of differential equations starting f…
The class of second order ODE's cubic with respect to the first order derivative is considered. Using geometric structures associated with these equations, the subclasses of umbilical equations, zero mean curvature equations, and zero Gaussian curvature equations are defined. Zero mean curvature equations are studied w…
New method for linear connections in ODEs with constraints.
problem Constructing linear connections for ODEs with and without constraints.
method Novel method using submodule covariant derivatives.
result Closed form expressions for Massa-Pagani connection and its extension.
Soft-constrained PINN solves ODEs with minimal data, improving efficiency and robustness.
problem Sparse and noisy data in experiments and simulations.
method Soft-constrained Physics-informed Neural Network (PINN) with minimal labeled data.
result Soft-constrained PINN reduces need for labeled data and achieves strong generalization.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matchingstat.ML
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
The study derives generalization bounds for neural oscillators, improving their performance with regularization.
problem Quantifying the generalization capacities of neural oscillators.
method Using Rademacher complexity and squared Wasserstein-1 distances, the study derives theoretical upper PAC generalization bounds for neural oscillators.
result Theoretical bounds show polynomial growth in estimation errors with MLP size and time length, and regularization improves performance.
Investigates the relationship between ResNets and Neural ODEs, quantifying their closeness and providing training methods.
problem Quantifying the distance between ResNet dynamics and Neural ODE solutions.
method Bounding the distance between hidden state trajectories and Neural ODE solutions, using gradient descent and Heun's method.
result Gradient descent and Heun's method can implicitly regularize ResNets towards Neural ODEs, especially for smooth residual functions.
Paper proves higher-order flow matching preserves optimality in generative modeling.
problem Theoretical guarantees for higher-order flow matching in generative modeling.
method Neural network approximations with controlled depth, width, and sparsity.
result Proves worst case optimality for second-order flow matching.
This paper investigates the relationship between a system of differential equations and the underlying geometry associated with it. The geometry of a surface determines shortest paths, or geodesics connecting nearby points, which are defined as the solutions to a pair of second-order differential equations: the Euler-L…
The paper improves ODE solvers by integrating diverse information types.
problem Improving accuracy and physical meaningfulness of ODE solutions.
method Leveraging probabilistic solvers to include second-order information and physical conservation laws.
result Solutions become more accurate and physically meaningful with additional information.
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
Integrating Factors and ODE Patternsphysics.comp-ph
A systematic algorithm for building integrating factors of the form mu(x,y') or mu(y,y') for non-linear second order ODEs is presented. When such an integrating factor exists, the algorithm determines it without solving any differential equations. Examples of ODEs not having point symmetries are shown to be solvable us…
Study finds multiple periodic solutions to ODEs related to curvature problems.
problem Finding multiple positive periodic solutions to quasilinear ODEs.
method Global bifurcation techniques applied to second order quasilinear ODEs.
result Bifurcation-theoretic proof of nonuniqueness for conformal metrics with constant scalar curvature.
We study gradient-based optimization methods obtained by directly discretizing a second-order ordinary differential equation (ODE) related to the continuous limit of Nesterov's accelerated gradient method. When the function is smooth enough, we show that acceleration can be achieved by a stable discretization of this O…
Graph-Coupled Oscillator Networks (GraphCON) tackles graph-based learning problems.
problem The oversmoothing problem in Graph Neural Networks (GNNs).
method GraphCON is a novel framework based on discretizations of ODEs modeling oscillators coupled via graph adjacency.
result GraphCON mitigates the oversmoothing problem and exploding/vanishing gradients issues.
Paper improves neural ODEs for forecasting non-Markovian processes.
problem Forecasting irregularly observed time series with incomplete data.
method Path-dependent Neural Jump ODEs with signature transform.
result Path-dependent NJ-ODE outperforms original framework in non-Markovian data.
Optimizes curves on Riemannian manifolds to minimize curvature.
problem Minimizing curvature on curves with fixed length and endpoints on Riemannian manifolds.
method Solves a second order ODE system derived from the optimization problem.
result Solutions to the optimization problem satisfy a second order ODE system.
Study designs neural networks for fault localization, state estimation, and optimal PMU placement in power systems.
problem Fault localization, state estimation, and optimal PMU placement in power systems.
method Designs and compares various neural networks for fault localization, builds machine learning schemes for state estimation and parameter estimation, and designs an algorithm for optimal PMU placement.
result Comprehensive comparison of neural networks for fault localization shows that Graphical Convolutional NN and Neural Graph-based ODE perform best.
We provide five examples of conformal geometries which are naturally associated with ordinary differential equations (ODEs). The first example describes a one-to-one correspondence between the Wuenschmann class of 3rd order ODEs considered modulo contact transformations of variables and (local) 3-dimensional conformal …
We find the complete set of fundamental invariants for systems of ordinary differential equations of order under the group of point transformations generalizing similar results for contact invariants of a single ODE and point invariants of systems of the second and the third order. It turns out that starting fr…
In this work, we investigate the problem of finding surfaces in the Lorentz-Minkowski 3-space with prescribed skew () and mean () curvatures, which are defined through the discriminant of the characteristic polynomial of the shape operator and its trace, respectively. After showing that and can be interpr…
New symmetries found for scalar and vector ODEs of arbitrary dimensions.
problem Identifying symmetries for scalar and vector ODEs of arbitrary dimensions.
method Explicit expressions and abelian Lie algebra for non-Cartan symmetries in arbitrary dimensions.
result Non-Cartan symmetries characterize linearizable systems of ODEs but not nonlinear ones.
New optimality conditions for sub-Riemannian geodesics derived.
problem Optimality conditions for sub-Riemannian geodesics.
method Geometric translation and ODE derivation.
result New second-order necessary optimality conditions.
New method identifies physical constants from video data alone.
problem Identifying physical constants from video data.
method Proves level-set slope-coverage condition ensures local affine mapping to true physical state, enabling exact parameter recovery.
result Underdamped systems identifiable from a single video clip, other regimes require three diverse trajectories.
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightsstat.ML
We derive a second-order ordinary differential equation (ODE) which is the limit of Nesterov's accelerated gradient method. This ODE exhibits approximate equivalence to Nesterov's scheme and thus can serve as a tool for analysis. We show that the continuous time ODE allows for a better understanding of Nesterov's schem…
Balanced Neural ODEs combine VAEs and Neural ODEs for efficient time series modeling.
problem Efficiently modeling systems with time-varying inputs and varying complexity.
method Combines VAEs for dimensionality reduction and Neural ODEs for dynamics, using variational parameters to adaptively learn.
result Balanced Neural ODEs (B-NODE) efficiently approximate Koopman operator without predefined dimensionality.
New method calibrates complex ODEs from noisy data using neural networks.
problem Calibrating multi-dimensional complex ODEs from noisy data.
method Two-stage approach: de-noising and higher-order derivatives followed by a deep neural network.
result Consistent recovery of ODE system without curse of dimensionality.
Portable, Wearable and Wireless electrocardiogram (ECG) Systems have the potential to be used as point-of-care for cardiovascular disease diagnostic systems. Such wearable and wireless ECG systems require automatic detection of cardiovascular disease. Even in the primary care, automation of ECG diagnostic systems will …
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
We study complex analytic (possibly singular) projective connections on the plane. We characterize some of them in terms of their families of integral curves. We also give a beginning of classification of second order odes polynomial in the first and second derivatives, and with holomorphic coefficients.
Method studies equivalence of second order ODEs under specific transformations.
problem Classifying second order ODEs modulo fibre-preserving transformations.
method Using Moser's method of normal forms and Lie algebra computations.
result Normal forms can be used to prove fibre-preserving equivalence.
To understand the fundamental trade-offs between training stability, temporal dynamics and architectural complexity of recurrent neural networks~(RNNs), we directly analyze RNN architectures using numerical methods of ordinary differential equations~(ODEs). We define a general family of RNNs--the ODERNNs--by relating t…
ExNODE uses ODE to model sets with permutation equivariance.
problem Capturing intra-set dependencies in unordered sets.
method Exchangeable Neural ODE (ExNODE) using ODE.
result ExNODE achieves permutation equivariance for set modeling.
In the present paper we establish the necessary and sufficient conditions for two ordinary differential equations of the form to be equivalent under the action of the pseudogroup of contact transformations. These conditions are formulated in terms of integrals of some one-dimensional d…
New control theory shows neural networks can be sparsely active over time.
problem Optimizing neural networks for long-time control with sparsity constraints.
method Proving optimal controls vanish after a positive time and providing a stability estimate.
result Optimal controls for -penalized neural ODEs are sparsely active over time.