Neural networks can approximate complex stochastic equations well.
problem Approximating general stochastic differential equations.
method Identified neural network classes approximating continuous functions.
result Neural stochastic differential equations can approximate general stochastic differential equations arbitrarily well.
DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEqua…
Neural differential equations combine deep learning and differential equations for modeling complex systems.
problem Modeling complex systems with high capacity and efficiency.
method Combining neural networks and differential equations, focusing on neural ordinary, controlled, and stochastic differential equations.
result NDEs offer high-capacity function approximation, strong priors, and handle irregular data efficiently.
New method solves PDEs for any initial condition without retraining.
problem Solving PDEs for different initial conditions requires retraining neural solvers.
method Formulate solution as conditional probability distribution.
result Approximates PDE solution for arbitrary initial conditions.
Stabilized neural differential equations enforce constraints on dynamical systems.
problem Ensuring dynamical systems preserve known constraints like conservation laws.
method SNDEs with a stabilization term to enforce manifold constraints.
result SNDEs outperform existing methods and broaden constraint types.
New framework explains neural network bias in solving differential equations.
problem Understanding and controlling the bias in PINNs for differential equations.
method Deriving an integro-differential equation from PINNs and GPR equivalence.
result PINN predictions are influenced by a kernel term reflecting architecture choices.
Elvet solves differential equations and variational problems with neural networks.
problem Solving complex differential and variational equations with arbitrary conditions.
method Machine learning, specifically neural networks, to represent and solve equations.
result Elvet can solve a wide range of differential and variational problems.
This work integrates differentiation and integration in Physics-Informed Neural Networks.
problem Solving integro-differential equations and computing integral transforms.
method Augmenting Physics-Informed Neural Networks with automatic integration.
result Solving complex integral transforms and integro-differential equations.
Neural DEs improve single image super-resolution.
problem Challenging tasks in image super-resolution.
method Applied Neural Differential Equations to image super-resolution, using variational methods and backpropagation.
result Differential models match state-of-the-art performance.
Neural Laplace models diverse DEs in the Laplace domain for better dynamics.
problem Inadequate ODEs for long-range dependencies and discontinuities.
method Unified framework in Laplace domain, using stereographic map for smoothness.
result Superior performance in diverse DEs, including complex history dependency and abrupt changes.
Deep neural nets solve complex insurance math equations.
problem Optimal control problems in insurance math.
method Deep neural network algorithm for elliptic PDEs.
result Solves high-dimensional semilinear elliptic PDEs.
Study deep neural nets for solving complex insurance equations.
problem Solving linear and semilinear parabolic PIDEs in high dimensions.
method Deep neural network algorithms for integro-differential equations.
result Viability of deep learning for solving high-dimensional PIDEs.
ANCDEs improve time-series forecasting and classification using attention in NCDEs.
problem Improving time-series forecasting and classification using neural controlled differential equations.
method Integrating attention into neural controlled differential equations (ANCDEs).
result ANCDEs consistently show the best accuracy in time-series classification and forecasting.
Neural networks solve SPDEs using Wiener chaos expansion.
problem Solving stochastic partial differential equations (SPDEs) numerically.
method Using neural networks in the truncated Wiener chaos expansion.
result Approximation rates for learning SPDE solutions with noise.
The paper examines the neural tangent kernel for PINNs solving general PDEs and finds convergence conditions.
problem Analyzing the convergence of neural tangent kernel for PINNs solving general PDEs.
method Analysis of NTK initialization and convergence during training for general PDEs using PINNs.
result Homogeneity of differential operators is crucial for NTK convergence.
Physics-informed neural networks are developed to characterize the state of dynamical systems in a random environment. The neural network approximates the probability density function (pdf) or the characteristic function (chf) of the state of these systems which satisfy the Fokker-Planck equation or an integro-differen…
Neural controlled DEs model irregular time series by adjusting based on observations.
problem Modeling irregularly sampled multivariate time series with memory-efficient adjoint-based backpropagation.
method Neural controlled differential equations (CDEs) that adjust based on subsequent observations.
result Achieves state-of-the-art performance on various datasets.
PNDEs project neural dynamics onto constraint manifolds, improving accuracy and stability.
problem Learning dynamics from data without violating known constraints.
method Projecting the learned vector field onto the tangent space of the constraint manifold.
result PNDEs outperform existing methods in learning constrained dynamical systems.
ICON learns differential equation operators from prompts, reducing retraining and improving few-shot learning.
problem Training neural networks to solve differential equations without retraining for new problems.
method In-Context Operator Networks (ICON) that learns operators from prompted data and applies them to new problems.
result ICON can generalize to new operators beyond the training distribution and requires only a few demos.
DEQGAN uses GANs to solve differential equations without supervision.
problem Solving differential equations with neural networks.
method Generative Adversarial Networks (GANs) to learn the loss function.
result DEQGAN achieves lower mean squared errors and competitive solution accuracy compared to traditional methods.
Neural networks model financial data with Lévy processes.
problem Forecasting chaotic financial time series with big jumps.
method Lévy-induced stochastic differential equation network approximated by neural networks.
result The method improves prediction accuracy using non-Gaussian Lévy processes.
Neural GDEs improve graph prediction by blending discrete structures and differential equations.
problem Dynamic graph prediction challenges in irregularly sampled data.
method Continuous-depth graph neural networks (GNNs) with Neural GDEs.
result Neural GDEs enhance performance across various applications.
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
Proposes a method to train neural networks that solve differential equations faster.
problem Training neural networks that solve differential equations becomes computationally expensive.
method Introduces a differentiable surrogate for numerical solver time cost using higher-order derivatives.
result Trains models that are faster to solve while maintaining nearly the same accuracy.
MSLs use parallelizable root-finding for efficient ODE and PDE solutions.
problem Efficiently solving initial value problems for ODEs and PDEs.
method Leveraging time-parallel methods, MSLs use parallelizable root-finding algorithms.
result MSLs offer significant speedups in NFEs and inference time.
Graph neural controlled differential equations learn graph dynamics from vertex observations.
problem Predicting future states of dynamical systems on graphs with limited vertex data.
method Incorporates graph topology information into NCDE to predict graph dynamics.
result Informed NCDE requires fewer parameters and lower MAE compared to previous methods.
Paper discovers differential equations from data using neural networks and Bayesian methods.
problem Discovering differential equations from datasets using machine learning.
method Integrates neural network-based surrogates with Sparse Bayesian Learning (SBL).
result Proposes a robust model discovery algorithm and a Physics Informed Normalizing Flow (PINF).
Classical numerical methods for solving partial differential equations suffer from the curse dimensionality mainly due to their reliance on meticulously generated spatio-temporal grids. Inspired by modern deep learning based techniques for solving forward and inverse problems associated with partial differential equati…
FDNet learns PDEs from data with fast predictions.
problem Discovering complex systems behavior from data.
method Finite difference neural networks (FDNet) to learn PDEs from trajectory data.
result FDNet predicts future behavior with few trainable parameters.
Can neural networks learn to solve partial differential equations (PDEs)? We investigate this question for two (systems of) PDEs, namely, the Poisson equation and the steady Navier--Stokes equations. The contributions of this paper are five-fold. (1) Numerical experiments show that small neural networks (< 500 learnabl…
In this work we systematically analyze general properties of differential equations used as machine learning models. We demonstrate that the gradient of the loss function with respect to to the hidden state can be considered as a generalized momentum conjugate to the hidden state, allowing application of the tools of c…
Two neural network methods solve the master equation for MFGs.
problem Approximating Nash equilibria in stochastic, finite-agent games.
method Backward induction and direct PDE tackling neural networks.
result Neural networks can approximate the master equation's solution.
FBSJNN solves PIDEs and FBSDEJs with deep learning, offering theoretical and numerical efficiency.
problem Solving Partial Integro-Differential Equations and Forward-Backward Stochastic Differential Equations with Jumps.
method FBSJNN framework using a single neural network for both solution approximation and non-local integral.
result FBSJNN achieves numerical solutions with a relative error of 10−3, demonstrating efficiency. New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations.…
Recently, a lot of papers proposed to use neural networks to approximately solve partial differential equations (PDEs). Yet, there has been a lack of flexible framework for convenient experimentation. In an attempt to fill the gap, we introduce a PyDEns-module open-sourced on GitHub. Coupled with capabilities of BatchF…
Proposes SDE framework for uncertainty quantification in graph neural networks.
problem Lack of uncertainty quantification in graph neural networks.
method Introduces Latent Graph Neural Stochastic Differential Equations (LGNSDE) with Bayesian prior-posterior mechanism and Brownian motion.
result LGNSDEs provide theoretically sensible guarantees for uncertainty estimates and are robust to perturbations.
We prove that a single-layer neural network trained with the Q-learning algorithm converges in distribution to a random ordinary differential equation as the size of the model and the number of training steps become large. Analysis of the limit differential equation shows that it has a unique stationary solution which …
New method uses PINNs to efficiently compute Gerber-Shiu functions.
problem Calculating the Gerber-Shiu function efficiently.
method Physics-informed neural networks (PINNs) embedded with differential equations.
result Demonstrates good performance in approximating Gerber-Shiu functions.
Neural networks improve predictions of complex network dynamics.
problem Improving neural network predictions for complex network dynamics.
method Extended neural network models to complex systems, ensuring they conform to dynamical model assumptions and using a statistical significance test.
result Achieved advanced generalization of neural network predictions for complex systems.
Paper develops a consistent algorithm for learning graph structure from continuous-time stochastic differential equations.
problem Learning structure from continuous-time stochastic differential equations.
method Score-based structure learning using Neural Ordinary Differential Equations with adaptive regularization.
result The method consistently recovers directed graphs of local independencies in systems of stochastic differential equations.
Deep ResNets exhibit distinct scaling properties with depth, challenging neural ODE models.
problem Understanding the scaling properties of deep ResNets and their relation to neural ODEs.
method Detailed numerical experiments on weights trained by stochastic gradient descent.
result Deep ResNets can exhibit different scaling regimes, including stochastic differential equations or neither, challenging the neural ODE model.
We introduce physics informed neural networks -- neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this two part treatise, we present our developments in the context of solving two main classes …
PDGM uses neural nets to solve complex financial equations.
problem Solving path-dependent partial differential equations (PPDEs)
method Generalized Deep Galerkin Method (PDGM) combining feed-forward and LSTM architectures
result PDGM successfully models solutions to various PPDEs, including financial derivatives.
Steerable neural ODEs on homogeneous spaces for equivariant feature dynamics.
problem Learning continuous-time equivariant dynamics of vector-valued features on homogeneous spaces.
method Introduces steerable neural ordinary differential equations on homogeneous spaces, interpreting features as sections of associated vector bundles over M. result Steerable NODEs are G-equivariant when the flow and connection are G-invariant, and they incorporate existing models. This paper uses ODE to improve RNN models for time series data.
problem Improving RNN models for irregularly sampled time series data.
method Extending RNNs with Neural Ordinary Differential Equations (ODEs).
result New ODE-based RNN models reduce training and evaluation time.
Discover equations from data using neural networks with constraints.
problem Discover equations from noisy data without theoretical derivation.
method Solve constrained optimization problem with penalty or trust-region barrier methods.
result Constrained method outperforms penalty method for higher noise levels or fewer collocation points.
Gradients of neural networks can be computed efficiently for any architecture, but some applications require differential operators with higher time complexity. We describe a family of restricted neural network architectures that allow efficient computation of a family of differential operators involving dimension-wise…