DEMOTE uses neural diffusion-reaction processes to capture temporal dynamics in sparse tensor data.
problem Sparse and temporally associated tensor data with limited structural knowledge.
method Develops a neural diffusion-reaction process to estimate dynamic embeddings for tensor modes.
result Captures both commonalities and personalities in evolving tensor entries.
RandNet-Parareal uses neural networks to speed up time-parallel PDE solving.
problem Solving systems of time-dependent differential equations efficiently.
method Combines Parareal's sequential and parallel approach with random neural networks.
result Achieves up to 125x and 22x speedup compared to existing methods.
NKN deep neural network learns governing equations and classifies images.
problem Learning governing equations and classifying images with deep neural networks.
method Nonlocal kernel network (NKN) that is resolution independent, deep, and handles various tasks.
result NKN outperforms baseline methods in learning governing equations and image classification tasks.
VB-DeepONet uses Bayesian inference to improve DeepONet's predictions and uncertainty quantification.
problem Overfitting and lack of uncertainty quantification in DeepONet.
method Variational Bayes approach to approximate posterior distribution, reducing computational cost.
result VB-DeepONet alleviates DeepONet's limitations and provides uncertainty quantification.
We propose a minimal theory of non-linear price impact based on a linear (latent) order book approximation, inspired by diffusion-reaction models and general arguments. Our framework allows one to compute the average price trajectory in the presence of a meta-order, that consistently generalizes previously proposed pro…
Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when …
Modeling high-frequency speculative markets as auction search processes.
problem Understanding trading dynamics in high-frequency order-driven markets.
method Total order book model with diffusion-drift-reaction model, inspired by foraging and chemotaxis.
result Analytic and numerical analysis of trading performance in various search mechanisms.
Enhanced DeepONet framework with uncertainty quantification for complex operators.
problem Learning complex operators with uncertainty quantification.
method Generalised variational inference (GVI) using Rényi's α-divergence.
result Superior predictive accuracy and uncertainty quantification.
The Almost Hermitian Curvature flow was introduced by Streets and Tian in order to study almost hermitian structures, with a particular interest in symplectic structures. This flow is given by a diffusion-reaction equation. Hence it is natural to ask the following: which almost hermitian structures are dynamically stab…
New boundary treatment improves accuracy for complex PDEs.
problem Order reduction in high-order IMEX schemes for multidimensional PDEs.
method Novel boundary treatment algorithms for Cartesian meshes, treating implicit-explicit stages similarly to interior points.
result Recovery of designed order of convergence by numerical verification.
This paper presents a model order reduction (MOR) approach for high dimensional problems in the analysis of financial risk. To understand the financial risks and possible outcomes, we have to perform several thousand simulations of the underlying product. These simulations are expensive and create a need for efficient …
A model order reduction framework reduces financial risk analysis models efficiently.
problem Simulating high-dimensional financial risk models.
method Adaptive greedy sampling based on POD and surrogate modeling.
result Reduced models provide significant speedup with excellent accuracy.
Clarifies relation for solving control-affine Schrödinger bridge problems.
problem Solving control-affine Schrödinger bridge problems via Hopf-Cole transform.
method Applies Hopf-Cole transform to conditions of optimality, resulting in nonlinear PDEs.
result Generic control-affine Schrödinger bridge requires further algorithmic development.
A new model simulates non-linear adsorption using Gaussian KDEs.
problem Simulating non-linear adsorption processes in porous materials.
method Combines random walk particle tracking with Gaussian Kernel Density Estimators for nonlinear modeling.
result Effective reproduction of Langmuir and Freundlich isotherms.
This work presents a non-intrusive model reduction method to learn low-dimensional models of dynamical systems with non-polynomial nonlinear terms that are spatially local and that are given in analytic form. In contrast to state-of-the-art model reduction methods that are intrusive and thus require full knowledge of t…
We study a flow of G2 structures which induce the same Riemannian metric which is the negative gradient flow of an energy functional. We prove Shi-type estimates for the torsion tensor along the flow. We show that at a finite-time singularity the torsion must blow-up, so the flow exists as long as the torsion remain…
Improved DeepONet variants using Transformer cross-conditioning enhance PDE solution efficiency.
problem Solving partial differential equations efficiently and accurately.
method Transformer-inspired DeepONet variants with bidirectional cross-conditioning.
result Improved efficiency and accuracy compared to modified DeepONet, with variant effectiveness tied to PDE characteristics.
New guarantees for uniquely identifying transport maps and vector fields from finite measure-valued data.
problem Unique recovery of transport maps and vector fields from finite measure-valued data.
method Use of Whitney and Takens embedding theorems to establish conditions for unique identification.
result New metric for comparing diffeomorphisms and analogous results in infinitesimal settings.
M-CaStLe discovers causal structures in multivariate space-time data.
problem Challenges in causal graph discovery for high-dimensional gridded data.
method Generalizes CaStLe to multivariate analyses, using local embeddings and pooling spatial replicates.
result More accurately recovers multivariate causal structure and identifies physical dynamics.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Convolutional Neural Processes improve data efficiency in neural processes.
problem Improving data efficiency in neural processes for small datasets.
method Convolutional Neural Processes (ConvNPs) improve data efficiency by leveraging translation equivariance and convolutional neural networks.
result ConvNPs enhance the performance of neural processes in small-data problems.
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous ne…
Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
Investigates neural codes and their embeddings, proving conjectures and introducing new code types.
problem Analyzing neural codes and their embedding dimensions.
method Combinatorial, topological, and algebraic analysis; proving conjectures; introducing new neural code types.
result Proves conjectures about neural codes and their embeddings, introduces new code types.
Neural dynamical systems are dynamical systems that are described at least in part by neural networks. The class of continuous-time neural dynamical systems must, however, be numerically integrated for simulation and learning. Here, we present a compact neural circuit for two common numerical integrators: the explicit …
Quadratic models explain neural network behavior during training.
problem Understanding neural network dynamics during training with large learning rates.
method Developed and tested Neural Quadratic Models.
result Neural Quadratic Models exhibit the 'catapult phase' similar to neural networks.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
New metric compares noisy neural trajectories using optimal transport.
problem Existing metrics fail to capture differences in noisy, dynamic neural responses.
method Proposed an optimal transport distance metric for Gaussian processes.
result Metric effectively compares neural dynamics in different systems.
The neural tangent kernel equivalence theorem fails in practice.
problem Does the neural tangent kernel (NTK) equivalence theorem hold in practical neural network training?
method Rigorously derived NTK and conducted numerical experiments to evaluate the equivalence theorem.
result Adding a layer to a neural network and the corresponding updated NTK do not yield matching changes in predictor error.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Use simplified layerwise linear models to understand neural dynamics.
problem Complex neural network dynamics are hard to grasp.
method Apply simplified layerwise linear models to explain neural phenomena.
result Simplified models explain neural collapse, emergence, etc.
Analysis of over-parameterized neural networks has drawn significant attention in recentyears. It was shown that such systems behave like convex systems under various restrictedsettings, such as for two-level neural networks, and when learning is only restricted locally inthe so-called neural tangent kernel space aroun…
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
We introduce Graph Neural Processes (GNP), inspired by the recent work in conditional and latent neural processes. A Graph Neural Process is defined as a Conditional Neural Process that operates on arbitrary graph data. It takes features of sparsely observed context points as input, and outputs a distribution over targ…
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
Explains equivariant neural networks for machine learning.
problem Understanding equivariance in neural networks.
method Simple mathematical treatment of neural network concepts.
result Clarifies the mathematical basis of equivariant neural networks.
Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
Study of infinitely deep but narrow neural networks using NTK theory.
problem Analyzing the role of depth in deep learning with overparameterized networks.
method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.
result Established trainability guarantee for infinitely deep but narrow neural networks.
Q-learning with neural network function approximation (neural Q-learning for short) is among the most prevalent deep reinforcement learning algorithms. Despite its empirical success, the non-asymptotic convergence rate of neural Q-learning remains virtually unknown. In this paper, we present a finite-time analysis of a…
Paper introduces a noise-robust classification method using hypergraph neural networks.
problem Noisy label learning problem in image datasets.
method PCA for dimensionality reduction, then applies graph-based semi-supervised learning methods including hypergraph neural network.
result Our proposed hypergraph neural network achieves the best performance when noise level increases.
Paper introduces new neural network models and theories.
problem Understanding neural networks beyond over-parameterized regime.
method Develops two exact models and a novel representor theory.
result Provides insights into neural network training and kernel evolution.
Researchers analyze neural process architectures and their representational capacities.
problem Understanding what functions can be represented by different neural process architectures.
method Analyzing four types of neural process architectures: CNPs, ANPs, TNPs, and their latent variants.
result Prove these architectures form a strict hierarchy and characterize their representational capabilities.