Theory explains why neural nets better learn Calabi-Yau metrics.
problem Learning Calabi-Yau metrics with neural networks.
method Developed a theory of metric flows in neural network space.
result Finite-width neural networks learn Calabi-Yau metrics better than fixed kernel methods.
Graph neural network using Beltrami flow for feature and topology evolution.
problem Efficient feature learning and topology evolution on graphs.
method Discretized Beltrami flow applied to graph neural networks with positional encodings.
result Achieves state-of-the-art results on various benchmarks.
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
Study suggests using information flow measures to target interventions in neural networks.
problem Identifying neural network edges that can be pruned to reduce bias.
method Used M-information flow framework to measure and compare information flows about true labels and protected attributes, and evaluated pruning effects on bias reduction. result Pruning edges with larger information flows about protected attributes reduces bias at the output.
Graph neural networks are explained through energy gradient flow and framelet decomposition.
problem Understanding and improving graph neural networks.
method Viewing framelet-based models as gradient flows of energy, proposing a generalized energy via framelet decomposition.
result The proposed model leads to more flexible dynamics, enhancing graph neural networks.
Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their fl…
Neural networks' feature geometry evolves like discrete Ricci flow.
problem Understanding neural feature representations and their geometric transformations.
method Approximating input manifold with geometric graphs and analyzing their evolution during training.
result Neural feature geometry evolves like discrete Ricci flow, with nonlinear activations playing a crucial role.
The paper analyzes neural network dynamics after weights escape the origin.
problem Understanding gradient flow dynamics of neural networks after the origin.
method Analyzes gradient flow of homogeneous neural networks with locally Lipschitz gradients.
result Characterizes the first saddle point encountered after escaping the origin.
Efficiently quantifies uncertainty in subsurface flow using neural networks guided by theory.
problem Uncertainty in dynamic subsurface flow predictions.
method Theory-guided Neural Network (TgNN) for efficient uncertainty quantification.
result TgNN surrogate improves efficiency of uncertainty quantification compared to MC method.
New paradigm for Neural ODEs stabilizes training and improves model performance.
problem Gradient vanishing-explosion problem in training deep neural networks.
method ODEtoODE: Nested system of flows with orthogonal group constraints.
result Strong convergence results and improved downstream models in reinforcement learning and supervised learning.
Neural networks predict traffic flow in smart cities.
problem Forecasting stochastic and nonlinear traffic flow.
method Various recurrent neural networks trained on intersection data.
result Vector output model with gated recurrent units performed best.
Neural Manifold ODEs improve manifold data modeling.
problem Adapting deep generative models to non-Euclidean spaces.
method Introducing Neural Manifold ODEs for manifold generalization and continuous probability computation.
result Improves density estimation and downstream tasks on arbitrary manifolds.
New method uses neural nets in Hilbert space for option pricing on flow forwards.
problem Pricing options on flow forwards with neural networks in Hilbert space.
method Optimization problem in Hilbert space solved by a novel feedforward neural network architecture.
result Excellent numerical efficiency and superior performance over classical methods.
JKO-iFlow uses neural ODEs to improve generative models with reduced memory and training complexity.
problem Efficiently training deep generative models in high dimensions with reduced memory and training complexity.
method JKO scheme inspired neural ODE flow network with adaptive time reparameterization.
result JKO-iFlow achieves competitive performance compared to existing models at reduced computational and memory cost.
Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.
problem Analyzing implicit bias in shallow neural networks with mirror flow.
method Characterization through variational problems and scaled potentials.
result Mirror flow with scaled potentials induces a rich class of biases not captured by RKHS norms.
A neural flow method minimizes Willmore energy for 2-surfaces in 3D space.
problem Minimizing Willmore energy for closed oriented 2-surfaces in 3D space.
method Introducing neural Willmore flow to model and minimize the Willmore energy using neural architectures.
result The neural flow reproduces expected round sphere and Clifford torus for genus 0 and 1 surfaces, respectively, and finds minimal Willmore surfaces for genus 2.
Neural ODEs extended to manifolds for flexible sampling.
problem Sampling from complex multimodal distributions on non-trivial topologies.
method Extending Neural ODEs to smooth manifolds using vector fields.
result A general methodology for building normalizing flows on manifolds.
Neural execution solves complex graph problems like bipartite matching.
problem Solving complex graph algorithms like maximum bipartite matching.
method Reduces bipartite matching to a flow problem and uses Ford-Fulkerson for maximum flow.
result Neural network achieves optimal matching almost 100% of the time.
A new method computes high-dimensional optimal transport using flow neural networks.
problem Computing optimal transport for high-dimensional data.
method Optimizing a flow model to minimize transport cost between two arbitrary distributions.
result Trained optimal transport flow enables downstream tasks like DRE and domain adaptation.
Flowification enriches neural networks with an inverse pass and likelihood monitoring.
problem Neural networks lack an inverse pass and likelihood monitoring, limiting their generative capabilities.
method Introduce flowification, enriching neural networks with a stochastic inverse pass and likelihood monitoring.
result Certain neural network architectures can be enriched to fall under the generalized notion of a normalizing flow.
The study examines how shallow neural nets converge to training samples or manifold points during diffusion.
problem Understanding when and how shallow neural nets converge to training samples or manifold points during diffusion.
method Analysis of shallow ReLU neural network denoisers trained with minimal ℓ2 norm, comparing score flow and diffusion flow. result Probability flow converges to training points, sums of training points, or manifold points, depending on the diffusion time scheduler.
Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF). We unify a…
NanoFlow reduces parameter complexity in normalizing flows.
problem Efficient parameter complexity in flow-based models.
method Single neural density estimator with flow indication embedding.
result Sublinear parameter complexity achieved.
DRIFT uses neural flows to replace distributional regression models.
problem Lack of neural network representations for distributional regression models.
method Inverse flow transformations (DRIFT) for distributional regression.
result Neural representations in DRIFT match classical statistical methods in performance.
Gradient flows of neural networks converge to optimal values or diverge, with thresholds and asymptotic behaviors.
problem Understanding the convergence and divergence of gradient flows in neural networks.
method Analysis of gradient flows on loss landscapes of neural networks using o-minimal structures.
result Gradient flows either converge to optimal values or diverge to infinity, with thresholds and asymptotic behaviors.
Predicts short-term futures contract direction using neural networks and order flow data.
problem Challenges in predicting short-term directional movement of futures contracts.
method Engineering features from technical analysis, order flow, and order-book data; training a Tabnet neural network.
result Achieved an accuracy of 0.601 in predicting directional change on the Silver Futures Contract.
Graph Neural Networks model 3D granular flow simulations.
problem Accurate modeling of complex 3D granular flow processes.
method Graph Neural Networks approach to simulate 3D granular flow using LIGGGHTS.
result Machine learning trajectories match physical granular flow processes.
Kernelised flows improve density estimation and generation with fewer parameters.
problem Limited expressiveness of flow-based models due to invertibility constraints.
method Integrates kernels into normalising flows to enhance expressiveness and efficiency.
result Kernelised flows outperform neural network-based flows in parameter efficiency and low-data scenarios.
OT-Flow uses optimal transport to improve CNFs for faster and more accurate density estimation.
problem Computational challenges in continuous normalizing flows.
method OT-Flow leverages optimal transport to regularize CNFs and uses exact trace computation.
result OT-Flow achieves competitive performance with one-fourth the number of weights and significant speedups.
HFNO enhances interpretability of turbulent flows through parallel wavenumber bin processing.
problem Opaque inner workings of Fourier Neural Operators (FNOs) hinder physical interpretability.
method Introduces HFNO, a novel FNO-based architecture that processes wavenumber bins in parallel, enhancing interpretability.
result HFNO decomposes turbulent flows across various scales, enabling increased interpretability and multiscale modeling.
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables. We propose a new type of normalizing flow, inverse autoregressive flow (IAF), that, in contrast to earlier published flows, scales well to high-dimensional latent spaces. The proposed f…
The Normalizing Flow (NF) models a general probability density by estimating an invertible transformation applied on samples drawn from a known distribution. We introduce a new type of NF, called Deep Diffeomorphic Normalizing Flow (DDNF). A diffeomorphic flow is an invertible function where both the function and its i…
Study quantifies information flow in neural networks using relative entropy and RG analogy.
problem Quantifying information flow in deep neural networks.
method Explicit computation of relative entropy in Ising models and feedforward neural networks.
result Monotonic increase of relative entropy to an asymptotic value, confirming connection to c-theorem.
New method trains any neural network as a generative model.
problem Constrained design of normalizing flows due to analytical invertibility.
method Efficient gradient estimator for non-analytically invertible networks.
result Any dimension-preserving neural network can be used as a generative model.
Neural networks predict flow and elastic stresses in viscoelastic turbulence.
problem Predicting flow and elastic stresses in viscoelastic turbulent flows using limited experimental data.
method Convolutional neural networks trained on wall-normal velocity and pressure data.
result Neural networks accurately predict flow and elastic stresses, especially during low-drag events.
Improved neural ODEs learn adaptable flows.
problem Neural ODEs struggle with expressive power and adaptability.
method Introduce N-CODE modules with dynamic parameters controlled by a trainable map.
result N-CODE modules enhance expressivity of neural ODEs.
New method learns PDE solutions from low-fidelity data.
problem Challenges in learning PDE surrogates with scarce data.
method Flow matching in infinite-dimensional space with conditional neural operators.
result Accurately learns PDE solutions across different resolutions and fidelities.
The paper proposes a method to sample quantum field configurations using neural operators and flows.
problem Sampling lattice field configurations from Boltzmann distributions in quantum field theories.
method Approximating a time-dependent neural operator to map between free and target theories, discretizing to a normalizing flow, and training to diffeomorphism.
result The method can generalize to larger lattice sizes when pre-trained on smaller ones, improving efficiency.
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual information allows us to create an information measure that quantifies how much in…
TFM trains Neural SDEs without backpropagation, improving clinical time series modeling.
problem Modeling irregularly sampled time series in medicine.
method Trajectory Flow Matching (TFM) using flow matching for generative modeling.
result TFM improves performance on clinical time series datasets.
A new method for learning conditional distributions using ODEs and neural networks.
problem Learning conditional distributions efficiently and accurately.
method Conditional Föllmer Flow, discretized with Euler's method, using nonparametric velocity estimation.
result Effective approximation of target conditional distributions, with convergence results for Wasserstein-2 distance.
Study shows overparameterization helps shallow neural networks recover signals in high dimensions.
problem Signal recovery in shallow neural networks with overparameterization.
method Gradient flow on population risk, Gaussian distribution assumption, high-dimensional limit analysis.
result Minimal overparameterization is sufficient for strong recovery of signals.
This study explains gradient flow dynamics in neural networks for small initialisation.
problem Understanding the training dynamics of neural networks for small initialisation.
method Analysis of gradient flow dynamics for one-hidden layer ReLU networks with orthogonal inputs.
result Gradient flow converges to zero loss and characterizes implicit bias towards minimum variation norm.
New tensor formulation reveals gradient flow's bias in linear neural networks.
problem Understanding implicit bias in linear neural network training.
method Tensor formulation of neural networks, including fully-connected, diagonal, and convolutional networks.
result Gradient flow on linear tensor networks converges to solutions of specific optimization problems.
Unified geometric flows improve deep learning efficiency and simplify neural network topologies.
problem Improving deep learning performance and simplifying neural network structures.
method Proposes a thermodynamically coupled Ricci flow that dynamically adapts parameter space geometry to loss landscape topology, enabling automated singularity resolution and providing entanglement entropy bounds.
result Demonstrates 2.1× convergence acceleration and 63% topological simplification while maintaining O(NlogN) complexity, outperforming Riemannian baselines by 15.2% in few-shot accuracy. This paper proposes a deep neural network approach for predicting multiphase flow in heterogeneous domains with high computational efficiency. The deep neural network model is able to handle permeability heterogeneity in high dimensional systems, and can learn the interplay of viscous, gravity, and capillary forces fro…
A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choic…
New method uses RBM flows to find critical temperatures in Ising models.
problem Detecting critical temperatures in RBM flows without model topology information.
method Iterative sampling from RBM mapped on Ising model temperature space using a neural network thermometer.
result Flow of RBM trained on Ising spin configurations approaches critical temperature around kBTc/J≈2.269.