New DDMs use neural networks for solving equations on manifold shapes.
problem Solving equations on complex, high-dimensional shapes.
method Physics-informed neural networks combined with domain decomposition methods.
result Validated methods work well on various shapes in high dimensions.
Introduces MFCNs for better understanding manifold neural networks.
problem Understanding manifold neural networks (MNNs).
method Filter-combine framework on high-dimensional point clouds, approximating manifold by sparse graph.
result Method converges to continuum limit as data points increase.
New neural networks flatten and reconstruct manifolds from samples.
problem Learning from high-dimensional data embedded in submanifolds.
method Flattening Networks (FlatNet) that linearize and reconstruct embedded submanifolds.
result FlatNet achieves balance of interpretability, feasibility, and generalization.
Researchers extend ResNets to Riemannian manifolds, improving performance over existing methods.
problem Learning on Riemannian manifolds, especially for hierarchical graphs and manifold-valued data.
method Geometrically principled extension of ResNets to general Riemannian manifolds.
result Riemannian ResNets outperform existing manifold neural networks in relevant metrics and training dynamics.
MFCNs use sparse graphs to approximate manifold convergence.
problem Understanding manifold neural networks (MNNs).
method Sparse graph approximation for manifold convergence.
result Method converges to continuum limit as data points increase.
Paper builds neural networks on matrix manifolds using gyrovector spaces.
problem Lack of concepts in gyrovector spaces for matrix manifolds.
method Generalized gyrovector space concepts for SPD and Grassmann manifolds, proposing new neural network models.
result Demonstrated effectiveness in human action recognition and knowledge graph completion.
NeuroPMD estimates densities on complex product manifolds.
problem Density estimation on high-dimensional product manifolds.
method Neural network directly parameterizes density, trained with manifold differential operators.
result NeuroPMD outperforms traditional methods in density estimation.
Neural networks learn discrete tasks on continuous data via emergent geometry.
problem Understanding how neural networks perform discrete computations on continuous data.
method Analysis of Riemannian pullback metric across neural network layers.
result Neural networks learn to discretize continuous inputs and perform logical operations on these discretized variables.
Researchers develop neural networks for manifold data with a convergence rate.
problem Analyzing high-dimensional data on non-Euclidean domains.
method Constructing manifold neural networks using spectral decomposition of the Laplace Beltrami operator.
result Established a rate of convergence for the neural network scheme that depends on intrinsic manifold dimension.
Designs neural networks on matrix manifolds for improved performance in tasks like human action recognition.
problem Designing neural networks for tasks on non-Euclidean manifolds.
method Develops fully-connected and convolutional layers for SPD manifolds, and MLR on SPSD manifolds.
result Demonstrates improved performance in human action recognition and node classification tasks.
Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds w…
Finslerian graph neural networks recover nonlinear diffusion geometry
problem Graph neural networks on point clouds
method Estimates of the Finsler Laplacian
result Recovery of Finsler geometry
Deep neural networks estimate regression functions on manifolds.
problem Estimating regression functions on manifolds from data.
method Fully connected deep neural networks with ReLU activation, analyzing convergence rates.
result Estimates achieve a rate of convergence dependent on manifold dimension, not predictor dimension.
Proposes graph neural network layers for manifold-valued graphs.
problem Graphs with features in a Riemannian manifold.
method Diffusion layer and tangent multilayer perceptron.
result Outperforms state-of-the-art networks on Alzheimer's classification.
Paper investigates hardness of learning neural networks under manifold hypothesis.
problem Hardness of learning neural networks under the manifold hypothesis.
method Extending proofs of hardness in the SQ and cryptographic settings to the geometric setting.
result Learning is hard under input manifolds of bounded curvature but learnable with additional assumptions on manifold volume.
NDM incorporates geometric structure into neural networks for better optimization and interpretability.
problem Efficient and interpretable deep learning architectures.
method NDM is a neural network architecture that explicitly incorporates geometric structure into its design, using a Coordinate Layer, Geometric Layer, and Evolution Layer.
result NDM provides intrinsic regularization, enhancing generalization and robustness.
Deep networks can perfectly classify two low-dimensional manifolds on a sphere with large depth and width.
problem Binary classification of two low-dimensional submanifolds on a sphere.
method Analysis of a deep fully-connected neural network trained to separate two submanifolds of the unit sphere.
result Randomly-initialized gradient descent can perfectly classify the two manifolds with high probability when the network depth is large relative to certain geometric and statistical properties of the data.
Riemannian Neural OT maps improve scalability on manifolds.
problem Challenges in extending neural OT to high-dimensional Riemannian manifolds.
method Introduces Riemannian Neural OT (RNOT) maps that avoid discretization and incorporate geometric structure.
result RNOT maps approximate Riemannian OT maps with sub-exponential complexity in the dimension.
New bounds for neural networks on curved manifolds improve generalization.
problem Existing generalization theories fail to account for non-Euclidean manifold structures.
method Derive covering number bounds incorporating manifold-specific properties like curvature.
result Sharp Rademacher complexity bounds for neural networks on compact manifolds.
New Adam optimizer generalized for manifold training of neural networks.
problem Lack of clear physical intuition and difficulty in generalizing Adam optimizer to manifolds.
method Leverages the global tangent space representation of manifolds to perform Adam optimizer steps.
result Significant speed-ups in transformer training with orthogonality constraints.
Study reveals how neural network smoothness affects their vulnerability to adversarial attacks.
problem Understanding adversarial vulnerability in deep learning networks.
method Analysis of manifold smoothness and generalization capability of deep neural networks trained with local errors.
result High generalization accuracy requires a fast power-law decay of eigen-spectrum of hidden representations.
Understanding the reasons for the success of deep neural networks trained using stochastic gradient-based methods is a key open problem for the nascent theory of deep learning. The types of data where these networks are most successful, such as images or sequences of speech, are characterised by intricate correlations.…
Neural networks' performance scales with data size, explained by data manifold dimensionality.
problem Understanding the scaling of neural network performance with the number of parameters.
method Explained by the intrinsic dimension of the data manifold, confirmed through teacher/student framework and various datasets.
result The scaling exponent α is approximately 4 divided by the intrinsic dimension d of the data manifold.
Randomized neural networks improve function approximation on manifolds.
problem Slow learning in neural networks on manifolds.
method Random vector functional link networks for function approximation.
result Theoretical guarantees for function approximation on manifolds with high probability.
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encoura…
MSA compares neural representations' intrinsic geometry for better understanding.
problem Existing similarity measures fail to capture subtle distinctions between neural network solutions.
method Metric similarity analysis (MSA) using Riemannian geometry.
result MSA can disentangle features of neural computations and compare nonlinear dynamics.
This work proposes a new neural implicit manifold model for more accurate density estimation on manifolds.
problem Current generative models struggle with representing manifolds accurately and learning densities within them.
method Proposes a neural implicit manifold model and a constrained energy-based model to learn manifold-supported distributions.
result The proposed model can learn manifold-supported distributions with complex topologies more accurately than pushforward models.
We discuss approximation of functions using deep neural nets. Given a function f on a d-dimensional manifold Γ⊂Rm, we construct a sparsely-connected depth-4 neural network and bound its error in approximating f. The size of the network depends on dimension and curvature of the manifold Γ, the…
The paper uses Cartan moving frames to analyze data manifolds and neural network outputs.
problem Understanding the geometry and explainability of neural network outputs.
method Employing Cartan moving frames to study the Riemannian structure of data manifolds and their curvature.
result The relationship between neural network outputs and the geometry of inputs is exploited for explainable AI.
Graph Neural Networks solve topology problems in simple 3D models.
problem Deciding homeomorphism of 3-manifolds described by plumbing graphs.
method Supervised and reinforcement learning with Graph Neural Networks.
result High accuracy in determining homeomorphic 3-manifolds.
Novel neural network approach on hyperbolic and SPD spaces.
problem Developing neural networks on symmetric spaces of noncompact type.
method Unified formulation of distance from a point to a hyperplane.
result Closed-form expression for point-to-hyperplane distance in higher-rank spaces.
New neural networks model complex phenomena with fewer parameters.
problem Challenges in studying higher-order interactions in neural networks.
method Introducing curved neural networks using the maximum entropy principle.
result Curved neural networks accelerate memory retrieval and exhibit explosive phase transitions.
This paper improves causal inference using deep neural networks for low-dimensional covariates.
problem Improving causal inference with deep learning for high-dimensional covariates.
method Doubly robust off-policy learning with deep neural networks on low-dimensional manifolds.
result Nonasymptotic regret bounds for finite- and continuous-action scenarios, converging at a fast rate depending on intrinsic manifold dimension.
Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
VDWs enhance graph neural networks for analyzing complex data.
problem Analyzing data on non-Euclidean geometries.
method Incorporating vector diffusion wavelets into geometric graph neural networks.
result VDW-GNNs effectively analyze synthetic and real-world data.
Neural nets approximate Ricci flat metrics for Calabi-Yau manifolds.
problem Lack of analytical Ricci flat metrics for Calabi-Yau threefolds.
method Employed neural network approximations for several Calabi-Yau manifolds of dimensions two and three.
result Measures of Ricci flatness improved by three orders of magnitude after training.
Bayesian explanations are more resilient to adversarial attacks than deterministic ones.
problem Stability of saliency-based explanations under adversarial attacks in Neural Networks.
method Empirical and theoretical analysis of Bayesian vs deterministic Neural Networks.
result Bayesian explanations are more stable under adversarial perturbations and direct attacks.
CoMPNetX uses neural networks to efficiently solve constrained motion planning problems.
problem Finding collision-free paths on constraint manifolds efficiently.
method Neural generator and discriminator with neural gradients-based projection operator.
result CoMPNetX finds path solutions with high success rates and lower computation times.
This paper reconstructs equivalence classes in 1D neural networks.
problem Reconstructing equivalence classes in neural networks.
method Singular Riemannian geometry approach.
result Algorithm to build the set of points on the same equivalence class.
Proposes eDNNs and iDNNs for deep learning on manifolds.
problem Deep learning on manifolds with geometric preservation and intrinsic geometry incorporation.
method Intrinsic and extrinsic deep neural networks (iDNNs and eDNNs) with geometric embeddings and maps.
result Empirical risk minimizers of eDNNs and iDNNs converge optimally.
We extend manifold capacity to nonlinear neural representations with contextual information.
problem Efficient processing of information through neural representations.
method Theoretical framework leveraging latent directions in input space related to contextual information.
result Derivation of an exact formula for context-dependent manifold capacity.
Transforms between neural networks using manifold-learning techniques.
problem Establish equivalence between different neural networks.
method Diffusion maps with a Mahalanobis-like metric to construct transformations between network outputs and internal neuron activations.
result Established equivalence classes between neural networks trained on various data types.
A neural network learns efficient parametrizations of product shape spaces.
problem Efficiently parametrize complex shape spaces with high computational costs.
method Developed a neural network architecture that separately learns approximations for low-dimensional factors and combines them.
result Demonstrated the effectiveness of the approach on synthetic and real data.
Improved memory efficiency for deep neural networks.
problem Increasing model size makes deep learning less accessible.
method Inductive-bias-driven network design and manifold-oriented regularization.
result Improved performance and generalization error compared to traditional methods.
DMT enhances deep neural networks to better preserve data structures.
problem Preserving geometric, topological, and distributional structures of data in NLDR.
method Deep manifold transformation (DMT) using cross-layer LGP constraints.
result DMT networks outperform existing NLDR methods in preserving data structures.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued representations often relies on Euclidean approximations or costly geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs.
result Generalizes batch normalization and multinomial logistic regression to broader classes of manifolds.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued data lacks reusable modules, specific network architectures, and efficient geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs for broad classes of Lie groups and gyrogroups.
result Generalizes batch normalization and multinomial logistic regression to Riemannian manifolds, including SPD and hyperbolic spaces.
Random projections are able to perform dimension reduction efficiently for datasets with nonlinear low-dimensional structures. One well-known example is that random matrices embed sparse vectors into a low-dimensional subspace nearly isometrically, known as the restricted isometric property in compressed sensing. In th…