Improves stability in hyperbolic neural networks for complex data generation.
problem Numerical instability in hyperbolic neural networks hinders complex architecture development.
method Proposes a novel hyperbolic AE-GAN architecture with stable layers.
result Demonstrates state-of-the-art performance in generating complex data.
Paper presents a novel hyperbolic neural network for efficient data representation.
problem Efficient representation of hierarchical data in hyperbolic space.
method Develops a fully hyperbolic neural network using projections and equivariant embeddings.
result Proves the proposed embedding is isometric and equivariant under Lorentz transformations.
HKConv learns hyperbolic features by aggregating kernel points.
problem Challenges in learning good hyperbolic representations using Euclidean operations.
method Proposes HKConv, a trainable hyperbolic convolution that correlates local features with kernel points and aggregates them.
result HKConv learns expressive local features according to hyperbolic geometry and enjoys equivariance to permutation and invariance to parallel transport.
This work proposes hyperbolic deep convolutional neural networks for better pattern recognition.
problem The limitations of Euclidean deep convolutional neural networks in capturing intricate patterns.
method Developed Hyperbolic DCNN based on Poincaré Disc, analyzing expansive convolution in non-Euclidean space.
result Hyperbolic convolutional architecture outperforms Euclidean ones in pattern recognition tasks.
Enhances neural networks in hyperbolic space for better data structure capture.
problem Capturing hierarchical data structures efficiently.
method Unified hyperbolic model for neural network components.
result Superior parameter efficiency and outperformance over Euclidean methods.
Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean geometry, mostly because of the absence of corresponding hyperbolic neural network lay…
A new framework for hyperbolic neural networks using the Klein model is introduced.
problem Previous works focused on Poincaré and hyperboloid models, neglecting the Klein model.
method Formulation of operations using the Klein model, study of the Klein linear layer, and comparison with Poincaré ball model.
result The Klein HNN performs similarly to the Poincaré ball model, offering a third option.
Graph convolutional neural networks (GCNs) embed nodes in a graph into Euclidean space, which has been shown to incur a large distortion when embedding real-world graphs with scale-free or hierarchical structure. Hyperbolic geometry offers an exciting alternative, as it enables embeddings with much smaller distortion. …
Paper builds neural networks on matrix manifolds using gyrovector spaces.
problem Lack of concepts in gyrovector spaces for matrix manifolds.
method Generalized gyrovector space concepts for SPD and Grassmann manifolds, proposing new neural network models.
result Demonstrated effectiveness in human action recognition and knowledge graph completion.
Novel neural network approach on hyperbolic and SPD spaces.
problem Developing neural networks on symmetric spaces of noncompact type.
method Unified formulation of distance from a point to a hyperplane.
result Closed-form expression for point-to-hyperplane distance in higher-rank spaces.
Learning from graph-structured data is an important task in machine learning and artificial intelligence, for which Graph Neural Networks (GNNs) have shown great promise. Motivated by recent advances in geometric representation learning, we propose a novel GNN architecture for learning representations on Riemannian man…
Differentiates Fréchet mean for hyperbolic space applications.
problem Difficulty in applying Fréchet mean due to lack of closed-form derivative.
method Developed differentiation method and explicit gradient expressions for hyperbolic space.
result Fully integrated Fréchet mean into hyperbolic neural network pipeline.
A novel hyperbolic graph attention network for non-Euclidean graph data.
problem Non-Euclidean graph data requires specialized models to capture its unique properties.
method Employed gyrovector spaces to transform features and hyperbolic proximity attention mechanism for aggregation. Novel acceleration strategy using logarithmic and exponential mappings.
result Demonstrated superior performance on real-world datasets compared to state-of-the-art methods.
Proposes Siegel neural networks for improved classification tasks.
problem Classification on Siegel spaces is underexplored.
method Uses quotient structure and vector-valued distance notation.
result Demonstrates state-of-the-art performance in radar and node classification.
Proposes HypCSE for enhanced hierarchical clustering.
problem Challenges in existing hierarchical clustering methods.
method Hyperbolic Continuous Structural Entropy (HypCSE) neural networks.
result Superior performance on seven datasets.
The study analyzes neural network predictions of knot invariants and finds that braid representations work best.
problem Understanding and predicting knot invariants using neural networks.
method Investigated different knot representations and invariants, proposed a cosine similarity score.
result Braid representations are best for predicting knot invariants, and some invariants are easier to learn than others.
Designs neural networks on matrix manifolds for improved performance in tasks like human action recognition.
problem Designing neural networks for tasks on non-Euclidean manifolds.
method Develops fully-connected and convolutional layers for SPD manifolds, and MLR on SPSD manifolds.
result Demonstrates improved performance in human action recognition and node classification tasks.
Extends RCNNs to handle hypercomplex-valued data.
problem Processing high-dimensional data.
method Develops mathematical background and conditions for hypercomplex-valued RCNNs.
result Hypercomplex-valued RCNNs can always settle at an equilibrium.
Paper proposes HTAF for stable training of binary neural networks.
problem Challenges in training binary neural networks with gradient-based optimization.
method HTAF is a smooth approximation to the Heaviside function that enables stable training.
result HTAF enables stable training of various binary neural networks with gradient-based optimization.
Hyperbolic GANs improve image generation metrics.
problem Improving image generation quality in neural networks.
method Integrating hyperbolic layers into GAN architectures.
result Hyperbolic GANs achieve better metrics than Euclidean counterparts.
Machine learning predicts minimal surfaces for knots, supporting a conjecture.
problem Predicting minimal surfaces for knots in hyperbolic space.
method Physics-Informed Neural Networks (PINNs) to solve minimal surface equation.
result Computational minimal surfaces align with Fine's Conjecture.
New research shows hyperbolic embeddings are useful for global consistency tasks in graphs.
problem The usefulness of hyperbolic representations in graph learning tasks.
method Computed hyperbolic embeddings for node classification and link prediction tasks, addressing optimization issues at zero curvature.
result Hyperbolic embeddings are more effective for tasks requiring global consistency, while Euclidean models are superior for other tasks.
New method predicts neural network performance using free probability theory.
problem Stability and performance prediction of feed-forward neural networks.
method Free Probability Theory and homotopy method for Jacobian spectral density computation.
result FPT metrics correlate highly with final test accuracies of neural networks.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued representations often relies on Euclidean approximations or costly geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs.
result Generalizes batch normalization and multinomial logistic regression to broader classes of manifolds.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued data lacks reusable modules, specific network architectures, and efficient geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs for broad classes of Lie groups and gyrogroups.
result Generalizes batch normalization and multinomial logistic regression to Riemannian manifolds, including SPD and hyperbolic spaces.
Researchers compute and predict knot volumes using colored Jones polynomials.
problem Computing and predicting volumes of hyperbolic knots.
method Vertex model approach, neural network training, polynomial evaluations.
result 3-colored Jones polynomials predict knot volumes with high accuracy.
Develops weak PINNs for efficient manifold solutions of hyperbolic equations.
problem Challenges in approximating weak solutions of nonlinear hyperbolic equations on manifolds.
method Introduces a novel weak PINN (wPINN) formulation on manifolds leveraging well-posedness theory.
result Demonstrates efficient approximation of entropy solutions on manifolds with a complexity independent of ambient space dimension.
Simple Euclidean models outperform hyperbolic graph learning models.
problem The effectiveness of hyperbolic graph learning models is questioned.
method Careful analysis of hyperbolic graph representation learning, identifying and addressing issues with baselines, modeling assumptions, and metric usage.
result Simple Euclidean models often outperform hyperbolic graph learning models, even on hyperbolic datasets.
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
This paper improves HNNs by learning optimal curvature for better generalization.
problem Inappropriate curvatures in HNNs lead to suboptimal performance.
method Sharpness-aware curvature learning method to smooth loss landscape.
result Proposed method improves HNNs' generalization across various settings.
An important conjecture in knot theory relates the large-N, double scaling limit of the colored Jones polynomial JK,N(q) of a knot K to the hyperbolic volume of the knot complement, Vol(K). A less studied question is whether Vol(K) can be recovered directly from the original Jones polynomial …
Neural embeddings have been used with great success in Natural Language Processing (NLP). They provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks. The success of neural embeddings has prompted significant amounts of research into appli…
Adaptive Group Lasso selects important features in neural networks.
problem Lack of interpretability in neural networks.
method Adaptive Group Lasso for feature selection.
result Consistent feature selection for neural networks with theoretical guarantee.
Neural networks simplify complex data topologies into simpler ones.
problem Understanding why deep neural networks perform better than shallow ones and why ReLU activations are superior.
method Persistent homology analysis of neural network layers on various data sets.
result Neural networks reduce the topological complexity of input data sets, often to their simplest form.
Deep learning uncovers patterns between knot types.
problem Discovering connections between combinatorial and hyperbolic knot invariants.
method Statistical approach using linear regression and deep learning.
result Revealed empirical connections between knot types.
Training feedforward neural networks with standard logistic activations is considered difficult because of the intrinsic properties of these sigmoidal functions. This work aims at showing that these networks can be trained to achieve generalization performance comparable to those based on hyperbolic tangent activations…
Researchers extend ResNets to Riemannian manifolds, improving performance over existing methods.
problem Learning on Riemannian manifolds, especially for hierarchical graphs and manifold-valued data.
method Geometrically principled extension of ResNets to general Riemannian manifolds.
result Riemannian ResNets outperform existing manifold neural networks in relevant metrics and training dynamics.
In this paper, we address the stability of a broad class of discrete-time hypercomplex-valued Hopfield-type neural networks. To ensure the neural networks belonging to this class always settle down at a stationary state, we introduce novel hypercomplex number systems referred to as real-part associative hypercomplex nu…
We developed safe and ranged approximations for ReLU, tanh, and sigmoid functions to reduce neural network training time.
problem Expensive computation of hyperbolic tangent and sigmoid functions in neural networks.
method Function approximation techniques to create safe and ranged approximations.
result 10% to 37% improvement in training times on CPU and 20% to 53% improvement in ranged cases.
Automates machine learning of correlations between knot invariants.
problem Discovering and validating new relationships between knot invariants.
method Trained a neural network on 200,000 sets of knot invariants to predict an output invariant.
result Found novel correlations not explained by known results in knot theory.
Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.
problem Enhance graph representation learning by leveraging spectral and curvature signals.
method Proposes Spectro-Riemannian Graph Neural Networks (CUSP) that combines spectral and curvature insights.
result Empirical evaluation shows CUSP outperforms state-of-the-art models by up to 5.3%.
In this paper, we propose a geometric framework to analyze the convergence properties of gradient descent trajectories in the context of linear neural networks. We translate a well-known empirical observation of linear neural nets into a conjecture that we call the \emph{overfitting conjecture} which states that, for a…
Abstract Neural Networks (ANNs) improve DNN verification efficiency.
problem Efficiently verify safety-critical DNNs without slowing exponentially.
method Introduces ANNs that use abstract domains and activation functions to overapproximate DNNs.
result ANNs can soundly overapproximate DNNs with fewer nodes, improving verification efficiency.
Analyzes the structure and rank of neural network Hessians.
problem Understanding redundancy in overparameterized neural networks.
method Theoretical tools to analyze Hessian map range and rank deficiency.
result Exact formulas and tight upper bounds for Hessian rank of deep linear networks.
BIGUE algorithm provides credible intervals for hyperbolic network embeddings.
problem Uncertainty in hyperbolic network embeddings.
method Markov chain Monte Carlo (MCMC) algorithm for Bayesian hyperbolic random graph model.
result Samples from the posterior distribution provide credible intervals for hyperbolic coordinates and network properties.
The paper develops mathematical models for neural networks using non-compact symmetric spaces.
problem Developing mathematical models for neural networks using non-compact symmetric spaces.
method Introducing layers modeled as non-compact symmetric spaces, each mapped onto the next by solvable group homomorphisms.
result Group theoretical construction of separators for all non-compact symmetric spaces and uniformization of specific surfaces.
This paper proposes a spectral clustering algorithm for hyperbolic spaces, improving efficiency over Euclidean methods.
problem Inefficient clustering in Euclidean spaces for complex data structures.
method Developed a spectral clustering algorithm using hyperbolic similarity matrices.
result The algorithm converges at least as fast as Euclidean spectral clustering and performs better on complex datasets.
New Riemannian GNNs reduce over-squashing in graphs with negative curvature.
problem Over-squashing in Riemannian graph neural networks.
method Generalization of Hyperbolic GNNs to Riemannian manifolds of variable curvature.
result Bounds on sensitivity of node features in Riemannian GNNs as layers increase.