How to understand deep learning systems remains an open problem. In this paper we propose that the answer may lie in the geometrization of deep networks. Geometrization is a bridge to connect physics, geometry, deep network and quantum computation and this may result in a new scheme to reveal the rule of the physical w…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “geometric deep learning”
Geometric deep learning predicts knot invariants.
problem Predicting knot invariants from knot data.
method Constructing a functor from knots to graphs and using graph neural networks.
result High generalization capabilities demonstrated.
Why and how that deep learning works well on different tasks remains a mystery from a theoretical perspective. In this paper we draw a geometric picture of the deep learning system by finding its analogies with two existing geometric structures, the geometry of quantum computations and the geometry of the diffeomorphic…
tf_geometric simplifies graph deep learning in TensorFlow.
problem Efficient graph deep learning in TensorFlow.
method Kernel libraries and infrastructures for GNNs.
result tf_geometric supports various graph tasks and provides efficient GNN models.
We introduce two constructions in geometric deep learning for 1) transporting orientation-dependent convolutional filters over a manifold in a continuous way and thereby defining a convolution operator that naturally incorporates the rotational effect of holonomy; and 2) allowing efficient evaluation of manifold convol…
Geometric Occam's Razor shapes deep learning solutions.
problem Understanding the regularization in over-parameterized neural networks.
method Analyzing the geometric model complexity and Dirichlet energy in neural networks.
result Over-parameterized neural networks are implicitly regularized by geometric model complexity.
DMT enhances deep neural networks to better preserve data structures.
problem Preserving geometric, topological, and distributional structures of data in NLDR.
method Deep manifold transformation (DMT) using cross-layer LGP constraints.
result DMT networks outperform existing NLDR methods in preserving data structures.
Deep learning models complex multivariate extremes using geometric shapes.
problem Modeling complex extremal dependencies in high-dimensional data.
method Geometric representation and deep learning for flexible semi-parametric models.
result First approach to modeling limit sets using deep learning for high-dimensional data.
Explains deep learning models and their geometric properties.
problem Understanding the geometric intuition behind deep learning models.
method Geometrical intuition and novel insights into loss surfaces of deep learning models.
result Deep neural networks carve out manifolds with multiplication neurons.
Unified geometric flows improve deep learning efficiency and simplify neural network topologies.
problem Improving deep learning performance and simplifying neural network structures.
method Proposes a thermodynamically coupled Ricci flow that dynamically adapts parameter space geometry to loss landscape topology, enabling automated singularity resolution and providing entanglement entropy bounds.
result Demonstrates 2.1× convergence acceleration and 63% topological simplification while maintaining complexity, outperforming Riemannian baselines by 15.2% in few-shot accuracy.
Enhances quantum circuit synthesis using deep learning and geometric methods.
problem Optimizing quantum circuits for time efficiency.
method Combining deep learning with geometric control techniques.
result Improved time-optimal control in quantum circuit synthesis.
FiberNet integrates geometry into machine learning for clearer classification.
problem Lack of interpretability in traditional deep learning.
method Reformulates classification as geometric optimization on fiber bundles, introducing learnable Riemannian metrics and variational prototype optimization.
result Clear geometric interpretability and efficiency in classification.
GNPs learn operators on non-Euclidean geometries using neural networks.
problem Learning operators on complex geometries like manifolds.
method Geometric Neural Operators (GNPs) that incorporate geometric properties.
result GNPs can estimate metrics, solve PDEs, and learn LB operators on manifolds.
New measure shows various training techniques control model complexity.
problem Understanding how to control model complexity in deep learning.
method Developed geometric complexity measure and demonstrated its effectiveness.
result Many training techniques control geometric complexity, providing a unified framework.
MLDL preserves manifold geometry in vector transformations.
problem Geometric deterioration in neural network transformations.
method Locally isometric smoothness (LIS) and Markov random field (MRF) encoding.
result Enhanced vector transformations into well-behaved metric homeomorphisms.
New framework explains neural network behavior through geometric postulates.
problem Understanding neural network mechanisms and making them more transparent.
method Introducing the Pursuit of Subspaces (PoS) hypothesis as an axiomatic framework.
result Unified geometric perspective on neural network representation, computation, and generalization.
Parametric approaches to Learning, such as deep learning (DL), are highly popular in nonlinear regression, in spite of their extremely difficult training with their increasing complexity (e.g. number of layers in DL). In this paper, we present an alternative semi-parametric framework which foregoes the ordinarily requi…
Deep learning approximates geometric measures of planar curves.
problem Approximating differential invariants of planar curves.
method Utilizing deep neural networks to estimate geometric measures of planar curves.
result Deep neural networks can learn to overcome instabilities and sampling artifacts.
A deep learning model organizes RNA graphs to reveal folding patterns and properties.
problem Organizing and understanding the complex folding patterns of RNA secondary structures.
method Geometric scattering autoencoder (GSAE) network for learning graph embeddings.
result GSAE accurately reflects bistable RNA structures and can sample new folding trajectories.
Proposes eDNNs and iDNNs for deep learning on manifolds.
problem Deep learning on manifolds with geometric preservation and intrinsic geometry incorporation.
method Intrinsic and extrinsic deep neural networks (iDNNs and eDNNs) with geometric embeddings and maps.
result Empirical risk minimizers of eDNNs and iDNNs converge optimally.
Unified geometric principles unify neural network architectures.
problem High-dimensional learning tasks with underlying low-dimensionality and structure.
method Unified geometric principles applied to neural network architectures.
result Unified mathematical framework for neural network architectures.
Deep learning is the mainstream technique for many machine learning tasks, including image recognition, machine translation, speech recognition, and so on. It has outperformed conventional methods in various fields and achieved great successes. Unfortunately, the understanding on how it works remains unclear. It has th…
Topological parallax assesses AI models' geometric similarity to datasets for safety.
problem Ensuring AI models' robustness and safety in deep learning applications.
method Topological parallax compares a trained model to a reference dataset using Rips complexes and geodesic distortions.
result Topological parallax indicates whether a model shares similar multiscale geometric features with the dataset.
A new geometric shaping method is proposed, leveraging unsupervised machine learning to optimize the constellation design. The learned constellation mitigates nonlinear effects with gains up to 0.13 bit/4D when trained with a simplified fiber channel model.
Direct Feedback Alignment performs well on diverse deep learning tasks and architectures.
problem The limitations of backpropagation in parallelizing and scaling to modern deep learning tasks.
method Direct Feedback Alignment approach applied to neural view synthesis, recommender systems, geometric learning, and natural language processing.
result Direct Feedback Alignment successfully trains a wide range of state-of-the-art deep learning architectures with performance close to fine-tuned backpropagation.
Root Laplacian Eigenmaps help in spectral embedding of graphs.
problem Efficient spectral embedding of graphs.
method Square root of graph-Laplacian operator.
result Improved spectral embedding techniques.
Deep networks improve by progressively refining approximations at each layer.
problem Standard approximation theory doesn't explain the role of intermediate layers in deep neural networks.
method Developed a mixed-activation architecture with a geometric scale interpretation of depth.
result Each intermediate layer approximates the target function with a geometric rate.
Deep learning transforms data geometrically, akin to Ricci flow, improving classification accuracy.
problem Understanding geometric transformations in non-smooth activation functions.
method Developed a computational framework to quantify geometric changes in DNNs and introduced the concept of `global Ricci network flow`.
result Global Ricci network flow correlates with DNN accuracy, independent of network architecture and data set.
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
Deep models store facts in geometric embeddings, not just associative memory.
problem Understanding how deep models store and utilize atomic facts.
method Identified geometric memory, contrasting with associative lookup.
result Geometric memory transforms hard reasoning into easy tasks.
LightGCNet simplifies AI for soft sensors, reducing complexity and training time.
problem Complex and resource-intensive deep learning models for soft sensors.
method LightGCNet uses compact angle constraints and node pool strategy for efficient learning.
result LightGCNet achieves small network size, fast learning, and good generalization.
A new model for graph clustering using curvature spaces.
problem Graph clustering from a geometric perspective.
method Introducing a heterogeneous curvature space and a contrastive learning approach.
result CONGREGATE model outperforms state-of-the-art competitors.
The paper explores deep learning through algebra and geometry, highlighting geometric structures and differential processes.
problem Understanding the geometric and algebraic foundations of deep learning.
method Investigates neural networks from perceptron to transformer, emphasizing geometric structures and differential processes.
result A coordinate-free formulation of backpropagation equations using canonical scalar products on matrix spaces.
Measuring global properties of neural generative model outputs via generating mathematical objectscs.LG
Deep models generate geometric objects with global properties.
problem Comparing neural models' global properties from generated samples.
method Training on datasets of reflexive polytopes, comparing different representations.
result Models learn non-trivial global properties of geometric objects.
Deep neural networks favor symmetric structures, enabling multilevel symmetries.
problem Understanding and optimizing deep neural networks.
method Formulating DNN training as convex Lasso problems with geometric algebra.
result Deep networks inherently favor symmetric structures, enabling multilevel symmetries.
New geometric regularizers improve deep learning generalization.
problem Improving deep learning models' ability to generalize to unseen data.
method Using Bregman divergence loss and bounded spectral products, we propose a novel geometric regularizer to enhance model generalization.
result Good generalization can be achieved by designing deep models with specific structural regularizers.
SpatialSim benchmarks machine learning in recognizing object spatial configurations.
problem Machine learning in recognizing precise geometrical configurations of groups of objects.
method SpatialSim benchmark with tasks of Identification and Comparison, using Graph Neural Networks (MPGNNs).
result MPGNNs outperform baselines in recognizing spatial configurations, highlighting current limits.
AIDN uses deep learning to represent algebraic structures.
problem Building learning systems to uncover algebraic laws from data.
method AIDN is a deep learning algorithm that represents algebraic objects using neural networks.
result AIDN can robustly compute representations of various algebraic structures.
Study shows latent space OOD detection isn't a reliable proxy for model performance.
problem Evaluating and interpreting deep learning systems on real-world data.
method Empirical investigation of latent space OOD detection and classification accuracy using SAR datasets.
result OOD detection cannot be used as a proxy measure for model performance.
Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
Geometric Approaches to Increase the Expressivity of Deep Neural Networks for MR Reconstructioncs.CV
Enhances deep neural networks for MRI reconstruction by increasing expressivity.
problem Balancing network complexity and performance in deep learning MRI reconstruction.
method Geometric approach using bootstrapping and subnetwork aggregation with attention module.
result Significant improvement in MRI reconstruction performance with minimal complexity increase.
A new DL framework preserves geometric structures for causal predictions.
problem Designing deep learning models for geometrically structured data.
method Introduces a universal causal geometric DL framework.
result DL models can approximate any regular map between metric spaces.
The paper constructs minimizers for deep learning networks and analyzes their geometric structure.
problem Underparametrized deep learning networks and their minimizers.
method Direct construction of minimizers without gradient descent, considering specific settings.
result Explicit family of minimizers for the global minimum and a set of degenerate local minima.
New theory for local parameterization of deep ReLU networks.
problem Determining local parameters of deep ReLU neural networks.
method Introducing local lifting operators and charts of a manifold, deriving necessary and sufficient conditions for local identifiability.
result Sharp and testable conditions for local identifiability of deep ReLU networks.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued representations often relies on Euclidean approximations or costly geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs.
result Generalizes batch normalization and multinomial logistic regression to broader classes of manifolds.
In spite of achieving revolutionary successes in machine learning, deep convolutional neural networks have been recently found to be vulnerable to adversarial attacks and difficult to generalize to novel test images with reasonably large geometric transformations. Inspired by a recent neuroscience discovery revealing t…
Study on limits and cut-off phenomena in deep neural networks.
problem Understanding the behavior of deep neural networks as the number of layers increases.
method Analysis of semi-invariant metrics and application of non-commutative ergodic theorems.
result Observation of a cut-off phenomenon in the number of layers for random network initialization.
GDB bridges geometric states with improved accuracy and generality.
problem Challenges in predicting geometric state evolution in complex systems.
method Geometric Diffusion Bridge (GDB) framework using equivariant diffusion bridges.
result GDB surpasses existing methods in accurately bridging geometric states.