Enhances group convolutional networks with attention to learn meaningful relationships.
problem Lack of explicit means to learn meaningful relationships among symmetry patterns.
method Introduces attentive group equivariant convolutions, applying attention during convolution.
result Consistently outperforms conventional group convolutional networks on benchmark datasets.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
L-CNNs maintain gauge symmetry on non-Abelian lattice theories.
problem Applying convolutional neural networks to non-Abelian lattice gauge theories while preserving gauge symmetry.
method Developed a geometric formulation of L-CNNs that are equivariant under global symmetries and gauge transformations.
result Convolutional operations in L-CNNs are a specific case of gauge-equivariant neural networks on SU(N) principal bundles. Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
LieTransformer extends self-attention to Lie groups for improved deep learning tasks.
problem Improving deep learning performance through group equivariant self-attention.
method LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
result Competitive experimental results on various tasks.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
PDE-based G-CNNs add geometric symmetries to CNNs without augmentation.
problem Designing CNNs with built-in symmetries like rotation.
method Formulate CNN layers as PDE solvers on homogeneous spaces.
result PDE-G-CNNs achieve better performance with fewer parameters.
Paper defines mathematical framework for neural network explainability.
problem Neural network explainability and equivariant operators.
method Mathematical framework based on Group Equivariant Non-Expansive Operators (GENEOs) and complexity measures.
result Formal properties and interpretability of Group Equivariant Operators (GEOs) defined.
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …
The paper studies geometric properties of group equivariant operators and their Riemannian structure.
problem Understanding the geometric structure of group equivariant operators.
method Endowing the space of group equivariant non-expansive operators with a Riemannian manifold structure and using gradient descent methods.
result Gradient descent methods can be applied to minimize cost functions on the space of group equivariant non-expansive operators.
GSA-Nets apply group equivariance to self-attention for vision tasks.
problem Improving self-attention networks for vision tasks.
method Define group-equivariant positional encodings.
result GSA-Nets outperform non-equivariant self-attention networks on vision benchmarks.
We propose a semantic segmentation model that exploits rotation and reflection symmetries. We demonstrate significant gains in sample efficiency due to increased weight sharing, as well as improvements in robustness to symmetry transformations. The group equivariant CNN framework is extended for segmentation by introdu…
The aim of this paper is to provide a general mathematical framework for group equivariance in the machine learning context. The framework builds on a synergy between persistent homology and the theory of group actions. We define group-equivariant non-expansive operators (GENEOs), which are maps between function spaces…
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.
We present a general theory of Group equivariant Convolutional Neural Networks (G-CNNs) on homogeneous spaces such as Euclidean space and the sphere. Feature maps in these networks represent fields on a homogeneous base space, and layers are equivariant maps between spaces of fields. The theory enables a systematic cla…
Characterizes group-equivariant neural networks for three groups.
problem Understanding equivariant neural networks for orthogonal, special orthogonal, and symplectic groups.
method Characterized all possible group-equivariant neural networks for three groups.
result Found spanning sets of matrices for learnable, linear equivariant layer functions.
A new capsule network framework that preserves input transformations.
problem Inefficiency in learning part-whole relationships and lack of equivariance guarantees in capsule networks.
method Proposes a new capsule network framework that learns to projectively encode pose-variations for every capsule-type of each layer using a trainable, equivariant function over a grid of group-transformations.
result The proposed framework is equivariant and preserves the compositional representation of an input under transformations.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
Category theory enhances understanding of group-equivariant neural networks.
problem Understanding and working with group-equivariant neural networks.
method Application of category theory to tensor power spaces of Rn for groups Sn, O(n), Sp(n), and SO(n). result New insights and an algorithm for computing equivariant linear layers.
New neural networks for non-commutative data.
problem No existing neural networks suitable for non-commutative data.
method Developed compact matrix quantum group equivariant neural networks.
result Characterized weight matrices for easy compact matrix quantum groups.
New algorithm speeds up group equivariant neural networks computations.
problem Challenging computations in group equivariant neural networks.
method Diagrammatic framework based on category theory for matrix multiplication.
result Exponential improvement in time complexity for matrix multiplication.
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
Characterizes a specific type of neural network for alternating group equivariance.
problem Understanding and characterizing neural networks with alternating group equivariance.
method Characterization of all possible An-equivariant neural networks using tensor powers of Rn. result Found a basis of matrices for learnable, linear An-equivariant layer functions. A projection maps geodesic currents to Teichmüller space.
problem Mapping geodesic currents to Teichmüller space.
method Equivariant, length-minimizing projection from filling currents to Teichmüller space.
result The projection is well-behaved and maps geodesic currents to Teichmüller space.
FGNNs improve game-playing AI by exploiting symmetries.
problem Symmetrical game states are not exploited by current AI.
method Introduces FGNNs for creating group-equivariant neural networks.
result FGNNs improve performance in games like checkers and image segmentation.
The study quantifies how many objects can be linearly classified under all views.
problem Understanding the expressivity of group-equivariant representations.
method Generalization of Cover's Function Counting Theorem to quantify separable dichotomies.
result The fraction of separable dichotomies is determined by the fixed space dimension of the group action.
One of the most fundamental problems in machine learning is to compare examples: Given a pair of objects we want to return a value which indicates degree of (dis)similarity. Similarity is often task specific, and pre-defined distances can perform poorly, leading to work in metric learning. However, being able to learn …
Paper shows mapping class group-equivariant Teichmüller space deformation to Thurston spine.
problem Mapping Teichmüller space to Thurston spine.
method Equivariant deformation retraction of Teichmüller space onto a cell complex.
result Thurston spine contains points corresponding to hyperbolic surfaces with shortest geodesics forming polygons.
GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.
problem Improving robustness of GCNNs to adversarial attacks.
method SVD-Universal technique applied to GCNNs trained with larger rotations.
result SVD-Universal becomes more effective as GCNNs gain rotation invariance.
Develops quantum circuits for faster learning with symmetry considerations.
problem Speeding up learning quantum states with symmetry considerations.
method Utilizes Okounkov-Vershik approach and Young-Jucys-Murphy elements to develop Sn-equivariant convolutional quantum circuits. result Proves Sn-CQA generates any unitary in any given Sn irrep sector, universal for SU(d) symmetry. Develops a framework for designing quantum neural networks that respect symmetries.
problem Trainability and generalization issues in quantum neural networks.
method Equivariant quantum neural networks (EQNN) for any symmetry group.
result Efficient construction of equivariant layers for EQNNs, including QCNNs.
This work introduces a method for almost equivariance in neural networks using Lie algebra convolutions.
problem Real-world data often does not conform to strict group equivariances, leading to underperformance in models.
method Definition and practical implementation of almost equivariance through Lie algebra convolutions.
result Demonstrated the validity of the approach through benchmarking against fully equivariant settings.
Equivariant neural network simplifies particle physics models.
problem Complexity and interpretability in particle physics classification.
method Lorentz group equivariant neural network architecture.
result Simplified, interpretable models with fewer parameters.
The Ptolemy groupoid is a combinatorial groupoid generated by elementary moves on marked trivalent fatgraphs with three types of relations. Through the fatgraph decomposition of Teichmüller space, the Ptolemy groupoid is a mapping class group equivariant subgroupoid of the fundamental path groupoid of Teichmüller space…
Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of s…
New method finds Lie group representations without explicit groups, enabling new neural network architectures.
problem Building neural networks equivariant to arbitrary Lie groups.
method Algorithm to find Lie group representations from Lie algebra structure constants. Self-contained method for constructing Lie group-equivariant neural networks.
result First object-tracking model equivariant to the Poincaré group.
Schmutz Schaller and Thurston's approaches are dual.
problem Mapping class group-equivariant deformation retractions of Teichmüller space.
method Comparing Schmutz Schaller's and Thurston's methods.
result Schmutz Schaller and Thurston's approaches are dual.
Translationally equivariant neural networks improve performance and generalization in physics problems.
problem Performance and generalization issues in machine learning applied to physics problems.
method Investigation of translationally equivariant convolutional neural networks for complex scalar field theory on a 2D lattice.
result Translationally equivariant neural networks significantly outperform non-equivariant architectures in various regression and classification tasks.
GE-autoencoder identifies spontaneous symmetry breaking in systems.
problem Locating phase boundaries and identifying spontaneously broken symmetries in systems.
method Group-equivariant autoencoder using group theory to constrain parameters and learn invariant order parameters.
result GE-autoencoder accurately determines spontaneous symmetry breaking and estimates critical temperatures more efficiently.
New framework for equivariant neural networks using Lie group decompositions.
problem Limitations of existing equivariant neural network methods for Lie groups.
method Lie group structure and geometry, decomposition into subgroups and submanifolds.
result Equivariant neural networks for affine transformations outperform previous methods.
GNPE improves inference for astrophysical systems.
problem Efficiently incorporating geometric properties like equivariances in neural density estimation.
method GNPE integrates equivariances into neural posterior estimation, standardizing data pose while estimating parameters.
result GNPE achieves state-of-the-art accuracy in astrophysical binary black hole inference, reducing inference times by 3 orders of magnitude.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
SCENE-Net improves 3D point cloud segmentation with low resource usage and transparency.
problem Lack of resources and transparency in 3D semantic segmentation models.
method SCENE-Net uses signature shapes identified via GENEOs to achieve semantic segmentation with minimal resources.
result SCENE-Net achieves comparable IoU to state-of-the-art methods with less data and computational resources.
Optimal classification requires choosing the right group symmetries, contrary to intuition.
problem Improving binary classification performance by selecting appropriate group symmetries.
method Developed a theoretical framework for designing group equivariant neural networks.
result Optimal classification performance is achieved by selecting the appropriate subgroups of symmetries, not the largest equivariant groups.
New approximative kernels improve PDE-G-CNNs for geometric deep learning.
problem Inaccurate approximations of exact kernels in PDE-G-CNNs.
method Developed new approximative kernels that work regardless of spatial anisotropy.
result New kernels provide better error estimates and maintain reflectional symmetries.