Enhances group convolutional networks with attention to learn meaningful relationships.
problem Lack of explicit means to learn meaningful relationships among symmetry patterns.
method Introduces attentive group equivariant convolutions, applying attention during convolution.
result Consistently outperforms conventional group convolutional networks on benchmark datasets.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
An impossibility result shows limitations in learning symmetries and equivariant functions.
problem Learning symmetries and equivariant functions simultaneously is impossible under certain conditions.
method Careful study of approximation for groups and semigroups, analysis of neural networks.
result Linearly equivariant networks can be used to learn equivariant functions, but group-convolutional networks have limitations.
L-CNNs maintain gauge symmetry on non-Abelian lattice theories.
problem Applying convolutional neural networks to non-Abelian lattice gauge theories while preserving gauge symmetry.
method Developed a geometric formulation of L-CNNs that are equivariant under global symmetries and gauge transformations.
result Convolutional operations in L-CNNs are a specific case of gauge-equivariant neural networks on SU(N) principal bundles. This work introduces a method for almost equivariance in neural networks using Lie algebra convolutions.
problem Real-world data often does not conform to strict group equivariances, leading to underperformance in models.
method Definition and practical implementation of almost equivariance through Lie algebra convolutions.
result Demonstrated the validity of the approach through benchmarking against fully equivariant settings.
Convolutional neural networks have been extremely successful in the image recognition domain because they ensure equivariance to translations. There have been many recent attempts to generalize this framework to other domains, including graphs and data lying on manifolds. In this paper we give a rigorous, theoretical t…
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …
We propose a semantic segmentation model that exploits rotation and reflection symmetries. We demonstrate significant gains in sample efficiency due to increased weight sharing, as well as improvements in robustness to symmetry transformations. The group equivariant CNN framework is extended for segmentation by introdu…
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translation-equivariant (ST-equivariant) CNN with joint convolutions across th…
LieTransformer extends self-attention to Lie groups for improved deep learning tasks.
problem Improving deep learning performance through group equivariant self-attention.
method LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
result Competitive experimental results on various tasks.
GCNNs on homogeneous spaces use vector bundles and Hilbert spaces.
problem Learning data on homogeneous spaces with global symmetry.
method Analysis of G-equivariant convolutional layers on homogeneous G/K spaces, using vector bundles and reproducing kernel Hilbert spaces. result A precise criterion for expressing G-equivariant layers as convolutional layers, leading to stronger results for some groups. PDE-based G-CNNs add geometric symmetries to CNNs without augmentation.
problem Designing CNNs with built-in symmetries like rotation.
method Formulate CNN layers as PDE solvers on homogeneous spaces.
result PDE-G-CNNs achieve better performance with fewer parameters.
Lie groupoid equivariant neural networks are a new type of neural network.
problem Designing neural networks that respect the structure of Lie groupoids.
method Introducing Lie groupoid equivariant convolutions and layers, and showing their equivalence to Lie algebroid-equivariant networks.
result Lie groupoid equivariant neural networks are equivalent to certain Lie algebroid-equivariant networks.
We present a general theory of Group equivariant Convolutional Neural Networks (G-CNNs) on homogeneous spaces such as Euclidean space and the sphere. Feature maps in these networks represent fields on a homogeneous base space, and layers are equivariant maps between spaces of fields. The theory enables a systematic cla…
Develops a framework for designing quantum neural networks that respect symmetries.
problem Trainability and generalization issues in quantum neural networks.
method Equivariant quantum neural networks (EQNN) for any symmetry group.
result Efficient construction of equivariant layers for EQNNs, including QCNNs.
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.
Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
Unified method for CNNs to approximate equivariant maps across various groups.
problem Limited universal approximation theorems for CNNs with specific groups and settings.
method Unified approach to derive universal approximation theorems for equivariant maps by CNNs in diverse settings.
result Ability to handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
Coordinate-independent convolutions on manifolds avoid reference frame ambiguity.
problem Applying convolutions on non-Euclidean manifolds without reference frame ambiguity.
method Developed coordinate-independent and gauge-equivariant convolutions on Riemannian manifolds.
result Coordinate-independent convolutions are equivariant under local gauge transformations.
The study quantifies how many objects can be linearly classified under all views.
problem Understanding the expressivity of group-equivariant representations.
method Generalization of Cover's Function Counting Theorem to quantify separable dichotomies.
result The fraction of separable dichotomies is determined by the fixed space dimension of the group action.
Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations, in which data characteristics are effectively learned, geometri…
Equivariant neural networks are a class of neural networks designed to preserve symmetries inherent in the data. In this paper, we introduce a general method for modifying a neural network to enforce equivariance, a process we refer to as equivarification. We further show that group convolutional neural networks (G-CNN…
Paper defines mathematical framework for neural network explainability.
problem Neural network explainability and equivariant operators.
method Mathematical framework based on Group Equivariant Non-Expansive Operators (GENEOs) and complexity measures.
result Formal properties and interpretability of Group Equivariant Operators (GEOs) defined.
Geometric models improve feature extraction and equivariance in image generation.
problem Improving feature extraction at multiscale levels and reducing network complexity.
method Proposes a geometric generative model based on morphological PDEs and GANs, incorporating equivariance for geometric interpretability.
result Preliminary results show GM-GAN outperforms classical GANs on MNIST data.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
New framework for equivariant neural networks using Lie group decompositions.
problem Limitations of existing equivariant neural network methods for Lie groups.
method Lie group structure and geometry, decomposition into subgroups and submanifolds.
result Equivariant neural networks for affine transformations outperform previous methods.
We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equivariant convolutions utilize kernels which are parameterized as a …
L-CNNs learn gauge invariant quantities on lattices.
problem Learning gauge invariant quantities on lattices.
method Novel convolutional layer preserving gauge equivariance and forming Wilson loops.
result L-CNNs can approximate any gauge covariant function on the lattice.
In this article, we start to recall the inversion formula for the convolution with the Box spline. The equivariant cohomology and the equivariant K-theory with respect to a compact torus G of various spaces associated to a linear action of G in a vector space M can be both described using some vector spaces of distribu…
New framework uses symmetry-based matrices for efficient, flexible NNs.
problem Designing neural networks with relaxed equivariance.
method Symmetry-based structured matrices, Group Matrices (GMs).
result GMs enable competitive performance with fewer parameters.
L-CNNs preserve gauge symmetry in neural networks.
problem Applying machine learning to lattice gauge theory while preserving gauge symmetry.
method L-CNNs use gauge equivariance to construct a gauge equivariant convolutional layer and bilinear layer.
result L-CNNs achieve higher accuracy in non-linear regression tasks compared to non-equivariant CNNs.
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding…
Method learns equivariances from data without custom architecture design.
problem Learning equivariances for tasks without manually designed architectures.
method Reparameterization to learn equivariant parameter sharing.
result Can learn equivariances for any finite group of transformations.
Rotationally equivariant convolutions improve molecular property prediction.
problem Predicting molecular properties using graph neural networks.
method Ablation study with rotationally equivariant and invariant convolutions on QM9 data set.
result Rotationally equivariant layers decrease test error by an average of 23%.
New homological results for bordered Floer algebras derived from hypertoric categories.
problem Homological properties of bordered Floer algebras.
method Affine quasi hereditary property of equivariant hypertoric convolution algebras and computation of Ext groups.
result Existence of standard modules and isomorphism of Ext groups to bordered strands dg algebras.
The aim of this paper is to provide a general mathematical framework for group equivariance in the machine learning context. The framework builds on a synergy between persistent homology and the theory of group actions. We define group-equivariant non-expansive operators (GENEOs), which are maps between function spaces…
The paper generalizes equivariant neural networks on homogeneous spaces to the non-linear setting.
problem Equivariant neural networks on homogeneous spaces.
method Deriving generalized steerability constraints for non-linear equivariant layers.
result The universality of the derived construction for non-linear equivariant layers.
Translationally equivariant neural networks improve performance and generalization in physics problems.
problem Performance and generalization issues in machine learning applied to physics problems.
method Investigation of translationally equivariant convolutional neural networks for complex scalar field theory on a 2D lattice.
result Translationally equivariant neural networks significantly outperform non-equivariant architectures in various regression and classification tasks.
Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of s…
The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various t…
IsoGCNs learn invariant and equivariant graph features for efficient simulations.
problem Learning isometric transformation invariant and equivariant features in graphs for simulations.
method Transformation invariant and equivariant Graph Convolutional Networks (IsoGCNs).
result IsoGCNs outperform state-of-the-art methods on geometrical and physical simulation tasks.
The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be exte…
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds …
This paper develops optimal transport methods on the roto-translation group SE2.
problem Optimal transport on the roto-translation group SE2 for image analysis.
method Develops a computational framework for optimal transportation over Lie groups, focusing on SE2. Uses Sinkhorn-like algorithm with efficient distance approximations.
result Advances in image barycentric interpolation, orientation field interpolation, and Wasserstein flows on SE2.