Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
GE-autoencoder identifies spontaneous symmetry breaking in systems.
problem Locating phase boundaries and identifying spontaneously broken symmetries in systems.
method Group-equivariant autoencoder using group theory to constrain parameters and learn invariant order parameters.
result GE-autoencoder accurately determines spontaneous symmetry breaking and estimates critical temperatures more efficiently.
The paper studies geometric properties of group equivariant operators and their Riemannian structure.
problem Understanding the geometric structure of group equivariant operators.
method Endowing the space of group equivariant non-expansive operators with a Riemannian manifold structure and using gradient descent methods.
result Gradient descent methods can be applied to minimize cost functions on the space of group equivariant non-expansive operators.
GSA-Nets apply group equivariance to self-attention for vision tasks.
problem Improving self-attention networks for vision tasks.
method Define group-equivariant positional encodings.
result GSA-Nets outperform non-equivariant self-attention networks on vision benchmarks.
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
Characterizes group-equivariant neural networks for three groups.
problem Understanding equivariant neural networks for orthogonal, special orthogonal, and symplectic groups.
method Characterized all possible group-equivariant neural networks for three groups.
result Found spanning sets of matrices for learnable, linear equivariant layer functions.
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
Category theory enhances understanding of group-equivariant neural networks.
problem Understanding and working with group-equivariant neural networks.
method Application of category theory to tensor power spaces of Rn for groups Sn, O(n), Sp(n), and SO(n). result New insights and an algorithm for computing equivariant linear layers.
New neural networks for non-commutative data.
problem No existing neural networks suitable for non-commutative data.
method Developed compact matrix quantum group equivariant neural networks.
result Characterized weight matrices for easy compact matrix quantum groups.
Although group convolutional networks are able to learn powerful representations based on symmetry patterns, they lack explicit means to learn meaningful relationships among them (e.g., relative positions and poses). In this paper, we present attentive group equivariant convolutions, a generalization of the group convo…
Paper defines mathematical framework for neural network explainability.
problem Neural network explainability and equivariant operators.
method Mathematical framework based on Group Equivariant Non-Expansive Operators (GENEOs) and complexity measures.
result Formal properties and interpretability of Group Equivariant Operators (GEOs) defined.
SymPE breaks symmetries in equivariant networks, improving performance across various tasks.
problem Equivariant networks cannot break symmetries, leading to poor performance in tasks with symmetrical inputs.
method Novel equivariant conditional distributions and randomized canonicalization.
result SymPE significantly improves performance of group-equivariant and graph neural networks.
New algorithm speeds up group equivariant neural networks computations.
problem Challenging computations in group equivariant neural networks.
method Diagrammatic framework based on category theory for matrix multiplication.
result Exponential improvement in time complexity for matrix multiplication.
LieTransformer extends self-attention to Lie groups for improved deep learning tasks.
problem Improving deep learning performance through group equivariant self-attention.
method LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
result Competitive experimental results on various tasks.
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…
Characterizes a specific type of neural network for alternating group equivariance.
problem Understanding and characterizing neural networks with alternating group equivariance.
method Characterization of all possible An-equivariant neural networks using tensor powers of Rn. result Found a basis of matrices for learnable, linear An-equivariant layer functions. A projection maps geodesic currents to Teichmüller space.
problem Mapping geodesic currents to Teichmüller space.
method Equivariant, length-minimizing projection from filling currents to Teichmüller space.
result The projection is well-behaved and maps geodesic currents to Teichmüller space.
FGNNs improve game-playing AI by exploiting symmetries.
problem Symmetrical game states are not exploited by current AI.
method Introduces FGNNs for creating group-equivariant neural networks.
result FGNNs improve performance in games like checkers and image segmentation.
Paper shows mapping class group-equivariant Teichmüller space deformation to Thurston spine.
problem Mapping Teichmüller space to Thurston spine.
method Equivariant deformation retraction of Teichmüller space onto a cell complex.
result Thurston spine contains points corresponding to hyperbolic surfaces with shortest geodesics forming polygons.
The aim of this paper is to provide a general mathematical framework for group equivariance in the machine learning context. The framework builds on a synergy between persistent homology and the theory of group actions. We define group-equivariant non-expansive operators (GENEOs), which are maps between function spaces…
L-CNNs maintain gauge symmetry on non-Abelian lattice theories.
problem Applying convolutional neural networks to non-Abelian lattice gauge theories while preserving gauge symmetry.
method Developed a geometric formulation of L-CNNs that are equivariant under global symmetries and gauge transformations.
result Convolutional operations in L-CNNs are a specific case of gauge-equivariant neural networks on SU(N) principal bundles. Equivariant neural network simplifies particle physics models.
problem Complexity and interpretability in particle physics classification.
method Lorentz group equivariant neural network architecture.
result Simplified, interpretable models with fewer parameters.
The Ptolemy groupoid is a combinatorial groupoid generated by elementary moves on marked trivalent fatgraphs with three types of relations. Through the fatgraph decomposition of Teichmüller space, the Ptolemy groupoid is a mapping class group equivariant subgroupoid of the fundamental path groupoid of Teichmüller space…
New method finds Lie group representations without explicit groups, enabling new neural network architectures.
problem Building neural networks equivariant to arbitrary Lie groups.
method Algorithm to find Lie group representations from Lie algebra structure constants. Self-contained method for constructing Lie group-equivariant neural networks.
result First object-tracking model equivariant to the Poincaré group.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
Schmutz Schaller and Thurston's approaches are dual.
problem Mapping class group-equivariant deformation retractions of Teichmüller space.
method Comparing Schmutz Schaller's and Thurston's methods.
result Schmutz Schaller and Thurston's approaches are dual.
GNPE improves inference for astrophysical systems.
problem Efficiently incorporating geometric properties like equivariances in neural density estimation.
method GNPE integrates equivariances into neural posterior estimation, standardizing data pose while estimating parameters.
result GNPE achieves state-of-the-art accuracy in astrophysical binary black hole inference, reducing inference times by 3 orders of magnitude.
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …
SCENE-Net improves 3D point cloud segmentation with low resource usage and transparency.
problem Lack of resources and transparency in 3D semantic segmentation models.
method SCENE-Net uses signature shapes identified via GENEOs to achieve semantic segmentation with minimal resources.
result SCENE-Net achieves comparable IoU to state-of-the-art methods with less data and computational resources.
Optimal classification requires choosing the right group symmetries, contrary to intuition.
problem Improving binary classification performance by selecting appropriate group symmetries.
method Developed a theoretical framework for designing group equivariant neural networks.
result Optimal classification performance is achieved by selecting the appropriate subgroups of symmetries, not the largest equivariant groups.
Compactifies character varieties for group actions.
problem Compactify character varieties for better topological analysis.
method Real spectrum compactification of character varieties in semisimple Lie groups.
result Provides a compactification with good topological properties.
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.
This note is mostly an exposition of an unpublished result of Deligne, which introduces an analogue of perverse t-structure on the derived category of coherent sheaves on a Noetherian scheme with a dualizing complex. Construction extends to the category of coherent sheaves equivariant under an action of an algebraic …
Lisa Jeffrey and Frances Kirwan developed an integration theory for symplectic reductions. That is, given a symplectic manifold with symplectic group action, they developed a way of pulling the integration of forms on the reduction back to an integration of group-equivariant forms on the original space. We seek an anal…
Classifies linear embeddings of grassmannians and ind-grassmannians.
problem Understanding linear embeddings of grassmannians and ind-grassmannians.
method Classification through isomorphism of Picard groups and direct limits.
result Most linear embeddings of grassmannians are equivariant.
Characterizes area-minimizing maps for surfaces of genus ≥ 2.
problem Equivariant area-minimizing maps on surface covers.
method Classifies minimal surfaces in Hilbert spheres with constant negative Gaussian curvature.
result Characterizes all equivariantly area-minimizing maps from the universal cover of a surface to a Hilbert sphere.
We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these net…
Feature extraction becomes increasingly important as data grows high dimensional. Autoencoder as a neural network based feature extraction method achieves great success in generating abstract features of high dimensional data. However, it fails to consider the relationships of data samples which may affect experimental…
Autoencoders compress and reconstruct data for various applications.
problem Efficiently compress and reconstruct data.
method Neural network architecture that encodes and decodes data.
result Autoencoders can be applied to various data types and applications.
Proposes a Bayesian Autoencoder with sparse Gaussian process priors to capture data correlations.
problem Autoencoders' i.i.d. assumption of latent representations fails to capture data correlations.
method Imposes fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder and uses stochastic gradient Hamiltonian Monte Carlo for posterior estimation.
result Consistently outperforms alternatives relying on Variational Autoencoders on various tasks.
The paper investigates how symmetry in models affects their performance and generalization.
problem Understanding how symmetry in models impacts their performance and generalization.
method Formal unified investigation of intuitions about symmetry in models and data.
result Quantitative bounds and comparisons between model and data equivariance lead to optimal model performance.
Study on dynamics of non-linear autoencoders learning principal components.
problem Technical difficulty in studying non-linear autoencoders due to non-trivial correlations.
method Derive asymptotically exact equations for SGD training of shallow, non-linear autoencoders.
result Autoencoders learn principal components sequentially and tie weights are ineffective.
A new algorithmic framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a \emph{relational regularization} on the learnable latent prior. This regularization penalizes the fused Gromov-Wasserstein (FGW) distance between the la…
One of the most fundamental problems in machine learning is to compare examples: Given a pair of objects we want to return a value which indicates degree of (dis)similarity. Similarity is often task specific, and pre-defined distances can perform poorly, leading to work in metric learning. However, being able to learn …
Chart autoencoders learn latent features preserving manifold topology and geometry, with robust denoising capabilities.
problem Learning low-dimensional latent features of high-dimensional data sampled near a manifold.
method Chart autoencoders encode data into latent features on charts, preserving manifold topology and geometry.
result Chart autoencoders achieve a squared generalization error of n−d+22log4n under proper network architectures. POTATOES improves autoencoder UOD accuracy without tuning.
problem Improving unsupervised outlier detection accuracy.
method Randomly partition data, overfit each part with an autoencoder, use max reconstruction error as anomaly score.
result Significant improvement in UOD performance for dense inlier sets.