STAR improves equivariant and invariant representation learning by routing projection heads.
problem Redundant feature learning in equivariant and invariant representation learning.
method Soft Task-Aware Routing (STAR) for projection heads specialization.
result Lower canonical correlations between invariant and equivariant embeddings.
This work uses action equivariance to learn structured latent spaces for reinforcement learning.
problem Learning structured latent spaces for reinforcement learning.
method Introduced a contrastive loss function to enforce action equivariance on learned representations.
result Optimal policies in the abstract MDP can be successfully lifted to the original MDP.
Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding…
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
New model learns graph neural networks equivariant to various transformations.
problem Learning equivariant graph neural networks for complex transformations.
method E(n)-Equivariant Graph Neural Networks (EGNNs) that are computationally efficient and scalable.
result Achieves competitive or better performance without higher-order representations.
A new method uses algebraic insights to create approximately equivariant networks without complex architectures.
problem Designing equivariant neural networks with complex architectures and high computational cost.
method Imposes the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters.
result Matches or outperforms specialized models in several cases, even for infinite groups.
The paper explores how equivariant models' biases affect latent representations for better performance.
problem The impact of inductive biases on latent representations in equivariant models.
method Demonstrates the importance of accounting for inductive biases in latent representations of equivariant models.
result Effective invariant projections can be used to retain information in latent representations, improving downstream tasks.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.
problem Understanding how equivariance in networks impacts generalization error.
method Utilized PAC-Bayesian analysis for equivariant networks, deriving norm-based bounds for generalization error.
result The bound indicates that using larger group size in the model improves generalization error.
A neural scene representation framework enforcing 3D transformations.
problem Learning 3D scene representations from images without 3D supervision.
method Introducing a loss enforcing equivariance of the scene representation with 3D transformations.
result Real-time neural rendering with comparable results to models requiring minutes for inference.
Our work improves VAE latent space clustering by enforcing invariant and equivariant learning.
problem Current VAEs fail to learn invariant and equivariant clusters in latent space.
method We use a mixture model pdf like Gaussian mixtures to enforce deep, group-invariant learning and separate semantic and equivariant variables.
result Our model effectively learns to disentangle invariant and equivariant representations, improving learning rate and image recognition.
Gaussian processes for dynamical systems with Koopman equivariance.
problem Forecasting and learning representations of nonlinear dynamical systems.
method Koopman-equivariant Gaussian processes with linear time-invariant responses and trajectory-based equivariance.
result Enhanced forecasting performance compared to kernel-based methods.
This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformations while providing an equivariant representation of the data. We provide proof of principle demonstrations of how such flows can be learnt…
Representations learnt through deep neural networks tend to be highly informative, but opaque in terms of what information they learn to encode. We introduce an approach to probabilistic modelling that learns to represent data with two separate deep representations: an invariant representation that encodes the informat…
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translation-equivariant (ST-equivariant) CNN with joint convolutions across th…
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
New method finds Lie group representations without explicit groups, enabling new neural network architectures.
problem Building neural networks equivariant to arbitrary Lie groups.
method Algorithm to find Lie group representations from Lie algebra structure constants. Self-contained method for constructing Lie group-equivariant neural networks.
result First object-tracking model equivariant to the Poincaré group.
Enhances graph neural networks with structural message-passing for better generalization.
problem Limited representation power and inability to learn basic graph topological properties.
method Proposes a framework that includes a one-hot encoding of nodes and parametrized message and update functions ensuring permutation equivariance.
result Achieves state-of-the-art results on molecular graph regression on the ZINC dataset.
Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of s…
New method breaks symmetry in neural networks, improving sample efficiency.
problem Symmetry in neural networks limits their ability to learn unique features.
method Introduces 'relaxed equivariance' to overcome symmetry limitations.
result Equivariant multilayer perceptrons (E-MLPs) can now break symmetry at the sample level.
Machine learning uses invariant theory to restrict function classes.
problem Creating function classes that respect physical law constraints.
method Using equivariant machine learning and Malgrance's method to parameterize functions.
result Explicitly parameterizes equivariant functions between linear spaces.
One of the most fundamental problems in machine learning is to compare examples: Given a pair of objects we want to return a value which indicates degree of (dis)similarity. Similarity is often task specific, and pre-defined distances can perform poorly, leading to work in metric learning. However, being able to learn …
In this paper, we introduce the classification of equivariant principal bundles over the 2-sphere. Isotropy representations provide tools for understanding the classification of equivariant principal bundles. We consider a Γ-equivariant principal G-bundle over S2 with structural group G a compact connected Lie…
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds …
Enhances group convolutional networks with attention to learn meaningful relationships.
problem Lack of explicit means to learn meaningful relationships among symmetry patterns.
method Introduces attentive group equivariant convolutions, applying attention during convolution.
result Consistently outperforms conventional group convolutional networks on benchmark datasets.
We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equivariant convolutions utilize kernels which are parameterized as a …
Neural networks adapt to any input dimensionality.
problem Learning functions with inputs of varying dimensions.
method Equivariant neural networks using representation stability.
result Training on fixed dimensions allows extension to any input.
Conditions for equivariant bundles on 4-manifolds with cyclic actions.
problem Existence of equivariant bundles on 4-manifolds with cyclic actions.
method Conditions derived from the twisted signature formula and congruence relations between fixed point data and isotropy representations.
result Necessary and sufficient conditions for the existence of equivariant bundles.
New approach identifies latent properties from mechanisms, not just data.
problem Identifying latent properties from data generating processes.
method Equivariance perspective on identifiable representation learning.
result Identification of latent properties is possible up to shared equivariances in known mechanisms.
Harmonic maps depend analytically on representations.
problem Analyzing harmonic maps into symmetric spaces.
method Construction of deformation maps to transform equivariant harmonic maps into a fixed target space.
result Equivariant harmonic maps depend real analytically on the representation.
Proves representability of complex semigroup systems.
problem Representability of systems of proportionally modular numerical semigroups.
method Canonical equivariant resolution of weighted homogeneous surface singularities.
result Every system of proportionally modular numerical semigroups is representable.
NFT learns group actions without knowing the data's structure.
problem Learning equivariant representations without knowing the data's structure.
method Neural Fourier Transform (NFT) framework for learning latent linear actions of groups.
result Linear equivariant features are equivalent to group invariants.
Autoencoder learns group representations from actions, improving future prediction accuracy.
problem Learning internal models of interactions with the real world.
method Homomorphism autoencoder with group representation trained on equivariance-derived loss.
result Agents can predict future actions with improved accuracy.
EbC learns equivariant embeddings from unlabeled group actions.
problem Learning equivariant embeddings from unlabeled group actions.
method Equivariance by Contrast (EbC) method to learn equivariant embeddings from observation pairs (y,g⋅y). result High-fidelity equivariance in latent space for diverse groups.
The study quantifies how many objects can be linearly classified under all views.
problem Understanding the expressivity of group-equivariant representations.
method Generalization of Cover's Function Counting Theorem to quantify separable dichotomies.
result The fraction of separable dichotomies is determined by the fixed space dimension of the group action.
Equivariant neural networks improve performance and generalization in lattice field theory tasks.
problem Improving neural network performance and generalization in lattice field theory.
method Investigation of translationally equivariant neural networks in a two-dimensional scalar field model.
result Equivariant neural networks significantly outperform non-equivariant ones in various tasks, including physical parameters and lattice sizes.
Study fibrations of projective spaces for maximal representations.
problem Characterize maximal representations of surface groups.
method Analyze fibrations of projective spaces and use geometric structures.
result Maximal representations correspond to fibrations with specific properties.
Constructs a functor for equivariant smooth h-cobordisms.
problem Defines a functor for equivariant smooth h-cobordisms.
method Constructs an (∞,1)-functor mapping smooth G-manifolds to spaces of equivariant h-cobordisms. result The functor structure is subtle and relies on new ideas.
Non-Abelian actions are resolved using equivariant K-theory and delocalized cohomology.
problem Resolving non-Abelian actions on manifolds.
method Using equivariant K-theory and delocalized cohomology, the structure of the quotient space is described.
result A new model for non-Abelian equivariant K-theory and cohomology is developed.
Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architect…
SymPE breaks symmetries in equivariant networks, improving performance across various tasks.
problem Equivariant networks cannot break symmetries, leading to poor performance in tasks with symmetrical inputs.
method Novel equivariant conditional distributions and randomized canonicalization.
result SymPE significantly improves performance of group-equivariant and graph neural networks.
Weisfeiler and Leman enhance graph learning for machine learning tasks.
problem Learning from graph data in machine learning.
method Weisfeiler and Leman algorithm applied to graph and node representation learning.
result The algorithm improves graph and node representation learning in machine learning.
We compute the equivariant complex K-theory ring of a cohomogeneity-one action of a compact Lie group at the level of generators and relations and derive a characterization of K-theoretic equivariant formality for these actions. Less explicit expressions survive for a range of equivariant cohomology theories including …
Category theory enhances understanding of group-equivariant neural networks.
problem Understanding and working with group-equivariant neural networks.
method Application of category theory to tensor power spaces of Rn for groups Sn, O(n), Sp(n), and SO(n). result New insights and an algorithm for computing equivariant linear layers.
We classify SO(n)-equivariant principal bundles over Sn in terms of their isotropy representations over the north and south poles. This is an example of a general result classifying equivariant (Π,G)-bundles over cohomogeneity one manifolds.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.