Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
Measures equivariance in vision models using Lie derivative.
problem Understanding the role of equivariance in recent vision models.
method Introducing Lie derivative to measure equivariance with strong mathematical foundations and minimal hyperparameters.
result Many violations of equivariance can be linked to spatial aliasing in network layers, and larger models tend to display more equivariance.
This work introduces a method for almost equivariance in neural networks using Lie algebra convolutions.
problem Real-world data often does not conform to strict group equivariances, leading to underperformance in models.
method Definition and practical implementation of almost equivariance through Lie algebra convolutions.
result Demonstrated the validity of the approach through benchmarking against fully equivariant settings.
Develops approximately equivariant neural processes for better data modeling.
problem Real-world data often breaks exact equivariance; how to model this?
method General approach to creating approximately equivariant architectures, applicable to any model and symmetry group.
result Approximately equivariant neural processes outperform non-equivariant and strictly equivariant models in regression tasks.
Sign equivariant networks improve model expressiveness for spectral geometric learning.
problem Limited expressiveness of sign invariant models for tasks like graph link prediction.
method Developed sign equivariant neural network architectures based on new analytic sign equivariant polynomials.
result Sign equivariant models achieve theoretical benefits in spectral geometric learning tasks.
New model improves field learning with improved equivariance.
problem Learning equivariant stochastic fields.
method Equivariant Gaussian processes and Steerable Conditional Neural Processes.
result SteerCNPs significantly improve performance in transfer learning tasks.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
RCNPs extend equivariant neural processes to higher dimensions, improving performance on tasks with inherent symmetries.
problem Inherently equivariant tasks in spatio-temporal modeling, Bayesian Optimization, and continuous control.
method Relational Conditional Neural Processes (RCNPs) that extend equivariances to higher dimensions.
result Empirically competitive performance on tasks with equivariances.
Equivariant diffusion model generates 3D molecules efficiently.
problem Generating high-quality 3D molecules efficiently.
method Equivariant Diffusion Model (EDM) that operates on atom coordinates and types.
result Significantly outperforms previous methods in molecule quality and training efficiency.
SE(3)-Transformers maintain equivariance for 3D data under rotations and translations.
problem Ensuring stable and predictable performance in 3D data under transformations.
method Introducing a self-attention module that is equivariant under continuous 3D roto-translations.
result The SE(3)-Transformer outperforms non-equivariant and non-attention models on real-world datasets.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
SymPE breaks symmetries in equivariant networks, improving performance across various tasks.
problem Equivariant networks cannot break symmetries, leading to poor performance in tasks with symmetrical inputs.
method Novel equivariant conditional distributions and randomized canonicalization.
result SymPE significantly improves performance of group-equivariant and graph neural networks.
Using deep neural networks that are either invariant or equivariant to permutations in order to learn functions on unordered sets has become prevalent. The most popular, basic models are DeepSets [Zaheer et al. 2017] and PointNet [Qi et al. 2017]. While known to be universal for approximating invariant functions, DeepS…
New model learns graph neural networks equivariant to various transformations.
problem Learning equivariant graph neural networks for complex transformations.
method E(n)-Equivariant Graph Neural Networks (EGNNs) that are computationally efficient and scalable.
result Achieves competitive or better performance without higher-order representations.
Paper proves non-zero generalization boost for equivariant models.
problem Improving generalization in machine learning models.
method Analyzes simplest case of linear models, focusing on invariant/equivariant properties.
result First provably non-zero improvement in generalization for invariant/equivariant models.
In this paper, for a compact Lie group action,we prove the anomaly formula and the functoriality of the equivariant Bismut-Cheeger eta forms with perturbation operators when the equivariant family index vanishes. In order to prove them, we extend the Melrose-Piazza spectral section and its main properties to the equiva…
The paper shows how data augmentation and regularization can enforce group equivariance in machine learning models.
problem Improving model performance by leveraging known symmetries in machine learning tasks.
method Training with data augmentation and regularization to enforce group equivariance.
result Equivariance of the trained model can be achieved through training on augmented data in tandem with regularization.
A refined form of the `Folk Theorem' that a smooth action by a compact Lie group can be (canonically) resolved, by iterated blow up, to have unique isotropy type was established by the authors in the context of manifolds with corners; the canonical construction induces fibrations on the boundary faces of the resolution…
Improved set prediction model using multiset-equivariant operations and approximate implicit differentiation.
problem Existing set prediction models struggle with multisets and cannot represent certain functions.
method Introduced multiset-equivariance, improved DSPN with approximate implicit differentiation, and applied to CLEVR object property prediction.
result Significantly improved object property prediction on CLEVR dataset.
Equivariant MuZero improves generalization in procedurally generated environments.
problem Leveraging world models for robust generalization in real-world scenarios.
method Incorporates symmetries of the environment into MuZero's world-model architecture, making MuZero equivariant to certain groups of transformations.
result Improves data efficiency and generalization capabilities, demonstrated on procedurally generated MiniPacman and Chaser.
Develops a theory for equivariant networks with partial domain symmetry.
problem Limited analysis of equivariant networks with partial domain symmetry.
method Proposes pointwise definitions of correct, incorrect, and extrinsic equivariance.
result Establishes error lower bounds for networks with partial symmetry.
Equivalence proven between equivariant K-theory and K-homology for certain matrix group actions.
problem Establishing Poincaré duality for proper cocompact matrix group actions.
method Using equivariant K-theory and K-homology, with geometric models of Baum and Douglas.
result Poincaré duality holds between equivariant K-theory and K-homology for G-spinc manifolds with compact quotient. Paper develops equivariant basic cohomology for Lie groupoids.
problem Equivariant cohomology for Lie groupoids with weak actions.
method Using Kan fibrations and fiber structures, constructing models and comparing with existing theories.
result Equivariant basic cohomology theory for orbifolds and Lie groupoids.
New robustness measure accounts for task-specific symmetries.
problem Traditional robustness measures fail for tasks with inherent symmetries.
method Sound notion of adversarial robustness for equivariant tasks, using randomized smoothing and graph edit distance certificates.
result Provable robustness can be achieved for various tasks with inherent symmetries.
Unified method for CNNs to approximate equivariant maps across various groups.
problem Limited universal approximation theorems for CNNs with specific groups and settings.
method Unified approach to derive universal approximation theorems for equivariant maps by CNNs in diverse settings.
result Ability to handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
The study shows symmetry improves machine learning generalization.
problem Improving machine learning generalization through symmetry.
method Using an averaging operator to prove equivariance reduces test risk.
result Equivariant predictors reduce test risk compared to non-equivariant ones.
Tests for equivariance in non-parametric regression models.
problem Detecting false assumptions of symmetry in regression models.
method Develops tests for G-equivariance independent of the model. result Confidence in using equivariant models when symmetry is unknown.
The study examines how equivariance in networks affects generalization error using PAC-Bayesian bounds.
problem Understanding how equivariance in networks impacts generalization error.
method Utilized PAC-Bayesian analysis for equivariant networks, deriving norm-based bounds for generalization error.
result The bound indicates that using larger group size in the model improves generalization error.
The paper explores how equivariant models' biases affect latent representations for better performance.
problem The impact of inductive biases on latent representations in equivariant models.
method Demonstrates the importance of accounting for inductive biases in latent representations of equivariant models.
result Effective invariant projections can be used to retain information in latent representations, improving downstream tasks.
L-CNNs preserve gauge symmetry in lattice simulations.
problem Breaking gauge symmetry in neural network models.
method Lattice gauge equivariant convolutional neural networks (L-CNNs).
result L-CNNs represent gauge invariant functions on the lattice.
SymDiff uses stochastic symmetrisation for equivariant diffusion models.
problem Constructing equivariant diffusion models for data augmentation.
method Stochastic symmetrisation for lightweight, efficient, and easy-to-implement equivariance.
result SymDiff achieves significant empirical benefit for E(3)-equivariant molecular generation. New method breaks symmetry in neural networks, improving sample efficiency.
problem Symmetry in neural networks limits their ability to learn unique features.
method Introduces 'relaxed equivariance' to overcome symmetry limitations.
result Equivariant multilayer perceptrons (E-MLPs) can now break symmetry at the sample level.
Our work improves VAE latent space clustering by enforcing invariant and equivariant learning.
problem Current VAEs fail to learn invariant and equivariant clusters in latent space.
method We use a mixture model pdf like Gaussian mixtures to enforce deep, group-invariant learning and separate semantic and equivariant variables.
result Our model effectively learns to disentangle invariant and equivariant representations, improving learning rate and image recognition.
This paper explores how enforcing equivariance constraints limits neural network expressivity and proposes compensatory model size increases.
problem The impact of enforcing equivariance constraints on the expressive power of neural networks.
method Examined 2-layer ReLU networks, analyzed boundary hyperplanes and channel vectors, and constructed upper bounds on model size required for compensation.
result Enforcing equivariance constraints reduces the expressive power of neural networks, but this can be compensated by increasing model size.
Unified classification of equivariant principal bundles using higher homotopy theory.
problem Unified classification of equivariant principal bundles.
method Smooth Oka principle, singular-cohesive homotopy theory, internally describing principal bundles.
result Unified classification results for equivariant principal bundles.
New translation equivariant neural processes improve spatio-temporal data modeling.
problem Improving posterior prediction maps for spatio-temporal data.
method Introduced translation equivariant transformers within neural processes.
result TE-TNPs outperform non-equivariant TNPs and other baselines.
An impossibility result shows limitations in learning symmetries and equivariant functions.
problem Learning symmetries and equivariant functions simultaneously is impossible under certain conditions.
method Careful study of approximation for groups and semigroups, analysis of neural networks.
result Linearly equivariant networks can be used to learn equivariant functions, but group-convolutional networks have limitations.
The `Folk Theorem' that a smooth action by a compact Lie group can be (canonically) resolved, by iterated blow up, to have unique isotropy type is proved in the context of manifolds with corners. This procedure is shown to capture the simultaneous resolution of all isotropy types in a `resolution tower' which projects …
RPPs improve deep learning models with soft equivariance constraints.
problem Balancing expressiveness and inductive biases in deep learning.
method Introducing Residual Pathway Priors (RPPs) to convert hard constraints into soft priors.
result RPPs enable models to learn structured solutions while retaining flexibility.
IsoGCNs learn invariant and equivariant graph features for efficient simulations.
problem Learning isometric transformation invariant and equivariant features in graphs for simulations.
method Transformation invariant and equivariant Graph Convolutional Networks (IsoGCNs).
result IsoGCNs outperform state-of-the-art methods on geometrical and physical simulation tasks.
Non-Abelian actions are resolved using equivariant K-theory and delocalized cohomology.
problem Resolving non-Abelian actions on manifolds.
method Using equivariant K-theory and delocalized cohomology, the structure of the quotient space is described.
result A new model for non-Abelian equivariant K-theory and cohomology is developed.
Improves U-Net for scale equivariance in semantic segmentation.
problem Improving generalization in semantic segmentation tasks with varying scales.
method Introduces Scale Equivariant U-Net (SEU-Net) with carefully applied subsampling and upsampling layers and scale-equivariant layers.
result Significantly improved generalization to different scales compared to U-Net and scale-equivariant architecture without upsampling.
We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equivariant convolutions utilize kernels which are parameterized as a …
Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding…
Equivariant neural network simplifies particle physics models.
problem Complexity and interpretability in particle physics classification.
method Lorentz group equivariant neural network architecture.
result Simplified, interpretable models with fewer parameters.
We describe the Cartan and Weil models of twisted equivariant cohomology together with the Cartan homomorphism among the two, and we extend the Chern-Weil homomorphism to the twisted equivariant cohomology. We clarify that in order to have a cohomology theory, the coefficients of the twisted equivariant cohomology must…
We investigate an equivariant generalization of Morse theory for a general class of integrable models. In particular, we derive equivariant versions of the classical Poincaré-Hopf and Gauss-Bonnet-Chern theorems and present the corresponding path integral generalizations. Our approach is based on equivariant cohomology…
The paper investigates how symmetry in models affects their performance and generalization.
problem Understanding how symmetry in models impacts their performance and generalization.
method Formal unified investigation of intuitions about symmetry in models and data.
result Quantitative bounds and comparisons between model and data equivariance lead to optimal model performance.