Unified method for CNNs to approximate equivariant maps across various groups.
problem Limited universal approximation theorems for CNNs with specific groups and settings.
method Unified approach to derive universal approximation theorems for equivariant maps by CNNs in diverse settings.
result Ability to handle non-linear equivariant maps between infinite-dimensional spaces for non-compact groups.
Develops approximately equivariant neural processes for better data modeling.
problem Real-world data often breaks exact equivariance; how to model this?
method General approach to creating approximately equivariant architectures, applicable to any model and symmetry group.
result Approximately equivariant neural processes outperform non-equivariant and strictly equivariant models in regression tasks.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
Improved set prediction model using multiset-equivariant operations and approximate implicit differentiation.
problem Existing set prediction models struggle with multisets and cannot represent certain functions.
method Introduced multiset-equivariance, improved DSPN with approximate implicit differentiation, and applied to CLEVR object property prediction.
result Significantly improved object property prediction on CLEVR dataset.
In this paper, we develop a theory about the relationship between G-invariant/equivariant functions and deep neural networks for finite group G. Especially, for a given G-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip G-actions and each affine t…
Equivariant CNNs improve RL performance in symmetric environments.
problem Learning equivariant representations for RL in symmetric environments.
method Proposed and studied equivariant CNNs for RL.
result Equivariant CNNs enhance RL performance and sample efficiency.
New framework uses symmetry-based matrices for efficient, flexible NNs.
problem Designing neural networks with relaxed equivariance.
method Symmetry-based structured matrices, Group Matrices (GMs).
result GMs enable competitive performance with fewer parameters.
Method improves deep learning models for datasets with mixed approximate symmetries.
problem Improving deep learning models for datasets with mixed approximate symmetries.
method Regularizer-based approach to build models for datasets with mixed approximate symmetries.
result Our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
The paper investigates how symmetry in models affects their performance and generalization.
problem Understanding how symmetry in models impacts their performance and generalization.
method Formal unified investigation of intuitions about symmetry in models and data.
result Quantitative bounds and comparisons between model and data equivariance lead to optimal model performance.
An impossibility result shows limitations in learning symmetries and equivariant functions.
problem Learning symmetries and equivariant functions simultaneously is impossible under certain conditions.
method Careful study of approximation for groups and semigroups, analysis of neural networks.
result Linearly equivariant networks can be used to learn equivariant functions, but group-convolutional networks have limitations.
Proposes SE(3) equivariant graph neural networks with local frames for efficient geometric approximation.
problem Equivariance in deep learning for arbitrary transformations, especially in physics.
method Introduces SE(3) equivariant graph neural networks with complete local frames to efficiently approximate geometric quantities.
result Achieves best or competitive performance in Newton mechanics modeling and equilibrium molecule conformation generation.
Using deep neural networks that are either invariant or equivariant to permutations in order to learn functions on unordered sets has become prevalent. The most popular, basic models are DeepSets [Zaheer et al. 2017] and PointNet [Qi et al. 2017]. While known to be universal for approximating invariant functions, DeepS…
Improves U-Net for scale equivariance in semantic segmentation.
problem Improving generalization in semantic segmentation tasks with varying scales.
method Introduces Scale Equivariant U-Net (SEU-Net) with carefully applied subsampling and upsampling layers and scale-equivariant layers.
result Significantly improved generalization to different scales compared to U-Net and scale-equivariant architecture without upsampling.
A new method uses algebraic insights to create approximately equivariant networks without complex architectures.
problem Designing equivariant neural networks with complex architectures and high computational cost.
method Imposes the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters.
result Matches or outperforms specialized models in several cases, even for infinite groups.
We describe the Cartan and Weil models of twisted equivariant cohomology together with the Cartan homomorphism among the two, and we extend the Chern-Weil homomorphism to the twisted equivariant cohomology. We clarify that in order to have a cohomology theory, the coefficients of the twisted equivariant cohomology must…
New method uses scalars to approximate physics functions.
problem Designing neural networks that respect physical symmetries.
method Parameterizing polynomial functions equivariant to various symmetries using scalars.
result Universal approximation of polynomial functions under various symmetries using scalars.
This work relaxes GNN symmetries to approximate automorphisms, improving model performance.
problem Improving graph neural network performance on asymmetric graphs.
method Formalizing approximate symmetries via graph coarsening, introducing a bias-variance formula.
result Best generalization performance achieved by choosing a larger symmetry group than automorphisms but smaller than permutations.
Equivariant networks improve geometric prediction without scalar approximations.
problem Efficiently predicting geometric tensors in real-world scenarios.
method Equivariant networks for geometric prediction.
result Equivariant networks can generalize to unseen systems for geometric prediction.
New equivariant filters improve graph classification.
problem Designing deep learning models for graph symmetries.
method Nonlinear spectral filters (NLSFs) that are equivariant to graph functional shifts.
result NLSFs outperform existing spectral GNNs in graph classification.
New method uses scalar-based models to approximate spherical tensors efficiently.
problem Efficiently approximating spherical tensors with equivariant functions.
method Expressing equivariant functions as the product of a scalar function and a small tensor basis.
result Approximations are fast, simple to implement, and accurate in practical settings.
Survey discusses new ideas in geometric group theory and their applications.
problem Understanding geodesic metric spaces and their equivariant wall structures.
method Introduces and highlights the impact of injective metric spaces and cubical approximation theorem.
result Rich equivariant wall structures in various geodesic metric spaces.
GNP models predictive correlations and outperforms NPs.
problem Training and understanding of Neural Processes.
method Proposed a new model, Gaussian Neural Process (GNP), which incorporates translation equivariance and provides universal approximation guarantees.
result Demonstrates encouraging performance and provides universal approximation guarantees.
L-CNNs learn gauge invariant quantities on lattices.
problem Learning gauge invariant quantities on lattices.
method Novel convolutional layer preserving gauge equivariance and forming Wilson loops.
result L-CNNs can approximate any gauge covariant function on the lattice.
New fusion blocks improve equivariant neural networks for molecular dynamics.
problem Designing equivariant neural networks for tasks with global symmetries.
method Using fusion diagrams from tensor networks to design novel equivariant components.
result Improved performance with fewer parameters on chemical problems.
GNPE improves inference for astrophysical systems.
problem Efficiently incorporating geometric properties like equivariances in neural density estimation.
method GNPE integrates equivariances into neural posterior estimation, standardizing data pose while estimating parameters.
result GNPE achieves state-of-the-art accuracy in astrophysical binary black hole inference, reducing inference times by 3 orders of magnitude.
The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be exte…
Automatically learns flexible symmetry constraints in neural networks using gradients.
problem Fixed hard constraints on neural network functions that cannot be adapted.
method Improves parameterisations of soft equivariance and optimizes marginal likelihood using differentiable Laplace approximations.
result Achieves equivalent or improved performance on image classification tasks compared to baselines with hard-coded symmetry.
RPPs improve deep learning models with soft equivariance constraints.
problem Balancing expressiveness and inductive biases in deep learning.
method Introducing Residual Pathway Priors (RPPs) to convert hard constraints into soft priors.
result RPPs enable models to learn structured solutions while retaining flexibility.
This work exploits action equivariance for representation learning in reinforcement learning. Equivariance under actions states that transitions in the input space are mirrored by equivalent transitions in latent space, while the map and transition functions should also commute. We introduce a contrastive loss function…
New methods improve translation-equivariant neural processes for modeling unknown functions.
problem Modeling unknown latent functions from irregularly sampled measurements.
method Volterra series and set Fourier convolutions to address translation-equivariance and efficiency.
result Improved translation-equivariant neural processes with analytical transparency and linear scalability.
Numerous invariant (or equivariant) neural networks have succeeded in handling invariant data such as point clouds and graphs. However, a generalization theory for the neural networks has not been well developed, because several essential factors for the theory, such as network size and margin distribution, are not dee…
New graph foundation models respect symmetries for broader applicability.
problem Tailored graph machine learning architectures limit broader applicability.
method Investigates symmetries for label and feature permutations, proving network universal approximator.
result Universal approximator on multisets respecting node and feature permutations.
This paper shows how to approximate CAT(-1) representations by Fuchsian ones.
problem Approximating representations of surface groups in CAT(-1) spaces.
method Constructing equivariant maps from the hyperbolic plane to CAT(-1) spaces.
result Every CAT(-1) representation can be approximated by a Fuchsian one.
This work establishes universality for deep equivariant networks, overcoming limitations of previous approaches.
problem Rarity of universality results for equivariant neural networks, especially in high-dimensional settings.
method Develops a more general account of universality for equivariant networks, introducing entry-wise separability and readout layers.
result Deep equivariant networks achieve universality under entry-wise separability, with or without readout layers.
Consider the cyclic group C_2 of order two acting by complex-conjugation on the unit circle S^1. The main result is that a finitely dominated manifold W of dimension > 4 admits a cocompact, free, discontinuous action by the infinite dihedral group D_\infty if and only if W is the infinite cyclic cover of a free C_2-man…
Proposes a topological model for partial equivariance in neural networks.
problem Capturing partial equivariance in neural networks for data analysis.
method Introduces P-GENEOs and studies spaces of measurements and P-GENEOs between them.
result Spaces of measurements and P-GENEOs have convenient approximation and convexity properties.
Despite the widespread adoption of Transformer models for NLP tasks, the expressive power of these models is not well-understood. In this paper, we establish that Transformer models are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, which is quite surp…
Let M be a closed Riemannian manifold carrying an effective and isometric action of a compact connected Lie group G. We derive a refined remainder estimate in the stationary phase approximation of certain oscillatory integrals on T∗M×G with singular critical sets that were examined previously in order…
New method selects equivariant models using uncertainty metrics.
problem Selecting equivariant models among pretrained ones with varying symmetry biases.
method Uncertainty-aware model selection using frequentist, Bayesian, and calibration-based measures.
result Bayesian model evidence often misaligns with predictive performance.
Invariant and equivariant networks have been successfully used for learning images, sets, point clouds, and graphs. A basic challenge in developing such networks is finding the maximal collection of invariant and equivariant linear layers. Although this question is answered for the first three examples (for popular tra…
Graph Neural Networks (GNN) come in many flavors, but should always be either invariant (permutation of the nodes of the input graph does not affect the output) or equivariant (permutation of the input permutes the output). In this paper, we consider a specific class of invariant and equivariant networks, for which we …
In order to facilitate the comparison of Riemannian homogeneous spaces of compact Lie groups with noncommutative geometries ("quantizations") that approximate them, we develop here the basic facts concerning equivariant vector bundles and Dirac operators over them in a way that uses only global constructions and argume…
Graph neural networks improve systemic risk measures for financial networks.
problem Computing systemic risk measures for graph-structured financial networks.
method Extended permutation equivariant neural networks (X-PENNs) for numerical approximation.
result Graph neural networks outperform other methods in approximating optimal allocations.
Construct minimal Lagrangian surfaces in complex projective plane via loop group method.
problem Construct minimal Lagrangian immersions from arbitrary Riemann surfaces into complex projective plane.
method Loop group method, perturbed equivariant minimal Lagrangian surfaces, Delaunay cylinders approximation.
result Construct a class of minimal Lagrangian cylinders approximating Delaunay cylinders.
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
This paper develops optimal transport methods on the roto-translation group SE2.
problem Optimal transport on the roto-translation group SE2 for image analysis.
method Develops a computational framework for optimal transportation over Lie groups, focusing on SE2. Uses Sinkhorn-like algorithm with efficient distance approximations.
result Advances in image barycentric interpolation, orientation field interpolation, and Wasserstein flows on SE2.
Paper compares expressive power of GNNs, proving approximation guarantees for practical architectures.
problem Understanding the expressive power of Graph Neural Networks (GNNs).
method Theoretical framework comparing invariant and equivariant GNNs, proving approximation guarantees for practical architectures.
result Folklore Graph Neural Networks (FGNN) are the most expressive architectures for a given tensor order.
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.