Derives formulae for general permutation equivariant layers and presents a second order graph variational encoder.
problem Tackles the limitation of previous equivariant neural networks by considering permutations of matrices.
method Derives formulae for general permutation equivariant layers, including matrix permutations. Presents a second order graph variational encoder.
result Latent distribution of equivariant generative models must be exchangeable.
Using deep neural networks that are either invariant or equivariant to permutations in order to learn functions on unordered sets has become prevalent. The most popular, basic models are DeepSets [Zaheer et al. 2017] and PointNet [Qi et al. 2017]. While known to be universal for approximating invariant functions, DeepS…
Permutation-equivariant neural networks improve auction mechanisms by reducing regret and sample complexity.
problem Designing optimal auction mechanisms that balance revenue and bidders' regret.
method Introduced permutation-equivariant neural networks to auction mechanisms.
result Permutation-equivariant neural networks decrease expected ex-post regret and improve model generalizability.
SMP model preserves proximity and permutation in graph neural networks.
problem Challenges in graph mining, such as community and leader finding.
method Stochastic Message Passing (SMP) model that maintains proximity and permutation-equivariance.
result SMP model effectively preserves node proximities and permutation-equivariance.
New neural networks respect symmetries in symmetric tensors, improving efficiency and generalization.
problem Learning from symmetric tensors efficiently and respecting their inherent symmetries.
method Developed two characterizations of linear permutation equivariant functions between symmetric power spaces of R^n.
result These functions are highly data efficient compared to standard MLPs and generalize well to different sizes of symmetric tensors.
Graphs benefit from unique node identifiers but lose permutation-equivariance. We propose a method to maintain this property.
problem Graph Neural Networks' inherent limitations due to message-passing structure.
method Propose permutation-equivariant UID models and a contrastive loss to regularize them.
result Our method improves generalization and convergence, achieving state-of-the-art performance.
New neural network architecture for auction design exploiting permutation symmetry.
problem Designing incentive-compatible auctions that maximize expected revenue.
method Constructed a permutation-equivariant neural network architecture.
result Permutation-equivariant architectures can perfectly recover optimal mechanisms.
ExNODE uses ODE to model sets with permutation equivariance.
problem Capturing intra-set dependencies in unordered sets.
method Exchangeable Neural ODE (ExNODE) using ODE.
result ExNODE achieves permutation equivariance for set modeling.
New model preserves symmetry in multivariate time series, improving performance.
problem Implicit ordering in MTS models violates inherent exchangeability.
method Permutation-equivariant 2D state space model with canonical architecture.
result Eliminates sequential dependency chains and simplifies stability analysis.
Graph neural networks improve systemic risk measures for financial networks.
problem Computing systemic risk measures for graph-structured financial networks.
method Extended permutation equivariant neural networks (X-PENNs) for numerical approximation.
result Graph neural networks outperform other methods in approximating optimal allocations.
Theoretical guarantees for permutation-equivariant QNNs avoid barren plateaus.
problem Excessive local minima and barren plateaus in QNNs training landscapes.
method Designing Sn-equivariant QNNs to encode permutation symmetry. result Equivariant QNNs do not suffer from barren plateaus, quickly reach overparametrization, and generalize well.
New graph foundation models respect symmetries for broader applicability.
problem Tailored graph machine learning architectures limit broader applicability.
method Investigates symmetries for label and feature permutations, proving network universal approximator.
result Universal approximator on multisets respecting node and feature permutations.
New basis for permutation equivariant layers reduces computation costs.
problem Efficiently computing permutation equivariant layers in neural networks.
method Generalized partition algebra basis with low-rank tensors.
result Low-rank tensors enable faster computation compared to orbit basis.
New method connects neural networks to diagrammatic algebra.
problem Constructing permutation equivariant neural networks.
method Schur-Weyl duality between symmetric group and partition algebra.
result Simple diagrammatic method for calculating weight matrices.
Introduces P-tensors for generalized higher-order message passing in graph neural networks.
problem Expanding the expressive power of graph neural networks through higher-order structures.
method Introduces P-tensors to define the most general form of permutation equivariant message passing.
result Achieves state-of-the-art performance on molecular datasets.
Paper presents a novel neural network for MIMO symbol detection.
problem Handling a variable number of users in MIMO systems.
method Recurrent and permutation equivariant neural network architecture with iterative decoding.
result The neural detector outperforms existing methods in accuracy and efficiency.
A new method learns graph distributions invariant to node ordering.
problem Graphs are hard to model due to node ordering invariance issues.
method Score-based generative modeling with permutation equivariant graph neural network.
result The method achieves better or comparable graph generation results.
SpeqNets improve graph neural networks by scaling and adapting to graph sparsity.
problem Graph neural networks struggle with permutation-equivariant functions and scalability to large graphs.
method Introducing sparsity-aware, permutation-equivariant graph networks with heuristics for graph isomorphism.
result Significantly improved predictive performance and reduced computation times compared to existing methods.
Novel neural GP kernels learn stable, flexible covariance structures.
problem Scalable and flexible covariance kernels for Gaussian processes.
method Directly learn kriging coefficients and conditional standard deviations using deep neural architectures exploiting permutation-equivariant structure.
result Improved training stability and data efficiency with expressive, non-stationary kernels.
New translation equivariant neural processes improve spatio-temporal data modeling.
problem Improving posterior prediction maps for spatio-temporal data.
method Introduced translation equivariant transformers within neural processes.
result TE-TNPs outperform non-equivariant TNPs and other baselines.
SE-RRMs solve structured problems like Sudoku and ARC-AGI by enforcing symbol equivariance.
problem Structured reasoning problems like Sudoku and ARC-AGI.
method Symbol-equivariant recurrent reasoning models enforcing permutation equivariance.
result SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize to larger and smaller instances.
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
Enhances graph neural networks with structural message-passing for better generalization.
problem Limited representation power and inability to learn basic graph topological properties.
method Proposes a framework that includes a one-hot encoding of nodes and parametrized message and update functions ensuring permutation equivariance.
result Achieves state-of-the-art results on molecular graph regression on the ZINC dataset.
EDGI improves sample efficiency and generalization in tasks with spatial and temporal symmetries.
problem Sample inefficiency and poor generalization in tasks with geometric symmetries.
method Equivariant Diffuser framework, SE(3)xZxSn-equivariant diffusion model.
result EDGI is more sample efficient and generalizes better than non-equivariant models.
In this paper, we develop a theory about the relationship between G-invariant/equivariant functions and deep neural networks for finite group G. Especially, for a given G-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip G-actions and each affine t…
Graph Neural Networks (GNN) come in many flavors, but should always be either invariant (permutation of the nodes of the input graph does not affect the output) or equivariant (permutation of the input permutes the output). In this paper, we consider a specific class of invariant and equivariant networks, for which we …
New model learns graph neural networks equivariant to various transformations.
problem Learning equivariant graph neural networks for complex transformations.
method E(n)-Equivariant Graph Neural Networks (EGNNs) that are computationally efficient and scalable.
result Achieves competitive or better performance without higher-order representations.
Enhances quantum computing for symmetrical systems, proving a new class of problems.
problem Proving the efficiency of a new quantum computing model for symmetrical systems.
method Introducing equivariant convolutional quantum algorithms tailored for SU(d) symmetries.
result Demonstrates a problem that can be solved efficiently on a new quantum model, suggesting it's not classically simulatable.
A2I Transformer predicts atom energies from coordinates, avoiding heavy featurization.
problem Efficiently predicting atom energies from molecular coordinates with minimal featurization.
method End-to-end model using self-attention, permutation-equivariant.
result Stable predictions with significantly smaller errors than molecular dynamics simulations.
A new training method for efficient Boltzmann generators.
problem Training equivariant continuous normalizing flows (CNFs) is computationally expensive.
method Equivariant flow matching, based on optimal transport flow matching.
result Equivariant flow matching yields more efficient flows with shorter integration paths.
The paper introduces fixed-point centralities for networks and graphons.
problem Defining network centralities for networks and graphons.
method Fixed-point centralities defined via permutation equivariant mappings and graphons.
result Variation bounds of fixed-point centralities under mild assumptions.
We use the equivariant Yang-Mills moduli space to investigate the relation between the singular set, isotropy representations at fixed points, and permutation modules realized by the induced action on homology for smooth group actions on certain 4-manifolds.
Improves modeling of sets with permutation invariant densities.
problem Challenges in calculating trace limit practicality of current methods.
method Proposes an alternative approach to define permutation equivariant transformations with closed form trace.
result Improves both training and final performance.
Proposes supervised method for whole DAG causal structure learning.
problem Learning causal directions from data, especially for whole DAG structure.
method Supervised learning approach using permutation equivariant models.
result Promising results compared to previous approaches on synthetic and real data.
This work relaxes GNN symmetries to approximate automorphisms, improving model performance.
problem Improving graph neural network performance on asymmetric graphs.
method Formalizing approximate symmetries via graph coarsening, introducing a bias-variance formula.
result Best generalization performance achieved by choosing a larger symmetry group than automorphisms but smaller than permutations.
PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.
problem Persistent positional biases and sensitivity to role assignments in fixed topologies.
method Dynamic reconfiguration of agent roles and sparse topologies based on evolving agent states.
result Significantly improves average accuracy over debate baselines across multiple reasoning benchmarks.
New neural networks learn graph symmetries.
problem Learning from graph data without considering vertex relations.
method Constructs equivariant neural networks to Aut(G) group.
result Characterizes learnable, linear, Aut(G)-equivariant functions.
Generative model for set-valued data using permutation invariant flows.
problem Modeling set-valued data with conditional generative models.
method Conditional generative probabilistic model using continuous normalizing flows with permutation equivariant dynamics.
result Significantly outperforms non-permutation invariant baselines in log likelihood and domain-specific metrics.
In this work we propose a new neural network architecture that efficiently implements and learns general purpose set-equivariant functions. Such a function f maps a set of entities x = {x1, . . . , xn} from one domain to a set of same cardinality y = f (x) = {y1, . . . , yn} in another domain regardless of the ordering…
Despite the widespread adoption of Transformer models for NLP tasks, the expressive power of these models is not well-understood. In this paper, we establish that Transformer models are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, which is quite surp…
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
We study the problem of designing models for machine learning tasks defined on \emph{sets}. In contrast to traditional approach of operating on fixed dimensional vectors, we consider objective functions defined on sets that are invariant to permutations. Such problems are widespread, ranging from estimation of populati…
This paper proposes a set of rules to revise various neural networks for 3D point cloud processing to rotation-equivariant quaternion neural networks (REQNNs). We find that when a neural network uses quaternion features under certain conditions, the network feature naturally has the rotation-equivariance property. Rota…
We introduce a simple permutation equivariant layer for deep learning with set structure.This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit sum…
New mesh network preserves symmetries in deep learning.
problem No existing mesh processing architecture is equivariant to all symmetries.
method Equivariant attention-based mesh network using relative tangential features.
result The network achieves improved performance and is equivariant to various transformations.
Improved phylogenetic inference with normalizing flows.
problem Limitations of current diagonal Lognormal branch length approximation in VBPI.
method Proposes VBPI-NF using normalizing flows to handle non-Euclidean branch length space.
result Significantly improves phylogenetic posterior estimation on real data.
New robustness measure accounts for task-specific symmetries.
problem Traditional robustness measures fail for tasks with inherent symmetries.
method Sound notion of adversarial robustness for equivariant tasks, using randomized smoothing and graph edit distance certificates.
result Provable robustness can be achieved for various tasks with inherent symmetries.
We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer φW:ℜM→ℜN, we show that φW is equivariant with respect to G-action iff $\m…