kth-order invariant graph networks are as powerful as kth-order WL in distinguishing graphs.
problem Measuring the expressive power of graph neural network formalisms.
method Considered kth-order invariant graph networks (k-IGNs) and compared their expressive power to kth-order WL.
result k-IGNs and k-WL are equally powerful in distinguishing graphs.
Survey on GNNs' power and limitations.
problem Theoretical limitations of GNNs.
method Comprehensive overview of GNNs and their variants.
result Provably powerful variants of GNNs.
New MPNNs match 2-WL, faster distinguishing graphs.
problem Improving graph neural network expressiveness.
method Introducing ℓ-walk MPNNs and second-order GNNs. result Walk MPNNs match 2-WL and can distinguish graphs faster.
Geometric GNNs improve graph discrimination through GWL.
problem Discriminating geometric graphs embedded in Euclidean space.
method Proposed a geometric version of the Weisfeiler-Leman test (GWL) for geometric graphs.
result Characterized the expressive power of geometric GNNs based on physical symmetries.
Graph machine learning lacks a balanced theory, focusing on expressive power and optimization.
problem Insufficient theoretical understanding of GNNs' generalization behavior.
method Develop a balanced theory focusing on expressive power, generalization, and optimization.
result Theoretical advancements need to align with practical success in graph machine learning.
Paper explores how GNNs can learn graph biconnectivity, finding ESAN is the only known expressive framework.
problem Understanding the expressive power of GNNs beyond the WL test.
method Introduces a novel class of expressivity metrics via graph biconnectivity and develops the GD-WL approach.
result GD-WL consistently outperforms prior GNN architectures in learning biconnectivity metrics.
GNNs with random node initialization are shown to be universally expressive.
problem Limitations of standard GNNs in distinguishing graphs.
method Random node initialization (RNI) to enhance GNNs' expressive power.
result GNNs with RNI are proven to be universally expressive.
Graph Neural Networks (GNNs) have achieved much success on graph-structured data. In light of this, there have been increasing interests in studying their expressive power. One line of work studies the capability of GNNs to approximate permutation-invariant functions on graphs, and another focuses on the their power as…
We investigate graph neural networks for multi-relational data.
problem Understanding and improving graph neural networks for multi-relational data.
method Aligning Relational GCN and Compositional GCN with the Weisfeiler-Leman test to understand their expressive power and introduce a new k-RN architecture. result The k-RN architecture overcomes the expressiveness limitations of Relational GCN and Compositional GCN. Graph Neural Networks outperform the Weisfeiler-Lehman algorithm in representation power.
problem Limited representation power of Graph Neural Networks compared to the Weisfeiler-Lehman algorithm.
method Algebraic analysis using eigenvalue decomposition of graph operators.
result Graph Neural Networks produce more discriminative representations than the Weisfeiler-Lehman algorithm.
Paper compares expressive power of GNNs, proving approximation guarantees for practical architectures.
problem Understanding the expressive power of Graph Neural Networks (GNNs).
method Theoretical framework comparing invariant and equivariant GNNs, proving approximation guarantees for practical architectures.
result Folklore Graph Neural Networks (FGNN) are the most expressive architectures for a given tensor order.
ESAN improves graph neural networks by processing subgraphs.
problem Limitations of MPNNs in graph isomorphism.
method ESAN represents graphs as sets of subgraphs and processes them with equivariant architectures.
result ESAN increases the expressive power of GNNs and more expressive architectures.
While graph kernels (GKs) are easy to train and enjoy provable theoretical guarantees, their practical performances are limited by their expressive power, as the kernel function often depends on hand-crafted combinatorial features of graphs. Compared to graph kernels, graph neural networks (GNNs) usually achieve better…
This work characterizes topological descriptors of graph products and their expressive power.
problem Capturing multiscale structural information in graph products using topological descriptors.
method Analysis of various filtrations on graph products, including Euler characteristic and persistent homology.
result Persistent homology of graph products contains more information than individual graphs.
This study compares GNNs and GA-MLPs, finding GA-MLPs can distinguish graphs but not count walks.
problem Comparing expressive power and graph isomorphism testing capabilities of GNNs and GA-MLPs.
method GA-MLPs augment node features with multi-hop operators and apply MLPs node-wise; GNNs are compared as a baseline.
result GA-MLPs can distinguish almost all non-isomorphic graphs but cannot count attributed walks, unlike GNNs.
Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been pro…
PathNNs improve graph neural networks by distinguishing non-isomorphic graphs.
problem Limitations of standard GNNs in distinguishing non-isomorphic graphs.
method Proposes PathNNs that aggregate paths to update node representations.
result Two variants of PathNNs are strictly more powerful than 1-WL algorithm.
CW Networks leverage cell complexes to enhance GNNs, achieving state-of-the-art results on molecular datasets.
problem Graph Neural Networks struggle with long-range interactions and lack principled ways to model higher-order structures.
method CW Networks use cell complexes to decouple computational and input graph structures, enabling flexible hierarchical message passing.
result CW Networks achieve state-of-the-art results on molecular datasets.
Graph Substructure Networks (GSN) improves GNN expressivity by counting subgraph isomorphisms.
problem Limited expressivity of GNNs in detecting and counting graph substructures.
method Topologically-aware message passing scheme based on substructure encoding.
result GSN is strictly more expressive than the Weisfeiler-Leman (WL) test and can disambiguate even hard graph isomorphism instances.
Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressive power of graph neural networks (GNN). It was shown that the popular message passing GNN cannot distinguish between graphs that are indistinguishable by the 1-WL test (Morris et al. 2018; Xu et al. 2019). Unfortunately, many s…
New framework extends graph neural networks by improving expressiveness and space efficiency.
problem Limitations of existing graph neural networks, particularly in expressive power and space complexity.
method Proposed (k,t)-FWL+, a flexible and powerful framework that combines (k,t)-FWL and k-FWL+. result Demonstrated N2-FWL, a practical and theoretically sound instance of (k,t)-FWL+, achieving record-breaking results on ZINC-Subset. Enhanced GNN with expanded attention window and partially random embeddings.
problem Limited expressivity of traditional GNNs in distinguishing non-isomorphic graphs.
method Graph attention network with expanding attention window and partially random initial embeddings. Head dropout for regularization.
result Improved ability to differentiate between non-isomorphic graphs.
Graph convolutional networks (GCNs) are powerful tools for graph-structured data. However, they have been recently shown to be vulnerable to topological attacks. To enhance adversarial robustness, we go beyond spectral graph theory to robust graph theory. By challenging the classical graph Laplacian, we propose a new c…
Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of g…
Graph neural networks (GNNs) have emerged recently as a powerful architecture for learning node and graph representations. Standard GNNs have the same expressive power as the Weisfeiler-Leman test of graph isomorphism in terms of distinguishing non-isomorphic graphs. However, it was recently shown that this test cannot…
New analysis shows over-squashing limits GNNs' power.
problem Understanding the expressive power of GNNs, especially under over-squashing.
method Quantitative measure of pairwise interactions in MPNNs, proving capacity requirements.
result Over-squashing hinders GNNs' ability to learn complex node feature functions.
Weisfeiler-Leman struggles with graph isomorphism; enhanced architectures improve generalization.
problem Graph isomorphism problem and limited expressivity of 1-WL. method Augmenting 1-WL and MPNNs with subgraph information, employing margin theory, and introducing provable generalization kernels. result Increased expressivity of graph neural networks and kernels does not necessarily correlate with improved generalization performance.
This work explores the relationship between expressivity and generalization in GNNs.
problem Understanding the trade-off between expressivity and generalization in GNNs.
method Introducing a novel framework that connects GNN generalization to the variance in graph structures they can capture.
result Theoretical findings align with empirical results, offering a deeper understanding of how expressivity enhances GNN generalization.
Explains differences between WL and folklore-WL formulations in graph neural networks.
problem Understanding the differences between WL and folklore-WL formulations in graph neural networks.
method Visual explanation of differences between WL and folklore-WL formulations.
result Clarifies the differences between WL and folklore-WL formulations.
This paper studies the expressive power of graph neural networks falling within the message-passing framework (GNNmp). Two results are presented. First, GNNmp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness. Second, it is discovered that GNNm…
The paper investigates how Global Self-attention improves GCNs.
problem Improving expressive power and addressing overfitting in GCNs.
method Applying Global Self-attention mechanism over graph features.
result GSA mechanism enhances expressive power and mitigates overfitting and over-smoothing in GCNs.
Graph neural networks (GNNs) are a powerful tool to learn representations on graphs by iteratively aggregating features from node neighbourhoods. Many variant models have been proposed, but there is limited understanding on both how to compare different architectures and how to construct GNNs systematically. Here, we p…
NePTuNe combines neural and tensor methods for efficient link prediction in knowledge graphs.
problem Incomplete knowledge graphs, especially in link prediction.
method Hybrid model combining neural and tensor factorization methods.
result NePTuNe achieves state-of-the-art performance on FB15K-237 and near state-of-the-art on WN18RR datasets.
To deepen our understanding of graph neural networks, we investigate the representation power of Graph Convolutional Networks (GCN) through the looking glass of graph moments, a key property of graph topology encoding path of various lengths. We find that GCNs are rather restrictive in learning graph moments. Without c…
Inspired by the immense success of deep learning, graph neural networks (GNNs) are widely used to learn powerful node representations and have demonstrated promising performance on different graph learning tasks. However, most real-world graphs often come with high-dimensional and sparse node features, rendering the le…
Graph neural networks (GNNs) are powerful machine learning models for various graph learning tasks. Recently, the limitations of the expressive power of various GNN models have been revealed. For example, GNNs cannot distinguish some non-isomorphic graphs and they cannot learn efficient graph algorithms. In this paper,…
Attention-based GNNs can't prevent oversmoothing, leading to homogeneous node representations.
problem The issue of oversmoothing in attention-based GNNs.
method Viewed attention-based GNNs as nonlinear time-varying dynamical systems and used tools from the theory of products of inhomogeneous matrices and the joint spectral radius.
result Graph attention mechanism cannot prevent oversmoothing and loses expressive power exponentially.
Improves hypergraph link prediction by breaking symmetry.
problem Limited expressivity of GWL-1 algorithm in hypergraph link prediction.
method Preprocessing algorithm to identify and replace symmetry-inducing subhypergraphs with covering hyperedges.
result Improves expressivity of GWL-1, leading to better link prediction.
DE improves GNNs by distinguishing graph substructures, enhancing accuracy.
problem Limited expressive power of GNNs in representing graph substructures.
method Introduces Distance Encoding (DE) to assist GNNs in distinguishing graph substructures.
result DE distinguishes graph substructures that traditional GNNs cannot, improving accuracy.
Training-free GNNs use labels as features to improve node classification.
problem Improving graph neural networks for transductive node classification.
method Advocates labels as features, designs training-free GNNs based on this.
result Training-free GNNs outperform traditional GNNs in node classification.
Many problems in machine learning can be cast as learning functions from sets to graphs, or more generally to hypergraphs; in short, Set2Graph functions. Examples include clustering, learning vertex and edge features on graphs, and learning features on triplets in a collection. A natural approach for building Set2Graph…
MV-GNN improves molecular property prediction by integrating atom and bond information.
problem Accurately predicting molecular properties using graph neural networks.
method Multi-View Graph Neural Network (MV-GNN) architecture with shared self-attentive readout and cross-dependent message passing.
result MV-GNN achieves superior performance on molecular property prediction benchmarks.
The paper connects GNNs to VC dimension theory to study their generalization performance.
problem Understanding GNNs' ability to make meaningful predictions beyond the training set.
method Using Vapnik-Chervonenkis (VC) dimension theory in two settings: no upper bound on graph order and known upper bound.
result Tight connections between GNNs' bitlength, number of colors, and VC dimension in different settings.
This paper explores GNN functions on random graphs, highlighting the importance of node Positional Encodings.
problem Understanding the expressive power of GNNs on large random graphs.
method General convergence notions, input node features, and Positional Encodings (PEs).
result GNNs can converge to certain functions on large random graphs, emphasizing the role of PEs.
New architectures improve topological deep learning's ability to capture complex data features.
problem Current TDL architectures struggle with fundamental topological and metric invariants.
method Developed multi-cellular networks (MCN) and scalable MCN (SMCN) to enhance expressivity.
result SMCN outperforms HOMP and expressive graph methods in learning topological properties.
We use Nathanson's g-adic representation of integers to relate metric properties of Cayley graphs of the integers with respect to various infinite generating sets S to problems in additive number theory. If S consists of all powers of a fixed integer g, we find explicit formulas for the smallest positive intege…
Recent advancements in graph representation learning have led to the emergence of condensed encodings that capture the main properties of a graph. However, even though these abstract representations are powerful for downstream tasks, they are not equally suitable for visualisation purposes. In this work, we merge Mappe…
Graph neural networks over-smooth when layers increase, reducing discriminative power.
problem Over-smoothing in graph neural networks reduces model performance as the number of layers increases.
method Analyzed over-smoothing in general graph neural network architecture using Dirichlet energy.
result The Dirichlet energy of embeddings converges to zero, leading to loss of discriminative power.