GRAPH-BERT uses only attention for graph representation learning.
problem Graph neural networks over-rely on graph links and suffer from performance issues.
method GRAPH-BERT uses only attention mechanism without graph convolution or aggregation, trained on sampled subgraphs.
result GRAPH-BERT outperforms existing GNNs in learning effectiveness and efficiency.