GAT-RWOS uses graph attention to improve imbalanced data classification.
problem Imbalanced data leads to biased models favoring majority classes.
method Combines GATs and random walks to generate synthetic minority samples.
result Improves classification performance on imbalanced datasets.
RoGAT enhances GAT robustness against adversarial attacks.
problem Vulnerability of GAT to adversarial attacks.
method Dynamic adjustment of edge weights and features, with an extra attention score.
result RoGAT outperforms other defensive methods in robustness tests.
While many recent advances in deep reinforcement learning (RL) rely on model-free methods, model-based approaches remain an alluring prospect for their potential to exploit unsupervised data to learn environment model. In this work, we provide an extensive study on the design of deep generative models for RL environmen…
H-GAT improves stock selection by capturing complex higher-order stock relations and integrating both technical and fundamental analysis.
problem Stock selection difficulty and lack of comprehensive analysis.
method Higher-order Graph Attention Network (H-GAT) that incorporates both technical and fundamental analysis.
result H-GAT outperforms existing methods in stock selection metrics.
GAT-GMM improves GANs' performance in learning Gaussian mixture models.
problem GANs struggle with multi-modal distributions like Gaussian mixtures.
method Proposes a minimax GAN framework using random linear generator and softmax-based quadratic discriminator.
result Gradient Descent Ascent method converges to an approximate minimax point.
Machine learning models that can exploit the inherent structure in data have gained prominence. In particular, there is a surge in deep learning solutions for graph-structured data, due to its wide-spread applicability in several fields. Graph attention networks (GAT), a recent addition to the broad class of feature le…
New interpretation of attention in Transformers and Graph Attention Networks.
problem Understanding and improving attention mechanisms in deep learning models.
method Decomposed attention into a kernel and a normalization term; generalized the kernel function and norm.
result Generalized attention leads to better performance on various tasks.
Graph Attention Networks (GATs) are the state-of-the-art neural architecture for representation learning with graphs. GATs learn attention functions that assign weights to nodes so that different nodes have different influences in the feature aggregation steps. In practice, however, induced attention functions are pron…
Graph Neural Networks improve volatility prediction in financial markets.
problem Traditional models struggle with complex, non-linear interdependencies in financial markets.
method Temporal Graph Attention Network (Temporal GAT) combines GCNs and GATs to capture dynamic graph structures.
result Temporal GAT outperforms traditional GARCH models in volatility forecasting, especially for short- to mid-term predictions.
New graph representation learning network improves scalability and feature integration.
problem Scalability and feature integration in graph neural networks for large, dense graphs.
method Adaptive sampling of neighbours based on weighted multi-step transition probabilities.
result Comparable or better results on various graph benchmarks.
GATs improve node regression on noisy graphs with provable advantage.
problem Improving node regression on graphs with noisy covariates and edges.
method Proposes a GAT designed for denoising proxy features in node regression.
result GAT achieves lower error in estimating regression coefficient and predicting responses.
GAT-AGNN learns stock trends using graph and attention mechanisms.
problem Predicting dynamic stock trends in a complex market.
method Sequential graph structure with attention mechanisms.
result GAT-AGNN outperforms state-of-the-art methods in stock trend prediction.
Graph Neural Networks (GNNs) have proved to be an effective representation learning framework for graph-structured data, and have achieved state-of-the-art performance on many practical predictive tasks, such as node classification, link prediction and graph classification. Among the variants of GNNs, Graph Attention N…
Lipschitz normalization boosts deep attention models, especially for graph neural networks.
problem Gradient explosion in deep graph attention networks leads to poor performance.
method Enforcing Lipschitz continuity by normalizing attention scores.
result Deep GAT models with LipschitzNorm achieve state-of-the-art results for tasks with long-range dependencies.
A new method uses GATs to optimise portfolios of mid-cap firms, outperforming traditional methods.
problem Optimising portfolios of mid-cap firms considering interdependencies and firms at risk of default.
method Graph Attention Networks (GATs) applied to large-scale financial data.
result The GAT-based portfolio outperforms traditional benchmarks over a long period.
Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, e.g. graph convolutional networks (GCN) and graph attention networks (GAT), adequately utilize edge features, especially multi-dimensional edge features. In this paper, we build…
Driven by the outstanding performance of neural networks in the structured Euclidean domain, recent years have seen a surge of interest in developing neural networks for graphs and data supported on graphs. The graph is leveraged at each layer of the neural network as a parameterization to capture detail at the node le…
Graph Attention Networks improve image classification with superpixels.
problem Classifying images with irregular shapes and edges.
method Transform images into superpixel graphs, then apply GATs.
result GATs outperform other GNN models in image classification.
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over thei…
Graph attention is not always beneficial; conditions for perfect node classification are identified.
problem Understanding when graph attention mechanisms improve node classification performance.
method Theoretical analysis using Contextual Stochastic Block Models (CSBMs).
result Graph attention mechanisms are more effective when structure noise exceeds feature noise, and simpler graph convolution operations are better when feature noise predominates.
A decentralized routing framework for lunar exploration robots.
problem Routing data in intermittent connectivity lunar networks.
method Graph Attention-based Multi-Agent Reinforcement Learning (GAT-MARL).
result Higher delivery rates, no duplications, fewer packet losses.
SpGAT learns graph representations using spectral attention for efficiency.
problem Efficiently capturing global graph patterns with minimal parameters.
method Introduces Spectral Graph Attention Network (SpGAT) using spectral domain attention mechanisms and a fast Chebychev approximation.
result SpGAT achieves better global pattern recognition with fewer parameters compared to GAT.
Graph Attention Networks predict power outage durations from natural disasters.
problem Accurately predicting power outage durations from geospatial and weather data.
method Graph Attention Networks (GAT) for semi-supervised learning.
result GAT model outperforms existing methods by 2% - 15% in accuracy.
Proposes a novel node embedding framework for graphs using Fisher Information.
problem Lack of theoretical understanding of attention-based GNNs.
method Uses hierarchical kernels and Fisher Information to learn node embeddings.
result Proposed method outperforms existing GNNs on node classification benchmarks.
I-AID categorizes disaster tweets into useful information types.
problem Filtering useful information from social media during disasters.
method Multimodel approach using BERT, GAT, and Relation Network.
result I-AID outperforms state-of-the-art approaches in F1 scores.
BGNN improves GNN by modeling interactions between neighbor nodes.
problem Existing GNN models fail to capture interactions between neighbor nodes, leading to suboptimal performance.
method Proposes a new graph convolution operator that augments the weighted sum with pairwise interactions of neighbor nodes.
result Empirical results show BGNN models outperform traditional GNN models in node classification accuracy.
LASE learns graph embeddings by unrolling GD iterations into a neural network.
problem Learning graph embeddings from graph inputs.
method Truncating and interpreting GD iterations as layers in a GNN.
result LASE outperforms eigendecomposition routines and GNNs in link prediction and node classification.
In recent years, there has been a surge of interest in developing deep learning methods for non-Euclidean structured data such as graphs. In this paper, we propose Dual-Primal Graph CNN, a graph convolutional architecture that alternates convolution-like operations on the graph and its dual. Our approach allows to lear…
Novel neural framework for scalable community detection and link prediction.
problem Efficiently learning graph topology and community structure in large graphs.
method Proposes a joint SBM loss function for community detection and link prediction, leveraging neural networks.
result Framework efficiently handles large graphs with a single forward pass, improving scalability and performance.
We present an attention mechanism inspired from definition of screened Coulomb potential. This attention mechanism was used to interpret the Graph Attention (GAT) model layers and training dataset by using a flexible and scalable framework (CoulGAT) developed for this purpose. Using CoulGAT, a forest of plain and resne…
The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify o…
BLISS optimizes GNN training by adaptively sampling nodes.
problem High computational costs in training GNNs on large graphs.
method Uses Bandit Layer Importance Sampling to dynamically select nodes.
result Improves GNN performance with reduced computational cost.
Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics systems, learning molecular fingerprints, predicting protein interface, and classifying diseases demand a model to learn from graph inputs. In other domains such as learning from non-structur…
FastGAT reduces GNN computation time by 10x using graph sparsification.
problem High computational burden in attention-based GNNs.
method Spectral sparsification to generate optimal graph pruning.
result Per-epoch time is almost linear in graph nodes, reducing computational time by up to 10x.
In this paper, from a theoretical perspective, we study how powerful graph neural networks (GNNs) can be for learning approximation algorithms for combinatorial problems. To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap b…
Community detection has long been an important yet challenging task to analyze complex networks with a focus on detecting topological structures of graph data. Essentially, real-world graph data contains various features, node and edge types which dynamically vary over time, and this invalidates most existing community…
Unified view of GNNs as graph signal denoising.
problem Understanding and improving GNNs for graph data.
method Established GNNs as graph denoising problems with smoothness assumptions.
result Unified framework UGNN for adaptive smoothness graphs.
Graph transformers outperform graph convolutions by preserving community information.
problem Understanding why graph transformers perform well in node-level prediction tasks.
method Analyzing the Gaussian process limits of graph transformers with infinite width and infinite heads.
result Graph transformers maintain discriminative node representations even in deep layers, preventing oversmoothing.
The recent advancements in graph neural networks (GNNs) have led to state-of-the-art performances in various applications, including chemo-informatics, question-answering systems, and recommender systems. However, scaling up these methods to huge graphs, such as social networks and Web graphs, remains a challenge. In p…
Deep learning model optimizes portfolios by integrating news sentiment, stock relationships, and price data.
problem Optimizing portfolio weights using traditional methods introduces instability.
method Combines LSTM, GAT, and sentiment analysis in a unified pipeline.
result Delivers higher cumulative returns and Sharpe ratios compared to benchmarks.
Attention-based GNNs can't prevent oversmoothing, leading to homogeneous node representations.
problem The issue of oversmoothing in attention-based GNNs.
method Viewed attention-based GNNs as nonlinear time-varying dynamical systems and used tools from the theory of products of inhomogeneous matrices and the joint spectral radius.
result Graph attention mechanism cannot prevent oversmoothing and loses expressive power exponentially.
Feature Quantization improves GAN training stability.
problem Stability issues in GAN training.
method Feature Quantization (FQ) for the discriminator, embedding true and fake data into a shared discrete space.
result FQ-GAN achieves new state-of-the-art performance on various GAN tasks.
Enhances graph neural networks by considering feature similarities in node aggregation.
problem Ignoring node feature similarities in traditional graph aggregation schemes.
method Interprets node aggregation as kernel weighting, proposing a framework that considers feature similarities.
result Proposed framework outperforms traditional GCNs in real-world applications.
Scattering GCN improves graph neural networks by filtering oversmoothing.
problem Oversmoothing in GCNs limits their ability to distinguish graph nodes.
method Augmenting GCNs with geometric scattering transforms and residual convolutions.
result Scattering GCN outperforms GAT in semi-supervised node classification.
The vulnerabilities of deep neural networks against adversarial examples have become a significant concern for deploying these models in sensitive domains. Devising a definitive defense against such attacks is proven to be challenging, and the methods relying on detecting adversarial samples are only valid when the att…
Enhances GNNs by improving input data quality from topology and labels.
problem Poor quality of graph data limits GNN performance.
method Improves graph data quality using model outputs for better semi-supervised node classification.
result SEG consistently improves GNN performance across various datasets.
New algorithm for training GNNs with learned weights.
problem Optimal sampling for GNNs with learned weights is intractable.
method Formulated as an adversary bandit problem, optimizing exploration and exploitation.
result Asymptotically approaches optimal variance within a factor of 3.
Ripple Walk Training tackles graph neural network training issues for large and deep graphs.
problem Neighbors explosion, node dependence, and oversmoothing in large and deep GNNs.
method Subgraph-based training framework with Ripple Walk Sampler for high-quality subgraph sampling.
result RWT improves training efficiency and reduces space complexity for deep and large GNNs.