Network embedding represents nodes in a continuous vector space and preserves structure information from the Network. Existing methods usually adopt a "one-size-fits-all" approach when concerning multi-scale structure information, such as first- and second-order proximity of nodes, ignoring the fact that different scal…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “multi-scale node attention”
UGRAPHEMB embeds graphs into vectors preserving their proximity, achieving competitive results.
problem Graph-level representation learning in an unsupervised and inductive manner.
method UGRAPHEMB uses graph-graph proximity to embed graphs into a vector space. MSNA generates multi-scale node attention for graph-level embedding.
result UGRAPHEMB achieves competitive accuracy in graph classification, similarity ranking, and visualization tasks.
Embed nodes with multi-scale attributes for robust network analysis.
problem Capturing complex node attributes across different scales.
method Multi-scale attributed node embedding (AE & MUSAE) using Skip-gram approach.
result Proves node-feature mutual information is implicitly factorized by embeddings.
HKT improves sequence processing with multi-scale attention and kernel analysis.
problem Processing sequences at multiple scales with efficient attention mechanisms.
method Trainable causal downsampling and convex weights for level-specific score matrices.
result HKT achieves consistent gains over standard attention across various tasks.
Boosting theory explains why multi-scale GNNs work.
problem Over-smoothing in graph neural networks.
method Gradient boosting and transductive learning analysis.
result Test error bound decreases with more node aggregations.
Paper proposes a new hierarchical attention mechanism for multi-scale data.
problem Challenges in applying neural attention to multi-scale, multi-modal data.
method Developed a mathematical framework for multi-modal, multi-scale data and derived optimal neural attention mechanics.
result Proposed hierarchical attention mechanism improves transformer performance in multi-scale, multi-modal settings.
New GCNs improve graph classification with deeper multi-scale information.
problem Limited expressive power of existing GCNs.
method Generalized spectral graph convolution and deep GCN architectures, showing equivalence under certain conditions.
result Two new architectures achieve better performance on node classification tasks.
Neural HMM with AGA captures multi-scale dynamics in financial markets.
problem Capturing multi-scale temporal dynamics in financial markets.
method Parallel multi-resolution encoders, adaptive gating, and multi-head attention.
result Outperforms fixed-resolution baselines in predicting price movements and liquidity shocks.
Node-link diagrams are a popular method for representing graphs that capture relationships between individuals, businesses, proteins, and telecommunication endpoints. However, node-link diagrams may fail to convey insights regarding graph structures, even for moderately sized data of a few hundred nodes, due to visual …
Pyramid Attention Networks improve image restoration by leveraging self-similarities across scales.
problem Lack of full exploitation of self-similarities in image restoration by recent deep learning methods.
method Introduces a Pyramid Attention module that processes multi-scale feature correspondences to borrow clean signals from coarser levels.
result Pyramid Attention module achieves state-of-the-art results in various image restoration tasks.
Preformer improves Transformer for long-term time series forecasting.
problem Transformer's quadratic complexity and lack of context-awareness for long-term forecasting.
method Introduces Multi-Scale Segment-Correlation mechanism for efficient time series segmentation and context-aware attention.
result Preformer outperforms other Transformer-based methods in long-term time series forecasting.
Transformer-based multi-scale model outperforms traditional methods in solving PDEs on irregular domains.
problem Solving partial differential equations on irregular domains using deep learning.
method Introduces Multi-Scale Attention Transformer (\msat{}) for solving PDEs.
result Achieves state-of-the-art generalization on complex geometry problems with significant speedup.
New approach learns graph representations by contrasting first-order neighbors and graph diffusion views.
problem Learning node and graph level representations from graph data.
method Self-supervised approach using contrastive learning of multi-scale encodings.
result Achieves state-of-the-art performance on 8 out of 8 benchmarks.
CRAUM-Net: Contextual Recursive Attention with Uncertainty Modeling for Salient Object Detectioncs.LG
CRAUM-Net improves salient object detection with context and uncertainty modeling.
problem Accurate salient object detection with precise boundary delineation.
method Contextual Recursive Attention with Uncertainty Modeling, multi-scale context aggregation, attention mechanisms, edge-aware decoder, Monte Carlo Dropout.
result Superior performance in producing accurate and reliable saliency maps.
Graph attention improves node classification by distinguishing important edges.
problem Node classification in graph-based learning models.
method Theoretical analysis of graph attention networks for node classification.
result Graph attention can perfectly classify nodes in an 'easy' regime but fails in a 'hard' regime.
GSAN learns adaptive node representations using geometric scattering and attention.
problem Oversmoothing in node representation learning.
method Attention-based architecture integrating geometric scattering and GCN channels.
result GSAN outperforms previous networks in semi-supervised node classification.
CoulGAT interprets GAT models by analyzing node interactions.
problem Understanding and interpreting the complex interactions within graph attention networks.
method Developed a CoulGAT framework to analyze and interpret GAT model layers and datasets.
result Extracted node-node and node-feature interactions to define a standard model for graph structure.
CPA models improve GNNs by preserving node cardinality.
problem Limited understanding of attention-based GNNs' discriminative power.
method Theoretical analysis and CPA models to preserve cardinality information.
result CPA models can improve GNNs' performance in node and graph classification.
Proposes a novel node embedding framework for graphs using Fisher Information.
problem Lack of theoretical understanding of attention-based GNNs.
method Uses hierarchical kernels and Fisher Information to learn node embeddings.
result Proposed method outperforms existing GNNs on node classification benchmarks.
HopGAT improves node classification in sparsely labeled graphs by learning from distant neighbors.
problem Classifying nodes in sparsely labeled graphs with limited labeled data.
method Hop-aware supervision mechanism and simulated annealing learning strategy.
result The model achieves high accuracy even with 40% labeled data, reducing performance loss to 3.9%.
New graph attention operators improve performance and reduce computational costs.
problem Excessive computational resources in graph attention operators.
method Proposed hGAO and cGAO using hard and channel-wise attention mechanisms.
result Improved performance and computational savings with new operators.
AdaCAD improves semi-supervised classification by focusing on intra-class nodes.
problem Improving semi-supervised classification by addressing inter-class connections in graphs.
method AdaCAD uses a class-attentive diffusion process to adaptively aggregate nodes based on their class similarity.
result AdaCAD significantly outperforms state-of-the-art methods in semi-supervised classification.
Langevin Dynamics speeds up mixing time with manifold hypothesis and multi-scale approach.
problem Langevin Dynamics struggles in high dimensions and nonconvex landscapes.
method Utilizes manifold hypothesis to reduce mixing time and employs multi-scale approach to improve image generation quality.
result Mixing time depends on intrinsic dimension rather than ambient dimension, significantly reducing computational complexity.
Graph attention auto-encoder reconstructs graph structure and attributes.
problem Lack of methods to reconstruct graph structure and node attributes in graph auto-encoders.
method Stacked encoder/decoder layers with self-attention mechanisms, regularized node representations to reconstruct graph structure.
result Competitive performance on node classification benchmarks, including inductive learning.
Graph attention networks improve performance on heterogeneous graphs.
problem Complex performance of GNNs on heterogeneous graphs.
method Integrating positional encodings into graph attention networks.
result Graph attention networks excel in node classification and link prediction.
SANNE model generates embeddings for unseen nodes in graph networks.
problem Lack of embeddings for unseen nodes in graph networks.
method SANNE uses a transformer self-attention network to generate embeddings.
result SANNE achieves state-of-the-art results for node classification.
MAGNA improves graph neural networks by incorporating multi-hop context information.
problem Limited context in current graph neural networks.
method Diffuses attention scores across the network, accounting for all paths between nodes.
result State-of-the-art performance on node classification and knowledge graph completion benchmarks.
GATs improve node regression on noisy graphs with provable advantage.
problem Improving node regression on graphs with noisy covariates and edges.
method Proposes a GAT designed for denoising proxy features in node regression.
result GAT achieves lower error in estimating regression coefficient and predicting responses.
Proposes MSTD-RCNN for improved financial time-series classification.
problem Combining Multi-Scale and Temporal Dependency for better financial time-series classification.
method Multi-Scale Temporal Dependent Recurrent Convolutional Neural Network (MSTD-RCNN).
result Achieves state-of-the-art performance in trend classification and simulated trading.
We propose the Lanczos network (LanczosNet), which uses the Lanczos algorithm to construct low rank approximations of the graph Laplacian for graph convolution. Relying on the tridiagonal decomposition of the Lanczos algorithm, we not only efficiently exploit multi-scale information via fast approximated computation of…
This paper explains GNNs using graph signal denoising.
problem Understanding how GNNs work for node representation learning.
method Spectral graph convolutional networks and graph attention networks are analyzed from the perspective of graph signal denoising.
result GNNs implicitly solve graph signal denoising problems.
GCL-LRR improves node classification in noisy graphs.
problem Noise in real-world graph data impairs GNNs' effectiveness.
method Two-stage transductive learning with low-rank regularization and attention.
result Improved node classification performance in noisy graphs.
CoMGNN models heterogeneous graphs with evolving nodes and edges.
problem Modeling complex, evolving graphs with diverse information.
method Meta graph attention on co-evolving heterogeneous graphs.
result Significant improvement over state-of-the-art methods.
Graph attention is not always beneficial; conditions for perfect node classification are identified.
problem Understanding when graph attention mechanisms improve node classification performance.
method Theoretical analysis using Contextual Stochastic Block Models (CSBMs).
result Graph attention mechanisms are more effective when structure noise exceeds feature noise, and simpler graph convolution operations are better when feature noise predominates.
FastGAT reduces GNN computation time by 10x using graph sparsification.
problem High computational burden in attention-based GNNs.
method Spectral sparsification to generate optimal graph pruning.
result Per-epoch time is almost linear in graph nodes, reducing computational time by up to 10x.
GSANet improves semantic segmentation accuracy with selective and global attention.
problem Semantic segmentation accuracy improvement.
method Global and selective attention mechanism with ASPP and sparsemax.
result GSANet achieves state-of-the-art accuracy on ADE20k and Cityscapes datasets.
Model infers temporal connections in dynamic graphs from node interactions.
problem Challenges in reasoning about evolving graphs, especially with human-specified edges.
method Temporal point processes and variational autoencoders with bilinear interactions.
result Model outperforms baselines and infers semantically interpretable connections.
EXFormer predicts foreign exchange returns with high accuracy using a multi-scale self-attention mechanism and dynamic variable selection.
problem Accurately forecasting daily exchange rate returns in international finance.
method EXFormer uses a multi-scale trend-aware self-attention mechanism with dynamic variable selection and embedded squeeze-and-excitation blocks.
result EXFormer outperforms other models in forecasting daily exchange rate returns, achieving statistically significant improvements in directional accuracy.
GISST interprets GNNs by combining attention and sparsity for graph structure and node feature importance.
problem Lack of joint consideration of graph structure and node features in GNN interpretation.
method Model-agnostic framework using attention mechanism and sparsity regularization.
result GISST achieves superior node feature and edge explanation precision in synthetic and real-world datasets.
Learning distributed node representations in networks has been attracting increasing attention recently due to its effectiveness in a variety of applications. Existing approaches usually study networks with a single type of proximity between nodes, which defines a single view of a network. However, in reality there usu…
Improves GATs by adding margin-based constraints to prevent over-fitting and over-smoothing.
problem Over-fitting and over-smoothing in GATs.
method Margin-based constraints on attention weights and graph structure.
result Significant improvements over previous GATs on various datasets.
Unified model combines GCN and LPA for better node classification.
problem Combining GCN and LPA for improved node classification.
method Unified model that unifies GCN and LPA, learns edge weights and attention weights.
result Unified model outperforms state-of-the-art GCN-based methods in node classification accuracy.
Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evol…
Paper proposes HGAT for detecting fake news on HIN.
problem Viral spread of fake news causing social harm.
method Hierarchical Graph Attention Network (HGAT) for node representation learning in HIN.
result HGAT outperforms text-based and network-based models.
LATTE tackles heterogeneous network embedding challenges with layer-stacked attention.
problem Aggregating higher-order indirect relations in heterogeneous networks.
method Layer-stacked ATTention Embedding (LATTE) that decomposes meta relations at each layer.
result LATTE achieves state-of-the-art performance on benchmark datasets.
Attentive Neural Controlled Differential Equations for Time-series Classification and Forecastingcs.LG
ANCDEs improve time-series forecasting and classification using attention in NCDEs.
problem Improving time-series forecasting and classification using neural controlled differential equations.
method Integrating attention into neural controlled differential equations (ANCDEs).
result ANCDEs consistently show the best accuracy in time-series classification and forecasting.
Enhanced GNN with expanded attention window and partially random embeddings.
problem Limited expressivity of traditional GNNs in distinguishing non-isomorphic graphs.
method Graph attention network with expanding attention window and partially random initial embeddings. Head dropout for regularization.
result Improved ability to differentiate between non-isomorphic graphs.
Graph Convolutional Networks (GCNs) have shown significant improvements in semi-supervised learning on graph-structured data. Concurrently, unsupervised learning of graph embeddings has benefited from the information contained in random walks. In this paper, we propose a model: Network of GCNs (N-GCN), which marries th…