Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
Introduces a novel spatial attention module for convolutional networks.
problem Irregular boundaries in position-wise spatial attention maps hamper model generalization.
method Introduces a convolutional rectangular attention module with 5 parameters.
result Systematically outperforms position-wise counterparts in experiments.
STCA discovers dynamic functional brain networks using spatial-temporal convolution and attention.
problem Lack of dynamic exploration of functional brain networks.
method Spatial-Temporal Convolutional Attention (STCA) model.
result STCA can discover dynamic functional brain networks in a novel way.
ST-SAN predicts flow with spatial-temporal dependencies using self-attention.
problem Challenges in predicting flow due to spatial-temporal dependencies.
method Spatial-Temporal Self-Attention Network (ST-SAN) that addresses temporal and spatial dependencies.
result Significant improvement in flow prediction accuracy (9% in inflow, 4% in outflow) compared to state-of-the-art methods.
Hybrid model integrates GATv2 and geostatistics for better spatial prediction and uncertainty.
problem Accurate spatial prediction and uncertainty quantification in epidemiology and risk analysis.
method Integrates Graph Attention Network (GATv2) with model-based geostatistics (MBG) to capture relational and spatial dependencies.
result Hybrid model improves predictive accuracy and uncertainty quantification compared to standalone models.
Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition. For spatial attention, we learn a saliency mask to allow the model to focus…
Study compares deep learning models for traffic forecasting, highlighting graph elements' impact.
problem Challenges in forecasting spatial-temporal traffic patterns.
method In-depth comparative study of four deep neural network models with different basic elements.
result Graph attention improves long-term predictions in traffic forecasting models.
SPACE models complex scenes by decomposing objects and backgrounds.
problem Scalability and unsupervised object-oriented scene representation learning.
method Generative latent variable model combining spatial-attention and scene-mixture approaches.
result SPACE achieves factorized object representations and decomposes complex scenes.
Simpler CNN model with spatial attention and temporal pooling outperforms complex models.
problem Emotion recognition from videos with small face deformations and identity variations.
method Spatial attention mechanism and temporal softmax pooling applied to a pre-trained CNN.
result The approach achieves higher accuracy than state-of-the-art methods on the EmotiW dataset.
Enhanced deep learning model forecasts household leverage series accurately.
problem Forecasting household leverage series due to complex temporal-spatial dynamics.
method TSEN model with multiple RNN-based layers and an attention layer.
result Captures temporal-spatial dynamics and provides more accurate predictions.
A3T-GCN improves traffic forecasting by capturing spatial and temporal dependencies.
problem Accurate real-time traffic forecasting in complex road networks.
method Attention Temporal Graph Convolutional Network (A3T-GCN) integrating recurrent units and graph convolutional network.
result Improved prediction accuracy through attention mechanism and global temporal information.
DSTP-RNN improves long-term multivariate time series prediction using attention-based RNN.
problem Long-term prediction of multivariate time series with spatial correlations and spatio-temporal relationships.
method Inspired by human attention mechanism, DSTP-RNN uses a dual-stage two-phase structure and multiple attentions to enhance spatial correlations and long-term dependence.
result DSTP-RNN outperforms nine baseline methods on four datasets in energy, finance, environment, and medicine.
ATFM predicts traffic flow using attention mechanism and neural networks.
problem Predicting traffic flow with diverse factors integration.
method Unified neural network (ATFM) with attention mechanism and ConvLSTM units.
result ATFM outperforms in predicting citywide short-term/long-term traffic flow.
New results on splitting tangles and spatial graphs.
problem Issues with previous splitting results about tangles and spatial graphs.
method Generalization of Menasco's result to other classes of links, tangles, and spatial graphs.
result New more general results for tangles and spatial graphs.
This is a survey article for the forthcoming `A Concise Encyclopedia of Knot Theory.' We focus on the topology of spatial graphs with few vertices and edges, paying particular attention to Brunnian θ-graphs.
SANST uses self-attentive networks with spatial and temporal embeddings for better POI recommendations.
problem Next point-of-interest (POI) recommendation for users based on their history.
method SANST incorporates spatio-temporal patterns into self-attentive networks.
result SANST outperforms state-of-the-art models by up to 13.65% in nDCG@10.
GACAN combines multi-granularity time series for traffic forecasting.
problem High dynamics and complex spatial-temporal dependency of road networks in traffic forecasting.
method Graph Attention-Convolution-Attention Networks (GACAN) with Att-Conv-Att (ACA) block.
result GACAN outperforms state-of-the-art baselines in traffic forecasting.
End-to-end image super-resolution using Attention-based DenseNet with residual deconvolution.
problem Challenging task of improving low-resolution images.
method Proposes a novel ADRD model with weighted dense blocks and spatial attention modules.
result Demonstrates promising performance on publicly available datasets.
Tiled Squeeze-and-Excite improves channel attention with local spatial context.
problem Improving channel attention mechanisms in neural networks.
method Proposes tiled squeeze-and-excite (TSE) framework for channel attention.
result Local context of 7 rows or columns is sufficient for matching global context performance.
CutMix training technique improves spatial locality in Vision Transformers.
problem Improving spatial locality in Vision Transformers trained from scratch.
method Comparison of Baseline and Modern training protocols on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
result CutMix training component significantly reduces Mean Attention Distance (MAD) in early layers of Vision Transformers.
Model traffic congestion events using multi-modal data and attention-based neural networks.
problem Capture non-homogeneous temporal and directional spatial dependencies in traffic congestion events.
method Attention-based neural networks for point processes, adapted tail-up model for spatial statistics.
result Superior performance compared to state-of-the-art methods on synthetic and real data.
GSANet improves semantic segmentation accuracy with selective and global attention.
problem Semantic segmentation accuracy improvement.
method Global and selective attention mechanism with ASPP and sparsemax.
result GSANet achieves state-of-the-art accuracy on ADE20k and Cityscapes datasets.
Discrete-AIR model identifies objects in images with interpretable latent codes.
problem Identifying objects in images without labeled data.
method Recurrent Auto-Encoder with structured latent distributions for discrete, continuous, and spatial attention.
result Discrete-AIR model uses minimal latent variables for efficient inference.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Diffusion Transformer captures spatial-temporal dependencies in sequential data.
problem Capturing rich spatial and temporal dependencies in sequential data.
method Established theoretical guarantees for diffusion transformers learning Gaussian process data.
result Spatial-temporal dependencies are captured within attention layers of diffusion transformers.
Person Re-Identification (person re-id) is a crucial task as its applications in visual surveillance and human-computer interaction. In this work, we present a novel joint Spatial and Temporal Attention Pooling Network (ASTPN) for video-based person re-identification, which enables the feature extractor to be aware of …
Proposes a transformer model with geostatistical inductive bias for spatio-temporal forecasting.
problem Combining probabilistic rigor of geostatistics with flexible deep learning representations.
method Spatially-informed transformer with learnable covariance kernel.
result Successfully recovers spatial decay parameters end-to-end via backpropagation.
EFA extends self-attention to handle mixed data types and dynamic relevance.
problem Handling high-dimensional, mixed data types with dynamic relevance.
method Probabilistic generative model using self-attention and latent factor model.
result EFA consistently outperforms existing models in complex latent structure capture and reconstruction.
Researchers develop a new spatial process model for non-Gaussian data.
problem Non-Gaussian spatial data with asymmetry and heavy-tailedness.
method Re-parameterized Unified Skew-Normal (SUN) distribution, GSUN process, neural Bayes inference with GATs.
result GSUN process captures non-Gaussian spatial data properties and outperforms conventional models.
MRA-BGCN improves traffic forecasting accuracy through complex graph interactions.
problem Challenging traffic forecasting due to spatial-temporal dependency and uncertainty.
method Proposes MRA-BGCN, a deep learning model that uses bicomponent graph convolution and multi-range attention.
result MRA-BGCN achieves state-of-the-art results on real-world traffic datasets.
Kriformer uses graph transformers to estimate data in sparse sensor areas.
problem Sparse sensor deployment and unreliable data in spatiotemporal kriging tasks.
method Graph transformer model with positional encoding and attention mechanisms.
result Kriformer excels in representing unobserved locations in spatiotemporal kriging tasks.
STG2Seq predicts multi-step passenger demand with graph and hierarchical structure.
problem Predicting passenger demand over multiple time horizons is challenging due to nonlinear and dynamic spatial-temporal dependencies.
method Proposes a graph-based model with a hierarchical graph convolutional structure to capture spatial and temporal correlations.
result Consistently outperforms baseline and state-of-the-art models on real-world datasets.
Two prediction models improve supply-demand forecasting for autonomous vehicles.
problem Improving accuracy and stability of supply-demand predictions for autonomous vehicles.
method Two prediction models based on residual network, LSTM, attention mechanism, and multi-attention mechanism.
result Our frameworks provide more accurate and stable prediction results than existing methods.
A video-based re-identification method using attention mechanisms.
problem Associating videos of the same person from different cameras.
method Siamese framework with attention mechanisms for spatial and temporal information.
result Achieves better performance than state-of-the-art on iLIDS-VID dataset.
EAGLE-Net enhances foundation models by integrating patch-level features for better tissue understanding.
problem Foundation models lack mechanisms for global tissue structure and local context in computational pathology.
method EAGLE-Net combines multi-scale spatial encoding, attention-guided loss functions, and background suppression to aggregate patch-level features into slide-level predictions.
result EAGLE-Net improves classification accuracy and concordance indices across multiple cancer types, producing biologically coherent attention maps.
GOAT improves attention mechanisms by learning better priors.
problem Standard attention mechanisms use a naive uniform prior, limiting flexibility and generalization.
method GOAT introduces a trainable, continuous prior that replaces the uniform assumption, maintaining compatibility with optimized kernels.
result GOAT avoids representational trade-offs and learns an extrapolatable prior that combines positional flexibility with length generalization.
GATGPT uses LLMs with graph attention for spatiotemporal data imputation.
problem Missing values in spatiotemporal data due to sensor malfunctions and data transmission errors.
method Integrates pre-trained large language models with graph attention mechanisms.
result GATGPT achieves comparable results to deep learning benchmarks on real-world datasets.
SCRAM accelerates image attention computation to O(n log(n)) from O(n^2).
problem Efficiently computing attention maps for large images.
method SCRAM uses PatchMatch to quickly find key patches and then approximates non-local mean operations.
result SCRAM achieves O(n log(n)) time complexity for image attention computation.
A scalable Bayesian linear regression framework for spatial data.
problem Scalable methodologies for analyzing large spatial datasets.
method Conjugate Bayesian linear regression framework.
result Exact sampling from joint posterior distribution without iterative algorithms.
Traffic flow prediction is crucial for urban traffic management and public safety. Its key challenges lie in how to adaptively integrate the various factors that affect the flow changes. In this paper, we propose a unified neural network module to address this problem, called Attentive Crowd Flow Machine~(ACFM), which …
Image clustering is an important but challenging task in machine learning. As in most image processing areas, the latest improvements came from models based on the deep learning approach. However, classical deep learning methods have problems to deal with spatial image transformations like scale and rotation. In this p…
Multistep traffic forecasting on road networks is a crucial task in successful intelligent transportation system applications. To capture the complex non-stationary temporal dynamics and spatial dependency in multistep traffic-condition prediction, we propose a novel deep learning framework named attention graph convol…
Graph Attention Networks predict power outage durations from natural disasters.
problem Accurately predicting power outage durations from geospatial and weather data.
method Graph Attention Networks (GAT) for semi-supervised learning.
result GAT model outperforms existing methods by 2% - 15% in accuracy.
Model classifies environment sounds using multiple feature channels and attention mechanisms.
problem Environment sound classification task.
method Multiple feature channels (MFCC, GFCC, CQT, Chromagram) and attention mechanism in a deep CNN.
result Achieves state-of-the-art performance on three benchmark datasets.
MetNet forecasts precipitation up to 8 hours with high spatial and temporal resolution.
problem Precise weather forecasting for long lead times.
method Neural network architecture using axial self-attention for global context aggregation.
result MetNet outperforms Numerical Weather Prediction at forecasts of up to 8 hours.
Language models trained on chess board states outperform those on moves, even with causal masking.
problem Applying causal masking to spatial data for training unimodal language models.
method Trained bidirectional and causal self-attention models on both spatial (board-based) and sequential (move-based) chess data.
result Models trained on spatial board states achieve stronger playing strength than those trained on sequential data, even with causal masking.
GraphDINO learns neuronal morphologies from unlabeled data.
problem Unsupervised learning of neuronal morphologies from unlabeled data.
method Transformer-based approach with novel attention mechanism and data augmentation.
result GraphDINO yields morphological clusterings on par with expert classification.
Deep learning model predicts traffic flows across entire network for multiple steps ahead.
problem Accurately forecasting future traffic flows across all network links.
method Spatial-Temporal Sequence to Sequence (STSeq2Seq) model combining seq2seq and graph convolution.
result STSeq2Seq achieves state-of-the-art performance in traffic forecasting.