THGFM models dynamic relational systems with cross-type and temporal fusion.
problem Learning on temporal heterogeneous graphs with diverse node and relation types.
method Dual-Path Architecture with Shared-Space and Relational Type-Partitioned Temporal Attention.
result THGFM outperforms baseline models on academic graph benchmarks.
Paper proposes new methods for improving interatomic potentials.
problem Limitations of conventional SO(2) Linear architectures in MLIPs.
method Direct Cartesian construction, recursive Clebsch-Gordan construction, Edge Complex Product Basis, Radial Rotary Complex Attention.
result TECE-OAM-RRA-1.0 achieves SOTA performance on Matbench Discovery.
DRFormer uses dynamic tokenization and multi-scale transformer to forecast long time series.
problem Forecasting long-term time series data across diverse scales.
method Dynamic tokenizer, multi-scale transformer, dynamic sparse learning, rotary position encoding.
result DRFormer outperforms existing methods in forecasting accuracy.
Deep learning diagnoses rotary machine faults without expert input.
problem Early detection of faults in rotary machinery to save time and money.
method Deep Convolutional Neural Network with three axis accelerometer signal input.
result High classification accuracy in fault diagnosis.
ElasTST improves time-series forecasting across varying horizons.
problem Robust forecasting across different time horizons in varied industrial sectors.
method Elastic Time-Series Transformer (ElasTST) with non-autoregressive design, rotary position embedding, and multi-scale patching.
result ElasTST provides robust forecasts across varying horizons without retraining.
This paper introduces a new task to better understand Transformers in quantitative contexts.
problem Understanding Transformers in high-stakes quantitative and scientific applications.
method Introduces a novel contextual counting task and analyzes it with causal and non-causal Transformer architectures.
result Causal attention is better suited for the contextual counting task, and no positional embeddings lead to the best accuracy.
ST-SAN predicts flow with spatial-temporal dependencies using self-attention.
problem Challenges in predicting flow due to spatial-temporal dependencies.
method Spatial-Temporal Self-Attention Network (ST-SAN) that addresses temporal and spatial dependencies.
result Significant improvement in flow prediction accuracy (9% in inflow, 4% in outflow) compared to state-of-the-art methods.
STRING improves 2D and 3D position encodings for better performance.
problem Efficient and accurate position encoding for 2D and 3D applications.
method STRING extends Rotary Position Encodings with a unifying theoretical framework, maintaining translation invariance and low computational cost.
result STRING shows substantial gains in open-vocabulary object detection and robotics.
TSAM predicts directed temporal links using GCN and self-attention.
problem Predicting links in directed temporal networks.
method GCN, self-attention mechanism, autoencoder architecture, graph attentional layers, graph convolutional layers, graph recurrent unit layer.
result TSAM outperforms benchmarks on four realistic networks.
Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition. For spatial attention, we learn a saliency mask to allow the model to focus…
Enhances SNNs for spatio-temporal feature extraction.
problem Insufficient temporal dependencies in existing SNN synaptic structures.
method Integrates temporal convolution and attention mechanisms into synaptic connections.
result Improves SNN performance on classification tasks.
A new neural network learns from acoustic scenes by suppressing irrelevant patterns.
problem Acoustic scenes are rich and redundant, making classification challenging.
method Spatio-temporal attention pooling layer coupled with a convolutional recurrent neural network.
result The method outperforms a strong convolutional neural network baseline and sets new state-of-the-art performance.
MEANTIME improves sequential recommendation by using multi-temporal embeddings and attention mechanisms.
problem Limited use of timestamp information and information bottleneck in sequential recommendation models.
method MEANTIME employs multiple types of temporal embeddings and attention mechanisms to capture diverse patterns from user behavior sequences.
result MEANTIME outperforms state-of-the-art sequential recommendation methods.
Model infers temporal connections in dynamic graphs from node interactions.
problem Challenges in reasoning about evolving graphs, especially with human-specified edges.
method Temporal point processes and variational autoencoders with bilinear interactions.
result Model outperforms baselines and infers semantically interpretable connections.
Self-attentive network improves emotion recognition in conversations.
problem Emotion recognition in dyadic conversations using deep learning.
method Introduces a novel self-attention mechanism for capturing temporal dynamics without a decoder.
result Outperforms state-of-the-art alternatives on the IEMOCAP benchmark.
STCA discovers dynamic functional brain networks using spatial-temporal convolution and attention.
problem Lack of dynamic exploration of functional brain networks.
method Spatial-Temporal Convolutional Attention (STCA) model.
result STCA can discover dynamic functional brain networks in a novel way.
Two prediction models improve supply-demand forecasting for autonomous vehicles.
problem Improving accuracy and stability of supply-demand predictions for autonomous vehicles.
method Two prediction models based on residual network, LSTM, attention mechanism, and multi-attention mechanism.
result Our frameworks provide more accurate and stable prediction results than existing methods.
SANST uses self-attentive networks with spatial and temporal embeddings for better POI recommendations.
problem Next point-of-interest (POI) recommendation for users based on their history.
method SANST incorporates spatio-temporal patterns into self-attentive networks.
result SANST outperforms state-of-the-art models by up to 13.65% in nDCG@10.
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
Federated learning interprets temporal dynamics across clients with graph attention.
problem Interpreting temporal patterns across decentralized, heterogeneous systems with nonlinear dynamics.
method Graph Attention Network for learning state transition models over latent states communicated between clients.
result First interpretable characterization of cross-client temporal interdependencies in decentralized nonlinear systems.
Paper introduces a conformer-based system for streaming language identification in long-form speech.
problem Language identification in long-form audio.
method Conformer layers with attentive temporal pooling and domain adaptation.
result Conformer-based models significantly outperform LSTM and transformer models.
RTFN extracts robust temporal features for time series analysis.
problem Challenges in extracting sufficient shapelets from time series data.
method Combines temporal feature networks and attentional LSTM networks.
result RTFN outperforms in supervised and unsupervised time series analysis.
A3T-GCN improves traffic forecasting by capturing spatial and temporal dependencies.
problem Accurate real-time traffic forecasting in complex road networks.
method Attention Temporal Graph Convolutional Network (A3T-GCN) integrating recurrent units and graph convolutional network.
result Improved prediction accuracy through attention mechanism and global temporal information.
TGAT learns node embeddings for evolving graphs, capturing both static and temporal features.
problem Learning node embeddings for dynamic graphs with evolving topological structures and temporal patterns.
method Temporal Graph Attention (TGAT) layer using self-attention and functional time encoding.
result TGAT model can inductively infer node embeddings for new and observed nodes as the graph evolves.
Simpler CNN model with spatial attention and temporal pooling outperforms complex models.
problem Emotion recognition from videos with small face deformations and identity variations.
method Spatial attention mechanism and temporal softmax pooling applied to a pre-trained CNN.
result The approach achieves higher accuracy than state-of-the-art methods on the EmotiW dataset.
New framework predicts urban traffic with high accuracy.
problem Urban traffic prediction challenges.
method Interpretable attention-based neural network combining multiple modules.
result Framework outperforms state-of-the-art alternatives.
There are three equivalent ways of representing two jointly observed real-valued signals: as a bivariate vector signal, as a single complex-valued signal, or as two analytic signals known as the rotary components. Each representation has unique advantages depending on the system of interest and the application goals. I…
Develops models for temporally abstract reasoning and attention.
problem Temporal abstraction and attention in reinforcement learning.
method Defines affordances for options and develops partial option models.
result Identifies trade-offs between estimation and approximation error.
Paper proposes deep learning model for dynamic stock repurchase forecasting.
problem Complex temporal dependencies in corporate financial conditions.
method Hybrid Temporal Convolutional Network (TCN) and Attention-based LSTM.
result Model significantly outperforms static baselines in stock repurchase forecasting.
Proposes a time-aware attention model for CTR prediction.
problem Lack of temporal signals in existing CTR prediction models.
method Time-aware attention model with absolute and relative temporal signals, regularized adversarial sampling.
result Significantly improves CTR prediction performance.
Innovative ball bearing converts rotary to reciprocating motion.
problem Designing efficient motion conversion mechanisms.
method Closed curve envelopment theory and diameter-stroke ratio concept.
result Compact and vibration-reduced ball bearing design.
FATHOM model improves sensor data analysis with attention and LSTM.
problem Scarcity of training data from multiple sensors.
method Federated multi-task hierarchical attention model (FATHOM) with attention mechanism and LSTM.
result FATHOM outperforms baselines in sensor data classification and regression.
Sentinel improves time series forecasting by modeling both temporal and channel dependencies.
problem Limited effectiveness of existing transformer-based architectures in multivariate time-series forecasting.
method Proposes Sentinel, a full transformer-based architecture with multi-patch attention mechanism.
result Sentinel achieves better or comparable performance compared to state-of-the-art approaches.
Enhanced deep learning model forecasts household leverage series accurately.
problem Forecasting household leverage series due to complex temporal-spatial dynamics.
method TSEN model with multiple RNN-based layers and an attention layer.
result Captures temporal-spatial dynamics and provides more accurate predictions.
A new model captures complex event data using attention and Fourier kernels.
problem Capturing complex non-linear temporal dependencies in discrete event data.
method Integrates attention mechanism into point processes' conditional intensity function and uses Fourier kernel embedding.
result Established theoretical properties and demonstrated competitive performance.
DSTP-RNN improves long-term multivariate time series prediction using attention-based RNN.
problem Long-term prediction of multivariate time series with spatial correlations and spatio-temporal relationships.
method Inspired by human attention mechanism, DSTP-RNN uses a dual-stage two-phase structure and multiple attentions to enhance spatial correlations and long-term dependence.
result DSTP-RNN outperforms nine baseline methods on four datasets in energy, finance, environment, and medicine.
GAttNHP predicts future events in temporal knowledge graphs by encoding long-range dependencies and handling mutual excitation.
problem Forecasting future events in temporal knowledge graphs due to long-range dependencies, mutual excitation, and heavy-tailed inter-arrival times.
method GAttNHP uses a self-attention encoder, semantic soft-grouping, and NCQ regression to address these issues.
result GAttNHP improves entity and time prediction on six benchmark TKG datasets compared to state-of-the-art baselines.
Study compares deep learning models for traffic forecasting, highlighting graph elements' impact.
problem Challenges in forecasting spatial-temporal traffic patterns.
method In-depth comparative study of four deep neural network models with different basic elements.
result Graph attention improves long-term predictions in traffic forecasting models.
New method embeds time span into self-attention for better temporal pattern recognition.
problem Capturing temporal patterns in event sequences without recurrent networks.
method Functional time representation learning with Bochner's and Mercer's Theorems.
result Proposed methods outperform baseline models in various continuous-time event sequence prediction tasks.
Unified framework analyzes and compares RFF and RoPE PEs for music generation.
problem Efficiently modeling music generation with positional encodings.
method Kernel methods to analyze and compare RFF and RoPE PEs.
result RoPEPool outperforms other methods in melody harmonization.
Graph Neural Networks improve volatility prediction in financial markets.
problem Traditional models struggle with complex, non-linear interdependencies in financial markets.
method Temporal Graph Attention Network (Temporal GAT) combines GCNs and GATs to capture dynamic graph structures.
result Temporal GAT outperforms traditional GARCH models in volatility forecasting, especially for short- to mid-term predictions.
Paper proposes HGTAN for better stock trend prediction.
problem Predicting stock price trends is challenging and crucial for investors.
method Temporal-relational hypergraph tri-attention network (HGTAN).
result HGTAN outperforms existing methods in stock trend prediction.
GACAN combines multi-granularity time series for traffic forecasting.
problem High dynamics and complex spatial-temporal dependency of road networks in traffic forecasting.
method Graph Attention-Convolution-Attention Networks (GACAN) with Att-Conv-Att (ACA) block.
result GACAN outperforms state-of-the-art baselines in traffic forecasting.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Deep learning detects protective movement behavior in chronic pain patients.
problem Detecting protective movement behavior in chronic pain patients for intervention.
method End-to-end deep learning architecture named BodyAttentionNet (BANet) that learns temporal and bodily parts.
result Statistically significant improvements in detecting protective behavior using attention mechanisms.
CASTNet forecasts opioid overdoses using crime patterns.
problem Forecasting opioid overdose occurrences.
method Community-attentive spatio-temporal networks incorporating multi-head attention.
result Superior forecasting performance and interpretable community contributions.
TFT improves multi-horizon forecasting with interpretable insights.
problem Complex multi-horizon forecasting with mixed inputs.
method Attention-based Temporal Fusion Transformer combining recurrent and self-attention layers.
result Significant performance improvements over existing benchmarks.