Paper introduces a conformer-based system for streaming language identification in long-form speech.
problem Language identification in long-form audio.
method Conformer layers with attentive temporal pooling and domain adaptation.
result Conformer-based models significantly outperform LSTM and transformer models.
Acoustic scenes are rich and redundant in their content. In this work, we present a spatio-temporal attention pooling layer coupled with a convolutional recurrent neural network to learn from patterns that are discriminative while suppressing those that are irrelevant for acoustic scene classification. The convolutiona…
Person Re-Identification (person re-id) is a crucial task as its applications in visual surveillance and human-computer interaction. In this work, we present a novel joint Spatial and Temporal Attention Pooling Network (ASTPN) for video-based person re-identification, which enables the feature extractor to be aware of …
Video-based emotion recognition is a challenging task because it requires to distinguish the small deformations of the human face that represent emotions, while being invariant to stronger visual differences due to different identities. State-of-the-art methods normally use complex deep learning models such as recurren…
Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level. Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation. In this work we propose the use …
Hierarchical pretraining with slow-fast ODEs
problem Causal self-attention vs. slow-fast ODEs
method Instantiating fast-slow ODE formalism as a concrete neural network
result Equilibrium manifold x=φ(y) is exactly the master-equation (ME) stationary distribution PiNet improves graph classification efficiency and accuracy.
problem Graph level classification challenges.
method Attention-based pooling mechanism for graph convolution operations.
result Superior performance and high sample efficiency.
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
New approach reduces malware detection memory requirements and speeds up training.
problem Efficiently classifying long sequences of malware detection data.
method Developed a new temporal max pooling method and global channel gating design.
result 116x more memory efficient and 25.8x faster training on original dataset.
Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs. The method of …
Recent studies have demonstrated the power of recurrent neural networks for machine translation, image captioning and speech recognition. For the task of capturing temporal structure in video, however, there still remain numerous open research questions. Current research suggests using a simple temporal feature pooling…
ExNODE uses ODE to model sets with permutation equivariance.
problem Capturing intra-set dependencies in unordered sets.
method Exchangeable Neural ODE (ExNODE) using ODE.
result ExNODE achieves permutation equivariance for set modeling.
Study reveals how attention helps in signal recovery from sequence models using random matrix theory.
problem Signal recovery from sequence models with attention mechanisms.
method Analysis of sample covariance matrices constructed from pooled sequence representations with attention weights.
result Optimal attention weights maximize signal-to-noise ratio and improve signal recovery.
Graph Neural Network (GNN) research has concentrated on improving convolutional layers, with little attention paid to developing graph pooling layers. Yet pooling layers can enable GNNs to reason over abstracted groups of nodes instead of single nodes. To close this gap, we propose a graph pooling layer relying on the …
While deep learning has achieved great success in computer vision and many other fields, currently it does not work very well on patient genomic data with the "big p, small N" problem (i.e., a relatively small number of samples with high-dimensional features). In order to make deep learning work with a small amount of …
ASAP improves graph pooling for hierarchical graph representations.
problem Pooling in graphs fails to effectively capture substructure or scale to large graphs.
method ASAP uses self-attention and modified GNN to capture node importance and learn sparse soft cluster assignments.
result Combining ASAP with GNN architectures leads to state-of-the-art results on graph classification benchmarks.
Graph convolutional networks (GCNs) have recently become one of the most powerful tools for graph analytics tasks in numerous applications, ranging from social networks and natural language processing to bioinformatics and chemoinformatics, thanks to their ability to capture the complex relationships between concepts. …
Sound event detection (SED) methods are tasked with labeling segments of audio recordings by the presence of active sound sources. SED is typically posed as a supervised machine learning problem, requiring strong annotations for the presence or absence of each sound source at every time instant within the recording. Ho…
Distributed representations have been used to support downstream tasks in healthcare recently. Healthcare data (e.g., electronic health records) contain multiple modalities of data from heterogeneous sources that can provide complementary information, alongside an added dimension to learning personalized patient repres…
GSANet improves semantic segmentation accuracy with selective and global attention.
problem Semantic segmentation accuracy improvement.
method Global and selective attention mechanism with ASPP and sparsemax.
result GSANet achieves state-of-the-art accuracy on ADE20k and Cityscapes datasets.
The spatio-temporal graph learning is becoming an increasingly important object of graph study. Many application domains involve highly dynamic graphs where temporal information is crucial, e.g. traffic networks and financial transaction graphs. Despite the constant progress made on learning structured data, there is s…
ST-SAN predicts flow with spatial-temporal dependencies using self-attention.
problem Challenges in predicting flow due to spatial-temporal dependencies.
method Spatial-Temporal Self-Attention Network (ST-SAN) that addresses temporal and spatial dependencies.
result Significant improvement in flow prediction accuracy (9% in inflow, 4% in outflow) compared to state-of-the-art methods.
TSAM predicts directed temporal links using GCN and self-attention.
problem Predicting links in directed temporal networks.
method GCN, self-attention mechanism, autoencoder architecture, graph attentional layers, graph convolutional layers, graph recurrent unit layer.
result TSAM outperforms benchmarks on four realistic networks.
Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition. For spatial attention, we learn a saliency mask to allow the model to focus…
Enhances SNNs for spatio-temporal feature extraction.
problem Insufficient temporal dependencies in existing SNN synaptic structures.
method Integrates temporal convolution and attention mechanisms into synaptic connections.
result Improves SNN performance on classification tasks.
Paper proposes MTL for weakly labelled SED, improving performance with 2-step attention.
problem Weakly labelled sound event detection.
method Multi-Task Learning framework with 2-step Attention Pooling.
result Improved SED performance with 22.3%, 12.8%, 5.9% gains at 0, 10, 20 dB SNR.
MEANTIME improves sequential recommendation by using multi-temporal embeddings and attention mechanisms.
problem Limited use of timestamp information and information bottleneck in sequential recommendation models.
method MEANTIME employs multiple types of temporal embeddings and attention mechanisms to capture diverse patterns from user behavior sequences.
result MEANTIME outperforms state-of-the-art sequential recommendation methods.
This paper predicts stock prices using LLMs and news embeddings.
problem Predicting stock prices with high accuracy and relevance.
method Integrates LLMs with stock name embeddings and attention mechanisms for news filtering.
result Reduces MAE by 7.11% compared to baseline.
STCA discovers dynamic functional brain networks using spatial-temporal convolution and attention.
problem Lack of dynamic exploration of functional brain networks.
method Spatial-Temporal Convolutional Attention (STCA) model.
result STCA can discover dynamic functional brain networks in a novel way.
Graph-augmented CNN predicts drug interactions with high accuracy.
problem Predicting drug-drug interactions (DDIs) with high accuracy.
method Combining graph CNN with an attentive pooling network to extract structural relations between drug pairs.
result Desirable performance with ROC 0.988, F1-score 0.956, and AUPR 0.986.
Proposes a graph pooling method leveraging node proximity for hierarchical graph representation learning.
problem Efficiently exploiting the geometry of graph data for hierarchical representation learning.
method Combines node proximity with kernel representation of topology and node features for adaptive node signal similarities evaluation.
result Achieves state-of-the-art performance on graph classification benchmark datasets.
SANST uses self-attentive networks with spatial and temporal embeddings for better POI recommendations.
problem Next point-of-interest (POI) recommendation for users based on their history.
method SANST incorporates spatio-temporal patterns into self-attentive networks.
result SANST outperforms state-of-the-art models by up to 13.65% in nDCG@10.
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
Federated learning interprets temporal dynamics across clients with graph attention.
problem Interpreting temporal patterns across decentralized, heterogeneous systems with nonlinear dynamics.
method Graph Attention Network for learning state transition models over latent states communicated between clients.
result First interpretable characterization of cross-client temporal interdependencies in decentralized nonlinear systems.
HGP-SL pools and learns graph structure for hierarchical representation learning.
problem Graph pooling is overlooked in GNN models, limiting hierarchical representation learning.
method Integrates graph pooling and structure learning into a unified module.
result HGP-SL improves graph classification performance on benchmarks.
RTFN extracts robust temporal features for time series analysis.
problem Challenges in extracting sufficient shapelets from time series data.
method Combines temporal feature networks and attentional LSTM networks.
result RTFN outperforms in supervised and unsupervised time series analysis.
Optimal rebalancing strategy improves AMM pool performance by 25%.
problem Optimizing the sequence of weights in dynamic AMM pools to minimize rebalancing costs.
method Using optimal interpolation and a cheap-to-compute approximation to achieve nearly optimal rebalancing.
result Approximately-optimal weight changes lead to significant increases in pool performance (up to 25%) under various conditions.
A3T-GCN improves traffic forecasting by capturing spatial and temporal dependencies.
problem Accurate real-time traffic forecasting in complex road networks.
method Attention Temporal Graph Convolutional Network (A3T-GCN) integrating recurrent units and graph convolutional network.
result Improved prediction accuracy through attention mechanism and global temporal information.
TGAT learns node embeddings for evolving graphs, capturing both static and temporal features.
problem Learning node embeddings for dynamic graphs with evolving topological structures and temporal patterns.
method Temporal Graph Attention (TGAT) layer using self-attention and functional time encoding.
result TGAT model can inductively infer node embeddings for new and observed nodes as the graph evolves.
Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence. In this paper, we analyze the usage of attention mec…
Graph neural networks, which generalize deep neural network models to graph structured data, have attracted increasing attention in recent years. They usually learn node representations by transforming, propagating and aggregating node features and have been proven to improve the performance of many graph related tasks…
New protocol evaluates synthetic data for temporal consistency.
problem Synthetic data generators can produce invalid timestamps and trajectories.
method Characterize datasets by four properties, then measure timestamp validity and dynamics.
result Temporal fidelity must be measured, not inferred from static data.
New framework predicts urban traffic with high accuracy.
problem Urban traffic prediction challenges.
method Interpretable attention-based neural network combining multiple modules.
result Framework outperforms state-of-the-art alternatives.
As one of the important functions of the intelligent transportation system (ITS), supply-demand prediction for autonomous vehicles provides a decision basis for its control. In this paper, we present two prediction models (i.e. ARLP model and Advanced ARLP model) based on two system environments that only the current d…
Develops models for temporally abstract reasoning and attention.
problem Temporal abstraction and attention in reinforcement learning.
method Defines affordances for options and develops partial option models.
result Identifies trade-offs between estimation and approximation error.
EarnMore uses masked stock representations to train RL agents for customizable stock pools efficiently.
problem Training RL agents for customizable stock pools (CSPs) is computationally expensive and unstable.
method EarnMore introduces a mechanism to mask out stocks outside the target pool, learns meaningful stock representations, and uses a re-weighting mechanism to focus on favorable stocks.
result EarnMore significantly outperforms state-of-the-art baselines in profit metrics with over 40% improvement.
Paper proposes deep learning model for dynamic stock repurchase forecasting.
problem Complex temporal dependencies in corporate financial conditions.
method Hybrid Temporal Convolutional Network (TCN) and Attention-based LSTM.
result Model significantly outperforms static baselines in stock repurchase forecasting.