TOQ-Nets learn to recognize complex temporal events with varying objects and sequences.
problem Recognizing complex relational-temporal events with varying numbers of objects and sequence lengths.
method Neuro-symbolic networks with reasoning layers for finite-domain quantification over objects and time.
result TOQ-Nets can generalize to scenarios with more objects than training data and temporal warpings.
TCNs can approximate complex input-output maps with limited memory.
problem Approximating complex input-output maps with limited memory.
method Proved TCNs can approximate a wide class of input-output maps with arbitrary error tolerance.
result Deep ReLU TCNs can approximate input-output maps with finite memory to arbitrary error.
Neurons predict future scalar inputs by learning top modes of lag vectors.
problem Predicting future scalar inputs with physiological delays.
method Normal Mode Decomposition to extract independently evolving modes.
result Temporal filters of neurons correspond to left eigenvectors of a generalized eigenvalue problem.
A novel RNN uses stigmergy for temporal input encoding.
problem Temporal input encoding in RNNs.
method Stigmergy-based computational memory in RNN design.
result Stigmergy enables coherent temporal input encoding.
In kernel methods, temporal information on the data is commonly included by using time-delayed embeddings as inputs. Recently, an alternative formulation was proposed by defining a gamma-filter explicitly in a reproducing kernel Hilbert space, giving rise to a complex model where multiple kernels operate on different t…
The paper evaluates different graph input representations for urban network analysis.
problem Challenges in analyzing urban networks due to their size and complexity.
method Design and evaluation of six graph input representations considering topological and temporal characteristics.
result Temporal information in graph input representations significantly improves model accuracy (RMSE of 1.42).
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
TFT improves multi-horizon forecasting with interpretable insights.
problem Complex multi-horizon forecasting with mixed inputs.
method Attention-based Temporal Fusion Transformer combining recurrent and self-attention layers.
result Significant performance improvements over existing benchmarks.
Paper proposes a novel approach to improve temporal clustering of time series data.
problem Challenges in clustering temporal data with varying sampling rates and high dimensionality.
method Transform time series into Euclidean space using similarity measures, then use CNN-GRU autoencoder for latent representation.
result Approach outperforms existing methods by up to 32% on various time series datasets.
MeshfreeFlowNet generates high-resolution spatio-temporal solutions from low-resolution inputs.
problem Generating high-resolution spatio-temporal solutions from low-resolution inputs.
method Physics-constrained deep learning framework using fully convolutional encoders.
result Significantly outperforms existing baselines in super-resolution of turbulent flows.
A2MT learns agents to select which modalities to acquire at test time.
problem Learning agents to select modalities for multimodal temporal data acquisition.
method Perceiver IO architecture for active acquisition of multimodal temporal data.
result Agents successfully learn cost-reactive acquisition behavior on real-world datasets.
Quantum reservoir computing tackles noisy quantum computers for temporal tasks.
problem Efficiently process input sequences on noisy quantum computers.
method Quantum reservoir computing using dissipative quantum dynamics.
result Small and noisy quantum reservoirs can handle high-order nonlinear temporal tasks.
Unified architecture for multi-modal multi-task learning using transformer.
problem Training multiple tasks concurrently with varying modalities.
method Spatio-temporal cache mechanism for multi-modal learning.
result Training multiple tasks together reduces model size by about three times.
New framework analyzes temporal features in state space models.
problem Understanding temporal dependencies in data streams.
method Proposes a framework for rigorous analysis of state representations in ESNs, using temporal feature spaces and kernel machines.
result Phase transition in kernel richness for cycle reservoir topology.
Model improves information transfer from visual streams.
problem Challenges in unsupervised learning from continuous visual data.
method Inspired by physics, maximizes mutual information through temporal process.
result Focus of attention enhances information transfer from input stream.
T-SVM improves learning in spiking neurons by maximizing dynamical margin.
problem Finding robust solutions in spiking neuronal networks with temporal correlations.
method Introduces Temporal Support Vector Machine (T-SVM) to maximize dynamical margin.
result T-SVM enables learning of tasks requiring nonlinear spatial integration.
Paper tackles fall detection using adversarial learning.
problem Detecting falls in the absence of training data due to class imbalance.
method Adversarial learning framework with spatio-temporal autoencoder and convolution network.
result Proposed framework outperformed baseline methods on publicly available datasets.
A new model for sequential memory using temporal predictive coding.
problem Forming accurate memory of sequential stimuli in the brain.
method Proposes a novel PC-based model called temporal predictive coding (tPC).
result Shows that tPC models can accurately memorize and retrieve sequential inputs.
STDGI learns node representations for spatio-temporal graphs via mutual information maximization.
problem Challenges in learning node representations for spatio-temporal graphs due to structural changes over time.
method STDGI is a fully unsupervised approach based on mutual information maximization that exploits both spatial and temporal dynamics.
result STDGI's learned node representations improve spatio-temporal auto-regressive forecasting models.
This paper investigates the role of sparsity in Reservoir Computing networks.
problem Designing efficient Recurrent Neural Networks (RNNs) with hidden recurrent layers.
method Empirical investigation of sparsity in input-reservoir connections and recurrent connections.
result Sparsity, particularly in input-reservoir connections, enhances the network's temporal memory and dimensionality.
Recent studies have highlighted adversarial examples as a ubiquitous threat to different neural network models and many downstream applications. Nonetheless, as unique data properties have inspired distinct and powerful learning principles, this paper aims to explore their potentials towards mitigating adversarial inpu…
This research unifies concepts of fading memory in RNNs.
problem Unclear relationships between fading memory concepts in RNNs.
method Unified language and new proofs for fading memory concepts.
result Clarified relationships between fading memory concepts.
A new model learns demand patterns from data, reducing complexity and improving accuracy.
problem Forecasting short-term demand from spatiotemporal data with complex patterns.
method Temporal-Guided Network (TGNet) using graph networks and temporal-guided embedding.
result TGNet achieves competitive performance with fewer parameters compared to state-of-the-art models.
A method for learning from unlabeled time-series data using temporal smoothing and entropy maximization.
problem Learning from unlabeled time-series data efficiently and accurately.
method Training a feedforward neural network with two objectives: temporal smoothing and entropy maximization.
result The method extracts slowly evolving information from time-series data, filtering out noise.
RTFN extracts robust temporal features for time series analysis.
problem Challenges in extracting sufficient shapelets from time series data.
method Combines temporal feature networks and attentional LSTM networks.
result RTFN outperforms in supervised and unsupervised time series analysis.
This study prioritizes temporal resolution over spatial in energy systems models due to higher influence.
problem The impact of spatial and temporal resolution on energy system models.
method Global sensitivity analysis to compare structural aspects, spatial, and temporal resolution.
result Temporal resolution has a higher influence on all results parameters compared to spatial resolution.
Interpretability has arisen as a key desideratum of machine learning models alongside performance. Approaches so far have been primarily concerned with fixed dimensional inputs emphasizing feature relevance or selection. In contrast, we focus on temporal modeling and the problem of tailoring the predictor, functionally…
End-to-end speech recognition using EEG without speech input.
problem Speech recognition without direct speech input.
method Implemented attention model and CTC-based ASR systems for EEG signals; fused EEG with noisy speech features.
result Demonstrated end-to-end speech recognition using EEG signals.
Enhances network intrusion detection in noisy data.
problem Robustness against contaminated and noisy data inputs in network intrusion detection.
method Probabilistic Temporal Graph Network Support Vector Data Description (TGN-SVDD) model.
result Significant improvements in detection performance with synthetic noise.
Enhanced LSTM model learns complex temporal dependencies.
problem Modeling long-term dependencies in sequential data.
method Gamma-LSTM with hierarchical memory units and gates.
result Gamma-LSTM outperforms regular and stacked LSTMs in sequence prediction.
ST-UNet models spatio-temporal graphs by pooling and unpooling operations.
problem Lack of effective means to extract dynamic features from spatio-temporal graphs.
method Designing a multi-scale architecture, Spatio-Temporal U-Net (ST-UNet), with paired sampling operations.
result Achieves substantial improvements in spatio-temporal prediction tasks.
A new method exposes motion-related relevance in video frames.
problem Deconstructing relevance in spatio-temporal models for video processing.
method Proposes a discriminative method to separate spatial and temporal relevance.
result Demonstrates effectiveness on UCF-101 action recognition dataset.
Hybrid model predicts flow and pressure in water systems.
problem Predicting flow and pressure in water distribution systems with complex spatial-temporal correlations.
method Hybrid dual-stage spatial-temporal attention-based recurrent neural networks (hDS-RNN).
result Our model outperformed 9 baseline models in flow and pressure series prediction.
CTCModel extends Keras for transparent CTC classification.
problem Handling unsegmented input sequences with labels related to subsets of frames.
method Combines Keras and CTC implementation in Tensorflow backend.
result CTCModel predicts sequences of labels from unsegmented input.
MetNet forecasts precipitation up to 8 hours with high spatial and temporal resolution.
problem Precise weather forecasting for long lead times.
method Neural network architecture using axial self-attention for global context aggregation.
result MetNet outperforms Numerical Weather Prediction at forecasts of up to 8 hours.
FNOs improve spatio-temporal forecasting without needing PDE details.
problem Complex spatio-temporal dynamics in physical and biological phenomena.
method Fourier Neural Operators (FNOs) for dynamic spatio-temporal modeling.
result FNO forecasts are accurate and capture complex real-world dependencies.
New insights link memory loss to system stability in dynamical systems.
problem Understanding the relationship between dynamics and computation, especially stability and memory loss.
method Analyzing driven dynamical systems responding to temporal inputs.
result Memory loss in driven systems leads to consistent responses to similar inputs and affects stability.
Sentinel improves time series forecasting by modeling both temporal and channel dependencies.
problem Limited effectiveness of existing transformer-based architectures in multivariate time-series forecasting.
method Proposes Sentinel, a full transformer-based architecture with multi-patch attention mechanism.
result Sentinel achieves better or comparable performance compared to state-of-the-art approaches.
New interpretation of RNN forget gate improves learnability for long-term sequential data.
problem Improving learnability of recurrent neural networks for long-term temporal dependencies.
method Generalized theory of gated RNNs, focusing on gradient behavior over time.
result Existing RNNs satisfy the gradient condition for initial training, suggesting validity of forget gate interpretation.
Algorithm mines environment assumptions for cyber-physical systems.
problem Modeling and verifying complex cyber-physical systems.
method Supervised learning to mine STL formulas for input signals.
result Algorithm learns both the structure and constants of STL formulas.
Paper tackles temporal overfitting in wind power curve modeling.
problem Temporal overfitting in wind power curve modeling.
method Proposes a Gaussian process-based method to partition and model time-invariant and time-varying components.
result Significant improvement in predicting responses for different time periods.
GRU-D detects age-specific missing patterns in vital signs.
problem Temporal missingness in clinical time series data.
method Gated recurrent unit with decay mechanisms (GRU-D) trained on MIMIC-IV vital signs.
result GRU-D achieves AUROC 0.780 and AUPRC 0.810 on bootstrapped data.
FS-GCLSTM predicts stock returns by leveraging value-chain relationships.
problem Traditional time series models fail to capture complex interdependencies in modern markets.
method FS-GCLSTM integrates value-chain networks and graph convolutions to predict stock returns.
result FS-GCLSTM consistently delivers superior portfolio performance compared to traditional models.
Proposes a GNN for multivariate time-series prediction with filtering.
problem Low signal-to-noise ratio in complex systems data.
method Integrates a spatial-temporal GNN with a matrix filtering module to generate filtered graphs.
result Proposed model outperforms baseline approaches in multivariate time-series prediction.
A new method aligns convolution filters for temporal sequences using Dynamic Time Warp.
problem Improving deep learning models' ability to handle temporal sequence data.
method Integrates Dynamic Time Warp algorithm into 1-D convolution layers for better alignment of input and filter.
result Exceeds or matches standard 1-D convolution layers in time series classification tasks.
FATHOM model improves sensor data analysis with attention and LSTM.
problem Scarcity of training data from multiple sensors.
method Federated multi-task hierarchical attention model (FATHOM) with attention mechanism and LSTM.
result FATHOM outperforms baselines in sensor data classification and regression.
TGR rewires temporal graphs to improve TGNN performance.
problem Temporal graphs in evolving networks can suffer from under-reaching and over-squashing issues.
method TGR uses expander graph propagation to create message-passing highways between temporally distant nodes.
result TGR achieves state-of-the-art results on temporal graph benchmarks.
ARM improves multivariate time series forecasting by better capturing series-wise relationships.
problem Challenges in handling complex temporal-contextual relationships in multivariate time series forecasting.
method ARM is an enhanced multivariate LTSF architecture that employs Adaptive Univariate Effect Learning, Random Dropping, and Multi-kernel Local Smoothing.
result ARM outperforms vanilla Transformers on multiple benchmarks without significantly increasing computational costs.