Transformers improve time series modeling by capturing long-range dependencies.
problem Capturing long-range dependencies in time series data.
method Summarized and reviewed adaptations of Transformers for time series analysis.
result Transformers enhance time series forecasting, anomaly detection, and classification.
W-Transformers use wavelets to improve time series forecasting.
problem Forecasting non-stationary time series with long-range dependencies.
method Wavelet-based transformer architecture.
result W-Transformers outperform baseline models on various time series datasets.
Paper uses Time Series Transformer for bank stability prediction.
problem Predicting bank stability using complex financial data.
method Time Series Transformer model with self-attention mechanism.
result Time Series Transformer model outperforms other models in MSE and MAE.
New method uses Transformers for flu forecasting.
problem Forecasting influenza-like illness trends.
method Transformer-based machine learning models with self-attention.
result Forecasting results are competitive with state-of-the-art methods.
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
Study compares LSTM and Transformer models in financial time series prediction.
problem Comparing LSTM and Transformer models for financial time series prediction.
method Various LSTM-based and Transformer-based models compared on financial tasks; DLSTM and new Transformer architecture designed.
result Transformer-based models show limited advantage in absolute price sequence prediction, while LSTM-based models perform better on difference sequences.
Transformers forecast time series in-context, improving efficiency and performance.
problem Overfitting and limited performance in time series forecasting.
method Reformulate time series forecasting as input tokens, aligning with in-context learning mechanisms.
result Consistently better performance across various settings (full-data, few-shot, zero-shot).
ALT improves TSC by capturing complex patterns in time series data.
problem Challenges in traditional TSC methods with time series complexity and variability.
method ALT incorporates variable-length shifted time windows to enhance LLT for better feature representation.
result ALT achieves state-of-the-art performance with few hyperparameters.
ALT transforms time series data for better classification.
problem Efficiently classifying time series data with varying temporal scales.
method ALT algorithm using variable-length shifted time windows.
result State-of-the-art performance with minimal computational overhead.
Timer-XL predicts multidimensional time series using a unified Transformer approach.
problem Unified time series forecasting across various tasks and contexts.
method Decoder-only Transformers with a universal TimeAttention mechanism and deft position embedding.
result State-of-the-art performance across multiple forecasting benchmarks.
Transformers improve stock forecasting with federated learning.
problem Overfitting, data scarcity, and privacy issues in transformer-based time series forecasting.
method Attentive federated transformers for time series stock forecasting.
result Proposed scheme outperforms conventional training schemes in stock forecasting.
Generates financial time series with stylized facts using diffusion models.
problem Generating realistic synthetic financial time series with statistical properties like fat tails, volatility clustering, and seasonality.
method Utilizes denoising diffusion probabilistic models (DDPMs) with wavelet transformation to convert and generate financial time series.
result Demonstrates that the proposed approach satisfies stylized financial time series properties.
FEDformer combines Transformer with seasonal-trend decomposition for efficient long-term forecasting.
problem Transformer's inefficiency and inability to capture global time series views.
method Combines seasonal-trend decomposition with Transformer, exploiting Fourier basis for frequency enhancement.
result Reduces prediction error by 14.8% and 22.6% for multivariate and univariate time series, respectively.
OLinear forecasts time series more efficiently by transforming data orthogonally.
problem Efficiently forecasting time series with entangled dependencies.
method OLinear uses OrthoTrans to transform data orthogonally, then applies NormLin for linear layer.
result OLinear achieves state-of-the-art performance with high efficiency.
TEAFormers preserve multi-dimensional time series structures for better forecasting.
problem Traditional Transformers flatten multi-dimensional time series data, losing critical multi-dimensional relationships.
method Tensor-Augmented Transformer (TEAFormer) with Tensor-Augmentation (TEA) module.
result Significant performance enhancements in time series forecasting across benchmarks.
Preformer improves Transformer for long-term time series forecasting.
problem Transformer's quadratic complexity and lack of context-awareness for long-term forecasting.
method Introduces Multi-Scale Segment-Correlation mechanism for efficient time series segmentation and context-aware attention.
result Preformer outperforms other Transformer-based methods in long-term time series forecasting.
New copula models capture volatility and directionality in financial time series.
problem Modeling financial return series with volatility and serial correlation.
method Stationary d-vine copula processes with v-transforms for stochastic volatility and directionality.
result Models can rival and sometimes outperform GARCH family models.
Novel deep learning model for multivariate time series prediction.
problem Challenges in multivariate time series prediction with correlations and complex temporal patterns.
method Temporal Tensor Transformation Network (TTNT) that transforms multivariate time series into tensors for improved feature extraction.
result TTNT outperforms state-of-the-art methods in window-based predictions across various tasks.
Sig-Splines model uses signatures and splines for time series data, achieving universality and convexity.
problem Creating a generative model for multivariate time series data.
method Combines linear transformations and signature transforms into a neural spline flow.
result Achieves universality and introduces convexity in model parameters.
This paper extends hypergraph construction to multivariate time series using signature transforms.
problem Constructing hypergraphs from collections of multivariate time series.
method Leveraging signature transforms to introduce controlled randomness and robustness.
result Validated on synthetic datasets, the method enhances robustness in hypergraph construction.
We present online prediction methods for time series that let us explicitly handle nonstationary artifacts (e.g. trend and seasonality) present in most real time series. Specifically, we show that applying appropriate transformations to such time series before prediction can lead to improved theoretical and empirical p…
Proposes a transformer-based approach for anomaly detection in time series data.
problem Inadequate evaluation metrics and inability to capture temporal features in time series anomaly detection.
method Introduces a proper evaluation metric and proposes a transformer-based approach for anomaly detection in time series data.
result Transformer-based approach outperforms state-of-the-art detectors in detecting sequential anomalies.
Combines CNN and Transformer for financial time series forecasting.
problem Forecasting financial time series, especially stock prices, is challenging due to short-term and long-term dependencies.
method Uses CNN for short-term dependencies and Transformer for long-term dependencies.
result Demonstrated superior performance in forecasting stock price changes compared to traditional methods.
New algorithm for uncertain time series classification.
problem Uncertainty in time series data.
method Uncertain dissimilarity measure based on Euclidean distance and uncertain shapelet transform.
result Effectiveness of the uncertain shapelet transform algorithm on state-of-the-art datasets.
Transformer models predict financial time series movements accurately.
problem Applying transformer models to financial time series prediction.
method Transformer architecture applied to synthetic and real S&P500 data.
result Transformer models predict financial time series movements accurately.
Paper develops large time series models using pre-trained transformers.
problem Performance bottlenecks in small models on data-scarce scenarios.
method Large-scale pre-training, unified time series format, GPT-style architecture.
result Generative pre-trained Time Series Transformer (Timer) for diverse tasks.
Rough Transformers improve efficiency for medical time-series data.
problem Efficiently modeling irregularly sampled, long-range time-series data.
method Introducing Rough Transformers, a Transformer variant with continuous-time representations and multi-view signature attention.
result Rough Transformers outperform vanilla Transformers while using less computational resources.
Rough Transformers improve time series modeling with lower costs and better performance.
problem Inefficient modeling of irregularly sampled time series data.
method Signature patching for continuous-time representations, reducing computational costs.
result Rough Transformers outperform vanilla Transformers and Neural ODE models.
Sentinel improves time series forecasting by modeling both temporal and channel dependencies.
problem Limited effectiveness of existing transformer-based architectures in multivariate time-series forecasting.
method Proposes Sentinel, a full transformer-based architecture with multi-patch attention mechanism.
result Sentinel achieves better or comparable performance compared to state-of-the-art approaches.
A new method models volatile financial time series using v-transforms and copulas.
problem Modeling volatile financial time series with standard methods.
method v-transforms and copulas to describe and estimate time series with arbitrary marginal distributions and copula dynamics.
result The model replicates stylized facts of financial return series and facilitates risk quantification.
A new model detects and localizes anomalies in multivariate time series data.
problem Anomaly diagnosis in multivariate time series data, especially localization.
method Attention Low-Rank Transformer (ALoRa-T) with low-rank regularization and Attention Low-Rank score.
result The proposed method significantly outperforms state-of-the-art methods in anomaly detection and localization.
TACTiS models time series uncertainty with transformer attention.
problem Estimating predictive uncertainty in high-dimensional multivariate time series.
method Transformer-Attentional Copulas using attention-based decoder.
result TACTiS produces state-of-the-art predictions on real-world datasets.
MultiRocket boosts TSC speed and accuracy with pooling and transformations.
problem Efficient time series classification with high accuracy.
method Multiple pooling operators and transformations applied to raw and differenced series.
result MultiRocket outperforms MiniRocket and is competitive with state-of-the-art methods in terms of accuracy and speed.
Shapelet transform improves time series classification for earthquake, wind, and wave events.
problem Autonomous detection of specific events from large time series datasets in civil engineering.
method Shapelet transform for local similarity in time series subsequences, combined with machine learning.
result Shapelet transform yields a new feature representation for time series signals in civil engineering.
Transformer RL optimizes A/B testing for time series experiments.
problem Challenges in applying A/B testing to time series experiments, especially with limited history and strong assumptions.
method Transformer reinforcement learning approach that conditions allocation on full history and optimizes MSE without restrictive assumptions.
result Consistently outperforms existing designs in synthetic, simulator, and real-world data.
SAMformer improves transformer performance in time series forecasting.
problem Transformers struggle with multivariate long-term forecasting.
method Sharpness-aware minimization and channel-wise attention.
result SAMformer surpasses state-of-the-art methods in multivariate time series forecasting.
Study geodesic properties of time series data using Wasserstein metric.
problem Modeling nonlinear time series with transport-based metrics.
method Generalized Wasserstein metric and signed cumulative distribution transforms.
result Geodesic properties provide added interpretability and robustness in time series classifiers.
ARM improves multivariate time series forecasting by better capturing series-wise relationships.
problem Challenges in handling complex temporal-contextual relationships in multivariate time series forecasting.
method ARM is an enhanced multivariate LTSF architecture that employs Adaptive Univariate Effect Learning, Random Dropping, and Multi-kernel Local Smoothing.
result ARM outperforms vanilla Transformers on multiple benchmarks without significantly increasing computational costs.
Enhances stock movement prediction using Higher Order Transformers for multimodal time-series data.
problem Predicting stock movements in financial markets with complex dynamics.
method Introduced Higher Order Transformers, extending self-attention and transformer architecture to capture complex market dynamics. Employed low-rank tensor decomposition and kernel attention to manage computational complexity. Integrated technical and fundamental analysis from historical prices and tweets.
result Demonstrated effectiveness of the method on the Stocknet dataset, improving stock movement prediction.
Mantis improves time series classification using a transformer model trained on synthetic data.
problem Insufficient application of foundation models to time series classification.
method Pre-trained transformer model on synthetic data, enhanced test-time methodology.
result Mantis achieves state-of-the-art performance across diverse datasets.
Transformer model removes noise from light curves efficiently.
problem Challenges in processing astrophysical light curves due to noise.
method Denoising Time Series Transformer (DTST) model trained with masked objective.
result DTST model excels at removing noise and outliers in time series datasets.
A new algorithm reduces time complexity for binary time series classification.
problem High time complexity of ensemble shapelet transform limits its application.
method Introduces short isometric shapelet transform with two strategies: fixed shapelet length and single linear classifier.
result Demonstrates superior performance and reduced time complexity.
DRFormer uses dynamic tokenization and multi-scale transformer to forecast long time series.
problem Forecasting long-term time series data across diverse scales.
method Dynamic tokenizer, multi-scale transformer, dynamic sparse learning, rotary position encoding.
result DRFormer outperforms existing methods in forecasting accuracy.
Extends shapelet transform to irregular time series, improving interpretability and efficiency.
problem Limitations of shapelet transform for irregular, partially observed time series.
method Continuous-time formulation, regularisation penalty, learned pseudometric.
result Efficient training without sacrificing interpretability for irregular, partially observed time series.
Enformer and GEnformer use Transformers with stochastic learning to forecast multivariate and spatiotemporal data with uncertainty.
problem Uncertainty quantification in multivariate time series and spatiotemporal forecasting.
method Synthesizing Transformer's expressive power with stochastic learning to model conditional distributions directly.
result Enformer and GEnformer yield calibrated probabilistic forecasts and outperform state-of-the-art baselines.
Time series forecasting is an important problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. In this paper, we propose to tackle such forecasting problem with Transformer [1]. Although impressed by its performance in our preliminary study, …
New method uses shared attention for multi-task time series forecasting.
problem Insufficient training instances in single-task forecasting.
method Self-attention based sharing schemes across multiple tasks.
result Outperforms state-of-the-art single-task forecasting baselines and RNN-based multi-task forecasting method.
Time series forecasting with limited data is a challenging yet critical task. While transformers have achieved outstanding performances in time series forecasting, they often require many training samples due to the large number of trainable parameters. In this paper, we propose a training technique for transformers th…