arXiv research

Combines CNN and Transformer for financial time series forecasting.

TimeBridge addresses non-stationarity in long-term time series forecasting.

Improved GRU model with weighted time-delay feedback for long-term dependencies.

CFTM uses fractional Brownian motion for dynamic topic modeling.

New RNN model handles long-term dependencies in irregularly-sampled time series.

ST-SAN predicts flow with spatial-temporal dependencies using self-attention.

Despite recent innovations in network architectures and loss functions, training RNNs to learn long-term dependencies remains difficult due to challenges with gradient-based optimisation methods. Inspired by the success of Deep Neuroevolution in reinforcement learning (Such et al. 2017), we explore the use of gradient-…

KEDformer improves long-term time series forecasting with seasonal-trend decomposition.

Paper questions RNN and LSTM's long-term memory and introduces a new definition.

LEM efficiently models long-term sequences with gradients.

Deep RNNs excel at capturing long-term dependencies in sequential data.

Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures lik…

Recurrent neural networks have gained widespread use in modeling sequential data. Learning long-term dependencies using these models remains difficult though, due to exploding or vanishing gradients. In this paper, we draw connections between recurrent networks and ordinary differential equations. A special form of rec…

We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory. We achieve this by extending unitary RNNs with a gating mechanism. Our model is able to outperform LSTM…

A major drawback of backpropagation through time (BPTT) is the difficulty of learning long-term dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason,…

TiDE uses MLP for fast, simple long-term time-series forecasting.

Paper proposes a graph neural network for accurate long-term ILI prediction.

Mamba Hawkes Process improves modeling of event sequences with long-term dependencies.

Transformer learns long-term dependencies from real-world data.

Preformer improves Transformer for long-term time series forecasting.

Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is to capture the spatial correlations at the same time, the spatio-temporal relationships at different times and the long-term dependence of the temporal relationships between different series.…

Improved recurrent neural networks learn long-term dependencies through multi-scale memory.

Structured state space models improve ECG classification and reveal new insights.

THP model captures complex dependencies in event sequences efficiently.

Successful recurrent models such as long short-term memories (LSTMs) and gated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these models have been found to improve the learning of medium to long term temporal dependencies and to help with vanishing gradient issues. We prove that learnable gates in a…

FPG uses fractional calculus for efficient reinforcement learning with long-term memory.

Sequence feature embedding is a challenging task due to the unstructuredness of sequence, i.e., arbitrary strings of arbitrary length. Existing methods are efficient in extracting short-term dependencies but typically suffer from computation issues for the long-term. Sequence Graph Transform (SGT), a feature embedding …

Learning long-term dependencies still remains difficult for recurrent neural networks (RNNs) despite their success in sequence modeling recently. In this paper, we propose a novel gated RNN structure, which contains only one gate. Hidden states in the proposed grouped distributor unit (GDU) are partitioned into groups.…

Time-related features improve time series forecasting models.

Hamiltonian RNN controls hidden states gradient for long-term dependencies.

A new memory-based fusion layer improves multi-modal deep learning performance.

The diagonal effect of orders is well documented in different markets, which states that orders are more likely to be followed by orders of the same aggressiveness and implies the presence of short-term correlations in order flows. Based on the order flow data of 43 Chinese stocks, we investigate if there are long-rang…

New algorithm optimizes for long-term user satisfaction in delayed reward settings.

We propose a particularly structured Boltzmann machine, which we refer to as a dynamic Boltzmann machine (DyBM), as a stochastic model of a multi-dimensional time-series. The DyBM can have infinitely many layers of units but allows exact and efficient inference and learning when its parameters have a proposed structure…

Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and achieve better generalization than vanilla recurrent neural networks (RNN). We suggest that memory augmented neural networks can reduce the ef…

We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game. The resulting formulation enables us to deal with non-stationary and/or long-term constraints , which arise, for example, in online display advertising problems. We propose a…

New interpretation of RNN forget gate improves learnability for long-term sequential data.

The study improves Monte Carlo simulations for long-term investments using advanced financial models.

New framework captures long-term decision dependence in online learning.

Understanding temporal dynamics has proved to be highly valuable for accurate recommendation. Sequential recommenders have been successful in modeling the dynamics of users and items over time. However, while different model architectures excel at capturing various temporal ranges or dynamics, distinct application cont…

Quaternion neural networks improve distant speech recognition.

Different investment strategies are adopted in short-term and long-term depending on the time scales, even though time scales are adhoc in nature. Empirical mode decomposition based Hurst exponent analysis and variance technique have been applied to identify the time scales for short-term and long-term investment from …

Researchers have used many different methods to detect the possibility of long-term dependence (long memory) in stock market returns, but evidence is in general mixed. In this paper, three different tests, (namely Rescaled Range (R/S), its modified form, and the semi-parametric method (GPH)), in addition to a new appro…

Despite recent advances in training recurrent neural networks (RNNs), capturing long-term dependencies in sequences remains a fundamental challenge. Most approaches use backpropagation through time (BPTT), which is difficult to scale to very long sequences. This paper proposes a simple method that improves the ability …

Statistical fields model explains capital allocation and accumulation among firms and investors.

Modeling daily river flow distribution with seasonal and long-term trends.

A novel online GP model captures long-term memory in sequential data.

New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.