arXiv research

Model combines long-term and short-term memory using conceptors.

We study the long-term memory in diverse stock market indices and foreign exchange rates using the Detrended Fluctuation Analysis(DFA). For all daily and high-frequency market data studied, no significant long-term memory property is detected in the return series, while a strong long-term memory property is found in th…

Paper questions RNN and LSTM's long-term memory and introduces a new definition.

QLSTM outperforms LSTM in predicting KSE 100 index movements.

LTM tackles long sequence language modeling by avoiding vanishing and exploding gradients.

Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.

LEM efficiently models long-term sequences with gradients.

Current generation of memory-augmented neural networks has limited scalability as they cannot efficiently process data that are too large to fit in the external memory storage. One example of this is lifelong learning scenario where the model receives unlimited length of data stream as an input which contains vast majo…

Long short-term memory network outperforms seasonal model in JSE Top 40 forecasting.

We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in t…

A novel online GP model captures long-term memory in sequential data.

FPG uses fractional calculus for efficient reinforcement learning with long-term memory.

Extended LSTMs improve volatility prediction by 20%.

New algorithms provide long-term memory guarantees for online learning.

New algorithms for multitask learning with long-term memory.

Enhanced LSTM model learns complex temporal dependencies.

Recurrent neural networks (RNN) are at the core of modern automatic speech recognition (ASR) systems. In particular, long-short term memory (LSTM) recurrent neural networks have achieved state-of-the-art results in many speech recognition tasks, due to their efficient representation of long and short term dependencies …

Generative diffusion models mimic biological memory networks, encoding associative dynamics in deep neural weights.

We focus on emergence of the power-law cross-correlations from processes with both short and long term memory properties. In the case of correlated error-terms, the power-law decay of the cross-correlation function comes automatically with the characteristics of separate processes. Bivariate Hurst exponent is then equa…

EnLSTM network improves log generation from small datasets.

New RNN model handles long-term dependencies in irregularly-sampled time series.

Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we pro…

The paper introduces a new stochastic volatility model with long-term memory and jumps.

Method extracts knowledge from LSTM for sequence validation.

New method combines long-memory reservoirs for accurate dengue forecasting from short data.

fSDE-Net generates time series with long-term memory using neural networks.

New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.

Deep learning predicts baseball home runs with better accuracy.

In this paper we outline initial concepts for an immune inspired algorithm to evaluate price time series data. The proposed solution evolves a short term pool of trackers dynamically through a process of proliferation and mutation, with each member attempting to map to trends in price movements. Successful trackers fee…

Efficient algorithm predicts unknown linear systems with long-term memory.

Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and achieve better generalization than vanilla recurrent neural networks (RNN). We suggest that memory augmented neural networks can reduce the ef…

NeuroMemFPP uses LSTM to estimate FPP parameters with high accuracy.

RNN beats Lee-Carter in forecasting mortality rates.

Deep neural networks estimate long memory parameters efficiently.

Paper tackles stutter detection using deep learning.

The well-known Mori-Zwanzig theory tells us that model reduction leads to memory effect. For a long time, modeling the memory effect accurately and efficiently has been an important but nearly impossible task in developing a good reduced model. In this work, we explore a natural analogy between recurrent neural network…

We investigate the temporal correlations and multifractal nature of trading volume of 22 liquid stocks traded on the Shenzhen Stock Exchange in 2003. We find that the trading volume exhibit size-dependent non-universal long memory and multifractal nature. No crossover in the power-law dependence of the detrended fluctu…

Improved recurrent neural networks learn long-term dependencies through multi-scale memory.

Much sequential data exhibits highly non-uniform information distribution. This cannot be correctly modeled by traditional Long Short-Term Memory (LSTM). To address that, recent works have extended LSTM by adding more activations between adjacent inputs. However, the approaches often use a fixed depth, which is at the …

We utilize long-term memory, fractal dimension and approximate entropy as input variables for the Efficiency Index [Kristoufek & Vosvrda (2013), Physica A 392]. This way, we are able to comment on stock market efficiency after controlling for different types of inefficiencies. Applying the methodology on 38 stock marke…

Language models are miscalibrated, leading to overestimated entropy rates and memory usage.

Study adds memory effect to Solow-Swan model for more accurate economic growth modeling.

New framework captures long-term decision dependence in online learning.

LSTMs improve demand forecasting for e-grocery products.

Study examines cryptocurrency behavior during and after the pandemic.

Fully convolutional neural networks (FCN) have been shown to achieve state-of-the-art performance on the task of classifying time series sequences. We propose the augmentation of fully convolutional networks with long short term memory recurrent neural network (LSTM RNN) sub-modules for time series classification. Our …

A new memory-based fusion layer improves multi-modal deep learning performance.

Deep RNNs excel at capturing long-term dependencies in sequential data.