Combines CNN and Transformer for financial time series forecasting.
problem Forecasting financial time series, especially stock prices, is challenging due to short-term and long-term dependencies.
method Uses CNN for short-term dependencies and Transformer for long-term dependencies.
result Demonstrated superior performance in forecasting stock price changes compared to traditional methods.
PBO methods improve RNN performance in learning long-term dependencies.
problem Training RNNs to learn long-term dependencies is challenging.
method Population-based global optimisation (PBO) techniques, including evolution strategies and particle swarm optimisation.
result PBO methods lead to performance improvements in RNNs for volatility forecasting.
Sparse Attentive Backtracking selectively backpropagates long-term dependencies in recurrent networks.
problem Difficulty in learning long-term dependencies in BPTT due to computational impracticality and biased gradient estimates.
method Sparse Attentive Backtracking learns an attention mechanism over past hidden states and selectively backpropagates through high-weight paths.
result Model learns long-term dependencies with fewer backpropagation steps, addressing biased gradient issues.
TimeBridge addresses non-stationarity in long-term time series forecasting.
problem Non-stationarity in multivariate time series leads to spurious regressions and obscures long-term relationships.
method TimeBridge segments series into patches, applying Integrated Attention for short-term non-stationarity and Cointegrated Attention for long-term cointegration.
result TimeBridge achieves state-of-the-art performance in both short-term and long-term forecasting.
A new recurrent unit alleviates vanishing gradients for long-term dependencies.
problem Vanishing gradients in recurrent neural networks make long-term dependencies hard to model.
method Proposes a new NRU architecture that avoids saturating activation functions and gates.
result Demonstrates superior performance across various tasks with and without long-term dependencies.
QLSTM improves speech recognition by considering internal quaternion dependencies.
problem Weak internal dependencies in traditional RNNs for speech recognition.
method Proposes QLSTM, a quaternion-based LSTM that considers both external and internal dependencies.
result QLSTM achieves better performance with up to 2.8 times fewer parameters.
New RNN model handles long-term dependencies in irregularly-sampled time series.
problem Handling long-term dependencies in irregularly-sampled time series data.
method Designing ODE-LSTMs that separate memory from continuous-time state.
result ODE-LSTMs outperform other RNN-based models on non-uniformly sampled data with long-term dependencies.
This paper improves RNNs' ability to handle long-term dependencies using an auxiliary loss.
problem Capturing long-term dependencies in RNNs is challenging.
method Adds an unsupervised auxiliary loss to improve long-term dependency capture.
result Improves performance and resource efficiency over competitive baselines.
New method improves learning of long-term dependencies in recurrent models.
problem Improving learning of long-term dependencies in recurrent neural networks.
method Proves learnable gates in recurrent models provide quasi-invariance to time transformations and recovers part of LSTM architecture from axiomatic approach.
result New chrono initialization of gate biases greatly improves learning of long-term dependencies.
Paper questions RNN and LSTM's long-term memory and introduces a new definition.
problem Whether RNN and LSTM have long-term memory.
method Introduced a new definition of long-term memory and modified RNN and LSTM to test it.
result RNN and LSTM do not meet the new definition of long-term memory.
LEM efficiently models long-term sequences with gradients.
problem Learning long-term sequential dependencies in data.
method Gradient-based multiscale ordinary differential equations system.
result LEM outperforms state-of-the-art models in various tasks.
This paper studies the dynamics of Brazilian interest rates for short-term maturities. The paper employs developed techniques in the econophysics literature and tests for long-range dependence in the term structure of these interest rates for the last decade. Empirical results suggest that the degree of long-range depe…
Paper introduces a simpler gated RNN structure to better capture long-term dependencies.
problem Difficulty in learning long-term dependencies in RNNs.
method Proposes a grouped distributor unit (GDU) with partitioned hidden states and adaptive update rates.
result GDU outperforms LSTM and GRU on various tasks, including pathological and natural data.
Improved GRU model with weighted time-delay feedback for long-term dependencies.
problem Modeling long-term dependencies in sequential data.
method Introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism.
result τ-GRU outperforms state-of-the-art models on various tasks.
ST-SAN predicts flow with spatial-temporal dependencies using self-attention.
problem Challenges in predicting flow due to spatial-temporal dependencies.
method Spatial-Temporal Self-Attention Network (ST-SAN) that addresses temporal and spatial dependencies.
result Significant improvement in flow prediction accuracy (9% in inflow, 4% in outflow) compared to state-of-the-art methods.
AntisymmetricRNN improves RNN trainability without extra computation.
problem Difficulty in learning long-term dependencies in RNNs.
method Connecting RNNs to ordinary differential equations and proposing AntisymmetricRNN.
result AntisymmetricRNN captures long-term dependencies more predictably and efficiently.
CFTM uses fractional Brownian motion for dynamic topic modeling.
problem Identifying long-term dependency or roughness in topic and word distributions over time.
method Continuous Time Fractional Topic Model (cFTM) incorporating fractional Brownian motion.
result cFTM captures long-term dependency or roughness in topic and word distributions.
Neural M3 model adapts to diverse user behaviors over short and long timeframes.
problem Adapting to diverse user behaviors over short and long timeframes.
method Neural Multi-temporal-range Mixture Model (M3) combining short-term and long-term models with a learned gating mechanism.
result M3 consistently outperforms state-of-the-art sequential recommendation methods.
KEDformer improves long-term time series forecasting with seasonal-trend decomposition.
problem Accurate long-term predictions in energy, finance, and meteorology.
method Knowledge extraction-driven framework integrating seasonal-trend decomposition.
result KEDformer enhances model's ability to capture short-term and long-term patterns.
Proposes FRU to stabilize gradients and improve long-term dependencies in RNNs.
problem Challenges in training RNNs for tasks with long-term dependencies.
method Introduces Fourier Recurrent Units (FRU) that stabilize gradients and improve expressivity.
result FRU stabilizes gradients and has stronger expressive power, leading to better performance.
Deep RNNs excel at capturing long-term dependencies in sequential data.
problem Lack of a formal measure for RNNs' long-term memory capacity.
method Introduced a measure called Start-End separation rank to quantify RNNs' ability to model long-term dependencies.
result Deep RNNs support Start-End separation ranks that are combinatorially higher than shallow ones.
TiDE uses MLP for fast, simple long-term time-series forecasting.
problem Long-term time-series forecasting challenges.
method Time-series Dense Encoder (TiDE) based on MLP.
result TiDE matches or outperforms Transformer models while being 5-10x faster.
Mamba Hawkes Process improves modeling of event sequences with long-term dependencies.
problem Modeling mutual inhibition and nonlinearity in asynchronous event sequences.
method Introduces Mamba Hawkes Process using Mamba state space architecture.
result MHP outperforms existing models across various datasets.
Proposes RMN for learning long-term dependencies in feed-forward networks.
problem Complicated training of deep RNN architectures.
method Residual Memory Neural Network (RMN) with residual and time-delayed connections.
result RMN and BRMN outperform LSTM and BLSTM networks in learning long-term and hierarchical information.
We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory. We achieve this by extending unitary RNNs with a gating mechanism. Our model is able to outperform LSTM…
VLSTM improves HFT by handling long sequences in financial trading.
problem Handling long sequences of thousands of data points in high-frequency trading.
method Proposed VLSTM, a variant of LSTM, to address long-term dependencies.
result 3.14\% increase in F1-score over state-of-the-art models.
DSTP-RNN improves long-term multivariate time series prediction using attention-based RNN.
problem Long-term prediction of multivariate time series with spatial correlations and spatio-temporal relationships.
method Inspired by human attention mechanism, DSTP-RNN uses a dual-stage two-phase structure and multiple attentions to enhance spatial correlations and long-term dependence.
result DSTP-RNN outperforms nine baseline methods on four datasets in energy, finance, environment, and medicine.
Paper proposes a graph neural network for accurate long-term ILI prediction.
problem Limited long-term prediction performance and spatio-temporal dependency in existing models.
method Cross-location attention based graph neural network (Cola-GNN) for time series embeddings and location aware attentions.
result Proposed method shows strong predictive performance and interpretable results for long-term epidemic predictions.
THP model captures complex dependencies in event sequences efficiently.
problem Inability of existing models to capture long-term dependencies in event sequences.
method Transformer Hawkes Process (THP) model using self-attention mechanism.
result THP outperforms existing models in likelihood and event prediction accuracy.
Enhanced LSTM model learns complex temporal dependencies.
problem Modeling long-term dependencies in sequential data.
method Gamma-LSTM with hierarchical memory units and gates.
result Gamma-LSTM outperforms regular and stacked LSTMs in sequence prediction.
SAMBA predicts stock returns efficiently using Mamba and graph neural networks.
problem Accurate stock price predictions for financial returns.
method SAMBA integrates Mamba architecture with graph neural networks to achieve near-linear computational complexity.
result SAMBA significantly outperforms state-of-the-art models in prediction accuracy.
Structured state space models improve ECG classification and reveal new insights.
problem Improving ECG analysis through deep learning.
method Applying structured state space models to capture long-term dependencies in ECG data.
result SSMs lead to significant improvements in ECG classification over current state-of-the-art.
We investigate the temporal correlations and multifractal nature of trading volume of 22 liquid stocks traded on the Shenzhen Stock Exchange in 2003. We find that the trading volume exhibit size-dependent non-universal long memory and multifractal nature. No crossover in the power-law dependence of the detrended fluctu…
NeuroMemFPP uses LSTM to estimate FPP parameters with high accuracy.
problem Estimating parameters of fractional Poisson process with memory and long-range dependence.
method Recurrent Neural Network (RNN), specifically Long Short-Term Memory (LSTM), for parameter estimation.
result The LSTM-based approach reduces MSE by about 55.3% compared to traditional MOM method.
This paper presents empirical evidence using recently developed techniques in econophysics suggesting that the degree of long-range dependence in interest rates depends on the conduct of monetary policy. We study the term structure of interest rates for the US and find evidence that global Hurst exponents change dramat…
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Quaternion neural networks improve distant speech recognition.
problem Challenges in distant speech recognition due to noise and reverberation.
method Quaternion neural networks process multi-channel audio signals as quaternion entities, capturing internal and external dependencies.
result QLSTM outperforms real-valued LSTM on multi-channel distant speech recognition tasks.
Transformer learns long-term dependencies from real-world data.
problem Sample inefficiency in deep reinforcement learning.
method Transformer architecture applied to autoregressive real-world episodes.
result Transformer-based world model generates meaningful experience.
Preformer improves Transformer for long-term time series forecasting.
problem Transformer's quadratic complexity and lack of context-awareness for long-term forecasting.
method Introduces Multi-Scale Segment-Correlation mechanism for efficient time series segmentation and context-aware attention.
result Preformer outperforms other Transformer-based methods in long-term time series forecasting.
The diagonal effect of orders is well documented in different markets, which states that orders are more likely to be followed by orders of the same aggressiveness and implies the presence of short-term correlations in order flows. Based on the order flow data of 43 Chinese stocks, we investigate if there are long-rang…
Stacked LSTM networks improve traffic volume forecasting.
problem Accurate traffic volume prediction for better planning.
method Applying stacked Long Short-Term Memory (LSTM) networks for time series forecasting.
result Stacked LSTM networks enhance the accuracy of traffic volume predictions.
Existing language models such as n-grams for software code often fail to capture a long context where dependent code elements scatter far apart. In this paper, we propose a novel approach to build a language model for software code to address this particular issue. Our language model, partly inspired by human memory, i…
Paper uses LSTM autoencoder for ADS-B data to detect surveillance aircraft.
problem Detecting surveillance aircraft from ADS-B flight data.
method LSTM-based sequence autoencoder for ADS-B data.
result LSTM autoencoder effectively learns features for detecting surveillance aircraft.
A new method backtracks through a few key past states to speed up credit assignment in long sequences.
problem Computational inefficiency of back-propagation through time for long sequences.
method Sparse attentive backtracking using learned attention mechanisms to skip connections.
result Matches or outperforms regular BPTT and truncated BPTT in tasks with long-term dependencies.
FPG uses fractional calculus for efficient reinforcement learning with long-term memory.
problem High variance and inefficient sampling in standard policy gradient methods for long-term temporal modeling.
method Fractional Policy Gradients (FPG) incorporating Caputo fractional derivatives for power-law temporal correlations.
result Achieves asymptotic variance reduction of order O(t^(-alpha)) and sample efficiency gains.
In this paper, we present the results of Monte Carlo simulations for two popular techniques of long-range correlations detection - classical and modified rescaled range analyses. A focus is put on an effect of different distributional properties on an ability of the methods to efficiently distinguish between short and …
Hamiltonian RNN controls hidden states gradient for long-term dependencies.
problem Challenges in learning long-term dependencies in RNNs.
method Symplectic discretization of Hamiltonian system to control gradient.
result Hamiltonian RNN outperforms other RNNs without hyperparameter optimization.
New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.
problem Memory constraints in training deep hierarchical RNNs.
method Replace gradient backpropagation with locally computable losses in deep HRNNs.
result Memory requirements reduced by a factor exponential in hierarchy depth.