Long Short-Term Memory (LSTM) is a well-known method used widely on sequence learning and time series prediction. In this paper we deployed stacked LSTM model in an application of weather forecasting. We propose a 2-layer spatio-temporal stacked LSTM model which consists of independent LSTM models per location in the f…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,878 papers · 148 categories
Trend · papers per month
48 results for “LSTM layers”
Long short-term memory (LSTM) has been widely used for sequential data modeling. Researchers have increased LSTM depth by stacking LSTM cells to improve performance. This incurs model redundancy, increases run-time delay, and makes the LSTMs more prone to overfitting. To address these problems, we propose a hidden-laye…
A deep learning network was used to predict future blood glucose levels, as this can permit diabetes patients to take action before imminent hyperglycaemia and hypoglycaemia. A sequential model with one long-short-term memory (LSTM) layer, one bidirectional LSTM layer and several fully connected layers was used to pred…
We introduce Independently Recurrent Long Short-term Memory cells: IndyLSTMs. These differ from regular LSTM cells in that the recurrent weights are not modeled as a full matrix, but as a diagonal matrix, i.e.\ the output and state of each LSTM cell depends on the inputs and its own output/state, as opposed to the inpu…
Adapts LRP for LSTM to explain sequential data.
problem Lack of explainable AI for LSTM models.
method Extends LRP to LSTM, introduces new propagation scheme.
result Delivers faithful explanations for LSTM predictions.
A new multi-layer attention mechanism improves speech keyword recognition accuracy.
problem Inaccurate attention weights in LSTM networks for speech keyword recognition.
method Introducing information from layers prior to feature extraction into attention weights calculations.
result The proposed multi-layer attention mechanism leads to more accurate attention weights and improved keyword spotting performance.
FedMA improves federated learning for neural nets, matching and averaging model elements.
problem Federated learning's communication burden and model performance.
method Layer-wise construction of shared model by matching and averaging hidden elements.
result FedMA outperforms state-of-the-art algorithms on deep CNN and LSTM architectures.
Bidirectional LSTM predicts seizures with 84% accuracy.
problem Predicting seizures for epilepsy patients to prevent drug side effects.
method Trained EEG data from canines on a double Bidirectional LSTM layer.
result AUC of 0.84 on test dataset, significantly better than SVM and GRU networks.
Develops LSTM for predicting neuronal dynamics over long time-horizons.
problem Understanding and controlling complex brain behaviors.
method Long Short-Term Memory (LSTM) neural network architecture for multi-time step predictions.
result LSTM improves short time-horizon prediction accuracy and multi-time step predictions of neuronal dynamics.
Stacked LSTM networks improve traffic volume forecasting.
problem Accurate traffic volume prediction for better planning.
method Applying stacked Long Short-Term Memory (LSTM) networks for time series forecasting.
result Stacked LSTM networks enhance the accuracy of traffic volume predictions.
Meta-learning consists in learning learning algorithms. We use a Long Short Term Memory (LSTM) based network to learn to compute on-line updates of the parameters of another neural network. These parameters are stored in the cell state of the LSTM. Our framework allows to compare learned algorithms to hand-made algorit…
Hybrid model combines LSTM and ETS for mid-term electric load forecasting.
problem Mid-term electric load forecasting accuracy.
method Combines LSTM, ETS, and ensemble learning; uses dilated LSTM for long-term relationships.
result High performance and competitiveness compared to classical and machine learning models.
Long Short-Term Memory (LSTM) is a popular approach to boosting the ability of Recurrent Neural Networks to store longer term temporal information. The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the…
Simplified LSTM models improve sentiment analysis on Twitter debate data.
problem Performing sentiment analysis on long sequence data from Twitter debates.
method Developed six parameter-reduced LSTM models (slim LSTM) for faster training and reduced computational cost.
result Slim LSTM models outperform standard LSTM model in sentiment analysis of GOP Debate Twitter dataset.
A model integrates CNN and LSTM with LLM for better stock forecasting.
problem Complex stock market prediction with volatile patterns and multifaceted data.
method Two-Level Conv-LSTM Neural Network with LLM for holistic stock advising.
result Improved prediction accuracy and contextual insights.
LSTM improves cross-network recommendations by capturing user preference changes and irregular time intervals.
problem Offline cross-network recommender solutions fail to capture user preference changes and dynamic environments.
method Proposes a multi-layered LSTM network with attention mechanisms, higher order interactions, and time-aware gates.
result The model consistently outperforms state-of-the-art in accuracy, diversity, and novelty.
A minimal RNN model learns to generate new characters from a single example.
problem Learning efficiency and compositional ability in AI.
method Minimal RNN model with one-shot inference mechanism.
result The model can recombine previously learned substructures to generate new characters.
Improved traffic forecasting model handles missing data.
problem Short-term traffic forecasting with missing values.
method Proposed SBU-LSTM architecture with bidirectional and unidirectional LSTM.
result Superior performance in accuracy and robustness for network-wide traffic prediction.
Paper proposes LSTM for automatic music generation.
problem Generating music without human intervention.
method Recurrent Neural Networks (RNN), specifically LSTM.
result Model generates new melodies with high accuracy.
Applying neural-networks on Question Answering has gained increasing popularity in recent years. In this paper, I implemented a model with Bi-directional attention flow layer, connected with a Multi-layer LSTM encoder, connected with one start-index decoder and one conditioning end-index decoder. I introduce a new end-…
Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the…
Enhanced LSTM with multiple kernels and attention improves video action recognition.
problem Improving motion understanding in video analysis.
method Proposed a Network-in-LSTM approach with multiple convolutional kernels and layers, and an attention-based mechanism.
result Improves accuracy in supervised classification on UCF-101 and Sports-1M datasets.
A new LSTM architecture improves time series forecasting efficiency.
problem Efficiency and accuracy in time series forecasting using linear models.
method Attention-free LSTM architecture for time series prediction.
result Improved prediction capacity and efficiency compared to LSTM.
Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within mu…
Sleep disorders are implicated in a growing number of health problems. In this paper, we present a signal-processing/machine learning approach to detecting arousals in the multi-channel polysomnographic recordings of the Physionet/CinC Challenge2018 dataset. Methods: Our network architecture consists of two components.…
Model compresses event-like contexts using gated surprise signals.
problem Perceiving a dynamic world as organized events.
method Hierarchical, surprise-gated recurrent neural network architecture.
result Achieves best performance on multiple event processing tasks.
As deep neural networks continue to revolutionize various application domains, there is increasing interest in making these powerful models more understandable and interpretable, and narrowing down the causes of good and bad predictions. We focus on recurrent neural networks, state of the art models in speech recogniti…
BiLSTM models outperform LSTM and ARIMA in financial time series forecasting.
problem Comparing forecasting accuracy of ARIMA, LSTM, and BiLSTM models.
method Behavioral analysis and comparison of BiLSTM and LSTM models.
result BiLSTM models provide better predictions than ARIMA and LSTM models.
The Long-Short-Term-Memory Recurrent Neural Networks (LSTM RNNs) are a popular class of machine learning models for analyzing sequential data. Their training on modern GPUs, however, is limited by the GPU memory capacity. Our profiling results of the LSTM RNN-based Neural Machine Translation (NMT) model reveal that fea…
Volatility-inspired -LSTM cellq-fin.CP
The paper introduces a new -LSTM cell for volatility forecasting using stylized facts.
problem Lack of explainability and stylized knowledge in neural network volatility modeling.
method Introduces a new -LSTM cell with a stochastic processing layer, designed to incorporate stylized facts about volatility.
result Shows good out-of-sample forecasting performance with the -LSTM cell.
One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we…
Improved NER performance on imbalanced data.
problem Highly unbalanced training data in NER tasks.
method Adapted a neural architecture with CRF and BI-LSTM layers, using pre-trained embeddings. Introduced a two-class split to optimize performance.
result Significant improvement in performance for weak classes with minimal training data.
Improved hydrological models using LSTM networks on large datasets.
problem Traditional hydrological models degrade when applied to multiple basins.
method Long Short-Term Memory (LSTM) networks trained on 531 basins from CAMELS data.
result Single LSTM model outperforms regional and basin-specific models.
Bi-LSTM with attention predicts gold and bitcoin prices accurately.
problem Predicting prices of gold and bitcoin in financial derivatives markets.
method Bidirectional LSTM neural network with attention mechanism, feature engineering, two-layer deep learning.
result Achieved 71.94% and 73.03% accuracy for bitcoin and gold respectively.
Recurrent neural networks (RNNs) have been drawing much attention with great success in many applications like speech recognition and neural machine translation. Long short-term memory (LSTM) is one of the most popular RNN units in deep learning applications. LSTM transforms the input and the previous hidden states to …
We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with drop-in replacements for common layers. This enables composition via a unified abstraction over deterministic and stochastic functions and allows for scalability via the unde…
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
Gating is a key technique used for integrating information from multiple sources by long short-term memory (LSTM) models and has recently also been applied to other models such as the highway network. Although gating is powerful, it is rather expensive in terms of both computation and storage as each gating unit uses a…
Predicting movement of objects while the action of learning agent interacts with the dynamics of the scene still remains a key challenge in robotics. We propose a multi-layer Long Short Term Memory (LSTM) autoendocer network that predicts future frames for a robot navigating in a dynamic environment with moving obstacl…
This work is about recognizing human activities occurring in videos at distinct semantic levels, including individual actions, interactions, and group activities. The recognition is realized using a two-level hierarchy of Long Short-Term Memory (LSTM) networks, forming a feed-forward deep architecture, which can be tra…
Improved fraud detection in finance with quantum-enhanced federated learning.
problem Challenges in detecting financial fraud with traditional methods.
method Hybrid quantum-enhanced federated learning framework combining quantum LSTM with privacy-preserving techniques.
result Approximately 5% improvement in performance metrics compared to conventional models.
Machine learning improves American option pricing accuracy.
problem Complexities of American options and traditional models' limitations.
method Monte Carlo simulations combined with machine learning algorithms (Least Square Method, LSTM, GRU).
result GRU model outperforms LSTM in predicting bid prices, enhancing accuracy and stability.
Study improves stock index prediction accuracy using TPE-GRNN models.
problem Enhancing prediction of stock index prices in volatile markets.
method Gated recurrent neural networks (LSTM, GRU) combined with TPE Bayesian optimization.
result TPE-LSTM method shows lowest MAPE (best accuracy) for NIFTY 50 index prediction.
ShrinkML uses RL to compress ASR models efficiently.
problem Large and complex ASR models reduce accuracy.
method Reinforcement Learning for compression of LSTM layers using SVD.
result RL-based compression achieves better accuracy than manual methods.
Neural language modeling (LM) has led to significant improvements in several applications, including Automatic Speech Recognition. However, they typically require large amounts of training data, which is not available for many domains and languages. In this study, we propose a multilingual neural language model archite…
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
Study clusters bank customers using LSTM and DTW.
problem Efficiently segmenting bank customers for targeted offers.
method Encoder-decoder LSTM network and Dynamic Time Warping (DTW).
result Hybrid method yields more accurate clusters.