A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Bayesian method sparsifies gated RNNs, improving speed and interpretability.
The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while…
We sparsify gated RNNs by simplifying their structure.
GCRNNs improve graph problem solving with fewer parameters.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Recurrent neural networks with various types of hidden units have been used to solve a diverse range of problems involving sequence data. Two of the most recent proposals, gated recurrent units (GRU) and minimal gated units (MGU), have shown comparable promising results on example public datasets. In this paper, we int…
Improved GRU model with weighted time-delay feedback for long-term dependencies.
Recurrent Neural Networks (RNNs) with sophisticated units that implement a gating mechanism have emerged as powerful technique for modeling sequential signals such as speech or electroencephalography (EEG). The latter is the focus on this paper. A significant big data resource, known as the TUH EEG Corpus (TUEEG), has …
New kernel-based models improve on traditional neural methods in sequence modeling.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Paper presents FPGA implementation for efficient recurrent neural networks.
Study on recurrent neural networks' feature selection and memorization using F1B test.
Cortical circuits exhibit intricate recurrent architectures that are remarkably similar across different brain areas. Such stereotyped structure suggests the existence of common computational principles. However, such principles have remained largely elusive. Inspired by gated-memory networks, namely long short-term me…
The standard LSTM recurrent neural networks while very powerful in long-range dependency sequence applications have highly complex structure and relatively large (adaptive) parameters. In this work, we present empirical comparison between the standard LSTM recurrent neural network architecture and three new parameter-r…
Recurrent neural networks have gained widespread use in modeling sequence data across various domains. While many successful recurrent architectures employ a notion of gating, the exact mechanism that enables such remarkable performance is not well understood. We develop a theory for signal propagation in recurrent net…
Successful recurrent models such as long short-term memories (LSTMs) and gated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these models have been found to improve the learning of medium to long term temporal dependencies and to help with vanishing gradient issues. We prove that learnable gates in a…
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
mGRN improves multivariate time series prediction by managing marginal and joint memories.
Gated recurrent neural networks have achieved remarkable results in the analysis of sequential data. Inside these networks, gates are used to control the flow of information, allowing to model even very long-term dependencies in the data. In this paper, we investigate whether the original gate equation (a linear projec…
We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory. We achieve this by extending unitary RNNs with a gating mechanism. Our model is able to outperform LSTM…
Paper introduces methods to integrate external knowledge into RNNs using attention mechanisms.
Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which i…
A new memory-efficient sign language translation model reduces weight usage.
Neural network predicts falls in elderly people up to 10 minutes in advance.
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SR…
RAU integrates attention into GRU for better sequence learning.
New interpretation of RNN forget gate improves learnability for long-term sequential data.
Paper presents RGNN for better graph node representation learning.
Paper introduces a simpler gated RNN structure to better capture long-term dependencies.
Bayesian approach improves neural network recurrence.
Linear Memory Network separates memory and function in RNNs.
DeepESNs outperform ESN and GRUs in multivariate time-series prediction.
Study improves stock index prediction accuracy using TPE-GRNN models.
T-GCN predicts traffic using neural networks for spatial and temporal data.
This paper uses ODE to improve RNN models for time series data.
Recurrent Neural Networks (RNNs) play a major role in the field of sequential learning, and have outperformed traditional algorithms on many benchmarks. Training deep RNNs still remains a challenge, and most of the state-of-the-art models are structured with a transition depth of 2-4 layers. Recurrent Highway Networks …
New method prunes recurrent networks efficiently, improving performance.
Improved LSTM cell for high-frequency trading forecasts.
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
Study shows GRU model with dropout outperforms in Bitcoin price prediction.
A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforwa…
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
Efficient keyword spotting model using dilated convolutions and gating.
We introduce MinimalRNN, a new recurrent neural network architecture that achieves comparable performance as the popular gated RNNs with a simplified structure. It employs minimal updates within RNN, which not only leads to efficient learning and testing but more importantly better interpretability and trainability. We…
New eGRU unit improves keyword spotting on ultra-low-power devices.
LSTMs and GRUs are the most common recurrent neural network architectures used to solve temporal sequence problems. The two architectures have differing data flows dealing with a common component called the cell state (also referred to as the memory). We attempt to enhance the memory by presenting a modification that w…
A new LSTM model reduces state updates and improves convergence for long sequences.
Model combines long-term and short-term memory using conceptors.