Three GRU variants reduce parameters in RNNs, improving efficiency.
problem Reducing computational expense in RNNs.
method Three variants of GRU with reduced parameters in update and reset gates.
result Variant models perform similarly to original GRU RNN models.
SRU learns long-term dependencies without gates, outperforming LSTMs and GRUs.
problem Learning long-term dependencies in data efficiently.
method Developed an un-gated statistical recurrent unit (SRU) that keeps moving averages of statistics.
result SRU outperforms LSTMs and GRUs in various tasks, often outperforming both.
Three MGU variants reduce parameters and improve recurrent neural network performance.
problem Improving recurrent neural network performance with fewer parameters.
method Introducing three MGU variants with simplified forget-gate dynamic equations.
result MGU2 variant outperformed MGU on MNIST and RNT datasets.
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Convolutional LSTM networks outperform GRU in EEG seizure detection.
problem Seizure detection in EEG signals.
method Comparison of LSTM and GRU units, hybrid CNN-RNN architecture, various initialization and regularization methods.
result Convolutional LSTM networks achieve 30% sensitivity at 6 false alarms per 24 hours.
Improved GRU model with weighted time-delay feedback for long-term dependencies.
problem Modeling long-term dependencies in sequential data.
method Introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism.
result τ-GRU outperforms state-of-the-art models on various tasks.
New method improves learning of long-term dependencies in recurrent models.
problem Improving learning of long-term dependencies in recurrent neural networks.
method Proves learnable gates in recurrent models provide quasi-invariance to time transformations and recovers part of LSTM architecture from axiomatic approach.
result New chrono initialization of gate biases greatly improves learning of long-term dependencies.
Combines gating and tensor products for RNNs to improve performance.
problem Improving RNNs' ability to capture long-term dependencies.
method Proposes a novel RNN architecture combining gating mechanism and tensor products.
result Significant performance improvement on word-level and character-level language modeling tasks.
GORU combines unitary and gated RNNs for better long-term memory management.
problem Learning to effectively manage long-term memory in neural networks.
method Extending unitary RNNs with a gating mechanism to forget irrelevant information.
result GORU outperforms LSTMs, GRUs, and Unitary RNNs on long-term dependency tasks.
Bayesian method sparsifies gated RNNs, improving speed and interpretability.
problem Sparsifying neural networks to reduce complexity and improve performance.
method Bayesian approach to sparsify weights, neurons, and gates in LSTM architectures.
result Sparsified gated RNNs speed up forward pass and improve compression.
Inspired by LSTMs, a new neural network model mimics cortical microcircuits.
problem Understanding the computational principles of cortical microcircuits.
method Introducing a gated-recurrent neural network (subLSTM) based on inhibitory cells.
result SubLSTM units achieve similar performance to LSTM units in sequential tasks.
We sparsify gated RNNs by simplifying their structure.
problem Improving efficiency of RNNs by reducing their complexity.
method Adjust existing sparsification techniques to gated RNNs, sparsifying preactivations of gates.
result Simplified LSTM structure improves model performance and efficiency.
Study shows GRU model with dropout outperforms in Bitcoin price prediction.
problem Predicting Bitcoin price and volatility using machine learning.
method Advanced machine learning methods including GRU with recurrent dropout, feature engineering, and RMSE evaluation.
result Gated Recurrent Unit (GRU) model with recurrent dropout outperforms traditional models in Bitcoin price prediction.
RAU integrates attention into GRU for better sequence learning.
problem Lack of attention mechanism in GRU leads to information redundancy or loss.
method RAU adds an attention gate to GRU to adaptively focus on regions of interest.
result RAU consistently outperforms GRU and other methods in various tasks.
New method estimates uncertainty in GRUs without sampling.
problem Uncertainty estimation in deep learning models.
method Exponential families for deterministic uncertainty quantification.
result Sampling-free uncertainty estimation for GRUs.
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
problem Understanding and predicting the dynamics of GRUs for neural data.
method Continuous time dynamical systems analysis of GRU networks.
result GRUs can represent stable limit cycles, multi-stable dynamics, and homoclinic bifurcations but not continuous attractors.
Paper introduces a simpler gated RNN structure to better capture long-term dependencies.
problem Difficulty in learning long-term dependencies in RNNs.
method Proposes a grouped distributor unit (GDU) with partitioned hidden states and adaptive update rates.
result GDU outperforms LSTM and GRU on various tasks, including pathological and natural data.
Machine learning predicts movie genres from summaries with high accuracy.
problem Predicting movie genres from plot summaries.
method Used Naive Bayes, Word2Vec+XGBoost, Recurrent Neural Networks, and Gated Recurrent Units (GRU) for text classification and multi-label problem.
result GRU neural networks achieve the best result with a Jaccard Index of 50.0%, F-score of 0.56, and hit rate of 80.5%.
Neural network predicts falls in elderly people up to 10 minutes in advance.
problem Falls prevention in elderly people, especially in aging societies.
method Gated Recurrent Unit (GRU) based neural networks model using heart rate and mean blood pressure signals.
result Predicted syncope occurrence approximately 10 minutes before manual markers.
Improved video prediction with bijective Gated Recurrent Units.
problem Ill-posed future video prediction with high variability and error propagation.
method Introduces bijective Gated Recurrent Units for state sharing in auto-encoders.
result Significant reduction in computational cost and memory usage compared to state-of-the-art approaches.
mGRN improves multivariate time series prediction by managing marginal and joint memories.
problem Extracting dependencies in multivariate sequential data with strong serial and cross-sectional dependencies.
method Developed a novel recurrent network architecture, Memory-Gated Recurrent Networks (mGRN), with gates for marginal and joint memories.
result mGRN consistently outperforms state-of-the-art architectures on various public datasets.
Paper proposes binary-valued gates for better LSTM training.
problem LSTMs struggle with soft gates, leading to unclear information flow.
method Introduces binary-valued gates to control information flow more clearly.
result Binary-valued gates improve LSTM performance and generalization.
New eGRU unit improves keyword spotting on ultra-low-power devices.
problem Resource constraints of edge devices for neural network deployment.
method Optimized recurrent unit architecture for ultra-low power.
result eGRU is 60x faster and 10x smaller than GRU, maintaining accuracy.
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
problem Training challenges in RNNs due to exploding or vanishing gradients.
method Random matrix theory and mean-field theory applied to GRUs and LSTMs.
result Gates in GRUs and LSTMs lead to accumulation of slow modes and control phase-space complexity.
New LSTM variants reduce complexity and parameters without sacrificing performance.
problem Complexity and large parameters in LSTM networks.
method Eliminating combinations of gating signals to reduce parameters.
result Three new LSTM variants achieve comparable performance to standard LSTM with fewer parameters.
DeepProteomics uses neural networks to classify protein families efficiently.
problem Lack of functional annotation for many protein sequences in databases.
method Used RNN, LSTM, GRU, and deep neural network models on a dataset of 40,433 proteins.
result Achieved maximum 78% accuracy in classifying protein families.
A new recurrent unit alleviates vanishing gradients for long-term dependencies.
problem Vanishing gradients in recurrent neural networks make long-term dependencies hard to model.
method Proposes a new NRU architecture that avoids saturating activation functions and gates.
result Demonstrates superior performance across various tasks with and without long-term dependencies.
T-GCN predicts traffic using neural networks for spatial and temporal data.
problem Accurate real-time traffic forecasting in urban networks.
method Combines GCN for spatial and GRU for temporal data analysis.
result T-GCN outperforms state-of-the-art baselines on real-world traffic datasets.
A forget-gate-only LSTM outperforms standard LSTM on benchmark datasets.
problem The necessity of all gates in LSTM networks.
method A forget-gate-only LSTM with chrono-initialized biases.
result The forget-gate-only LSTM outperforms standard LSTM on MNIST and pMNIST datasets.
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
DeepESNs outperform ESN and GRUs in multivariate time-series prediction.
problem Comparing DeepESNs and gated RNNs for multivariate time-series prediction.
method Experimental comparison of DeepESNs and gated RNNs (Gated Recurrent Units, Long Short-Term Memory) on 4 polyphonic music tasks.
result DeepESNs outperform ESN and GRUs in terms of prediction accuracy and efficiency.
Study proposes GRU-D networks for missing value handling in road surface friction prediction.
problem Missing values in road surface friction data affect prediction accuracy.
method Gated Recurrent Unit (GRU) network with decay mechanism.
result GRU-D networks outperform baseline models in road surface friction prediction.
Paper presents RGNN for better graph node representation learning.
problem Node representation learning with graph neural networks.
method Recurrent Graph Neural Network (RGNN) with recurrent units.
result RGNN achieves state-of-the-art results on three benchmarks.
New kernel-based models improve on traditional neural methods in sequence modeling.
problem Sequence modeling challenges in natural language processing and neuroscience.
method Kernel-based recurrent neural networks and convolutional neural networks.
result Kernel-based models perform on par or better than traditional neural methods.
Neural network combines GRU and SVM for better intrusion detection.
problem Improving accuracy in binary classification for network intrusion detection.
method Integrates GRU with SVM as final output layer, replacing Softmax and cross-entropy.
result GRU-SVM model outperforms conventional GRU-Softmax model in accuracy and prediction time.
Paper presents a neural network method for automated bug and ticket classification.
problem Automated classification of bug and ticket content in systems.
method Recurrent neural network with hierarchical attention mechanism.
result The method outperforms previous approaches on two datasets.
Bayesian approach improves neural network recurrence.
problem Improving neural network recurrence mechanisms.
method Introducing Bayesian recurrence relations and gates.
result Bayesian approach can perform as well as or better than conventional recurrent networks.
Paper proposes a hybrid MTL framework for improved stock market prediction accuracy.
problem Inaccurate stock market predictions due to financial data's complexities.
method Multi-layer hybrid MTL structure with Transformer, BiGRU, and KAN.
result Achieved low MAE (1.078), MAPE (0.012), and high R^2 (0.98) compared to other models.
CRUs model irregular time series with continuous hidden states.
problem Handling irregular time intervals in sequential data.
method Continuous Recurrent Units (CRUs) that integrate hidden states via a linear stochastic differential equation.
result CRUs outperform methods based on neural ordinary differential equations in irregular time series interpolation.
Improved time series classification with GRU-FCN model.
problem Time series classification challenges.
method Hybrid LSTM-GRU model for univariate time series classification.
result GRU-FCN model outperforms state-of-the-art models.
A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforwa…
Neural network predicts cardiovascular events from EHRs with high accuracy.
problem Predicting onset of cardiovascular diseases from electronic health records.
method Multi-task gated recurrent units with attention mechanism.
result Model outperforms clinical risk scores in predicting stroke and myocardial infarction.
GRU-D detects age-specific missing patterns in vital signs.
problem Temporal missingness in clinical time series data.
method Gated recurrent unit with decay mechanisms (GRU-D) trained on MIMIC-IV vital signs.
result GRU-D achieves AUROC 0.780 and AUPRC 0.810 on bootstrapped data.
Time-aware neural models improve system identification from unevenly sampled data.
problem Improving system identification from continuous variables with unevenly sampled time data.
method Introduced a time-aware and stationary extension of recurrent neural networks.
result Demonstrated improved performance on industrial input/output processes.
Deep learning predicts patient trajectories in open Mimic-III dataset.
problem Predicting future medical conditions from patient history.
method Two parallel bi-directional Minimal Gated Recurrent Unit networks trained on Mimic-III dataset.
result Significant improvements in automated medical prognosis measured by Recall@k.
Theory explains how recurrent networks remember sequences.
problem Understanding how recurrent networks remember sequences and perform well.
method Mean field theory and random matrix theory applied to RNNs with gating mechanisms.
result Gated RNNs outperform non-gated RNNs in remembering sequences.
Real-time fetal abdominal aorta measurement from ultrasound images.
problem Automating the challenging task of measuring fetal abdominal aorta diameter from ultrasound images.
method Proposes a neural network architecture with three blocks: convolutional layer, Convolution Gated Recurrent Unit (C-GRU), and CyclicLoss.
result Significantly improved accuracy and real-time execution speed compared to previous methods.