PSDs improve RNN performance by predicting future observations.
problem Modeling dynamic processes with unknown latent states.
method Augmenting RNNs with Predictive-State Decoders (PSDs) that target predicting future observations.
result PSDs improve statistical performance of state-of-the-art RNNs with fewer iterations and less data.
PSRNNs combine RNN and PSR insights for system filtering and prediction.
problem Modeling dynamical systems efficiently and accurately.
method Combines insights from RNNs and PSRs using bilinear transfer functions and tensor decomposition.
result PSRNNs outperform other models in filtering and prediction tasks across multiple datasets.
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
problem Highly variable and noisy sequential data.
method PF-RNNs maintain a latent state distribution as a set of particles, updating with Bayes rule.
result PF-RNNs outperform standard RNNs on various sequence prediction tasks.
Recurrent Neural Networks handle sequential data better than traditional networks.
problem Handling temporal context and gradient flow in training RNNs.
method Explains the structure and challenges of RNNs, including training complexities.
result RNNs improve performance in various sequential tasks.
Next-gen reservoir computers fail to predict complex processes, highlighting need for better architectures.
problem Predicting complex, non-Markovian processes with recurrent neural networks.
method Lower bound from Fano's inequality and analysis of large probabilistic state machines.
result Next-generation reservoir computers have an error probability at least 60% higher than optimal for highly non-Markovian processes.
RNF learns distinct representations for Bayesian filtering steps, improving time series prediction accuracy and uncertainty.
problem Improving time series prediction accuracy and uncertainty using distinct representations for Bayesian filtering steps.
method Introduces Recurrent Neural Filter (RNF) architecture that learns distinct representations for each Bayesian filtering step.
result RNF improves accuracy of one-step-ahead forecasts and provides realistic uncertainty estimates.
A novel RNN model with shuffled hidden states.
problem Improving recurrent neural network performance.
method Permuting hidden states and applying learned functions.
result Competitive results compared to literature baselines.
Neural network predicts purchasing intent without feature engineering.
problem Predicting purchasing intent in ecommerce with minimal feature engineering.
method Trainable vector spaces, multi-layer recurrent neural networks, parameter sharing, skip connections.
result Classification accuracy exceeds state-of-the-art on benchmark datasets.
GRNN predicts multiple medical concepts from patient discharge summaries.
problem Understanding medical concepts in patient discharge summaries.
method Grounded Recurrent Neural Network (GRNN) that ties labels to specific dimensions of the hidden state.
result GRNN outperforms baselines in predicting medical concepts from discharge summaries.
Mod-DeepESN improves echo state networks for complex, multi-scale tasks.
problem Efficiency in solving complex, multi-scale temporal tasks.
method Incorporates intrinsic plasticity into a modular deep echo state network architecture.
result Significantly outperforms state-of-the-art for time series prediction tasks.
Generative model uses recurrent neural networks to predict graph edges.
problem Graph generation with Machine Learning is an open problem.
method Sequential graph generation with two recurrent neural networks.
result Approach generates unique graphs with similar structural properties.
HMRNN combines HMMs and neural networks for Alzheimer's disease forecasting.
problem Improving disease progression modeling with hidden states not fully known.
method Developed HMRNN combining HMMs and recurrent neural networks.
result HMRNN improves disease forecasting and offers novel clinical interpretation.
Recurrent neural networks improve time series forecasting accuracy.
problem Time series forecasting is challenging, especially for sequential data.
method A recurrent neural network framework for feature engineering, prediction, and evaluation is presented.
result The LSTM and GRU networks outperform traditional methods in forecasting accuracy.
Paper uses AI to predict medications from medical codes, improving accuracy in healthcare.
problem Predicting medications from incomplete or incorrect medical codes is challenging.
method Robust Recurrent Neural Networks (RNNs) with decay mechanism and noise injection.
result The method accurately predicts medication orders from contaminated medical codes.
RPSP networks combine PSRs and RNNs for reinforcement learning in POE.
problem Learning in partially observable environments.
method Recurrent filter with PSR, reactive policy, gradient descent.
result RPSP networks outperform memory-preserving models.
MIM networks predict non-stationary spatiotemporal dynamics using differential signals.
problem Predicting non-stationary spatiotemporal processes with high-order variations.
method Memory In Memory (MIM) networks with cascaded memory modules.
result Achieved state-of-the-art results on four spatiotemporal prediction tasks.
Improved RNN predicts patient health from ICU data.
problem Slowed response times and missed true alarms from ICU alarms.
method Compare sliding window and recurrent predictors for ICU multivariate time series.
result RNN slightly improved for three out of four patient state-of-health targets.
This paper enhances LSTM neural networks for multi-variable time series data, providing interpretable insights.
problem Accurate prediction of multi-variable time series data with interpretable insights.
method Variable-wise hidden states and a mixture attention mechanism to model the generative process of the target variable.
result Enhanced prediction performance by capturing the dynamics of different variables.
A new model captures diffusion dynamics in networks using hidden states.
problem Capturing temporal relationships and hidden content trajectories in network diffusion.
method A topological recurrent neural model that embeds diffusion history as hidden states.
result Good experimental performances for diffusion modeling and prediction.
Reservoir computers and RNNs fall short of optimal prediction for stochastic PDFA.
problem Predicting stochastic processes generated by probabilistic deterministic finite-state automata.
method Generalized linear models, Reservoir computers, and Long Short-Term Memory (LSTM) RNNs were tested.
result Each method can fall short of maximal predictive accuracy by up to 50% after training.
Proposes Fusion Recurrent Neural Network for sequence data.
problem Improving sequence learning for practical applications.
method Fusion module and Transport module for sequence data.
result Fusion RNN performs comparably to state-of-the-art RNNs.
Reservoir computing predicts chaotic systems for long horizons with sparse updates.
problem Predicting chaotic systems with long horizons using limited data.
method Sparse, time-dependent data inputs into reservoir computing.
result Achieves arbitrarily long prediction horizons for chaotic systems.
Deep learning predicts business process events with high precision.
problem Predicting next events in business processes.
method Recurrent neural networks applied to deep learning.
result Deep learning surpasses state-of-the-art in prediction precision.
Recent approaches based on artificial neural networks (ANNs) have shown promising results for short-text classification. However, many short texts occur in sequences (e.g., sentences in a document or utterances in a dialog), and most existing ANN-based systems do not leverage the preceding short texts when classifying …
Method learns to predict agent interactions from partial observations.
problem Predicting interactions between multiple agents from incomplete data.
method Graph-Structured Variational Recurrent Neural Network (Graph-VRNN) trained end-to-end.
result Graph-VRNN outperforms baselines on sports datasets.
Improved multi-step prediction of drivable space for autonomous vehicles.
problem Accurate prediction of drivable space for safer, more comfortable navigation.
method Recurrent Neural Network (RNN) architectures trained on KITTI dataset, incorporating motion features.
result Significant improvement in prediction accuracy over state-of-the-art methods.
DA-RNN improves time series prediction by selectively using past values and relevant driving series.
problem Lack of effective methods to capture long-term temporal dependencies and select relevant driving series.
method Dual-stage attention-based recurrent neural network (DA-RNN) with input and temporal attention mechanisms.
result DA-RNN outperforms state-of-the-art methods in time series prediction.
SHARE predicts city-wide parking availability using a hierarchical graph neural network.
problem Predicting city-wide parking availability is challenging due to spatial and temporal autocorrelation.
method SHARE uses a hierarchical graph convolution structure with contextual and soft clustering blocks, a recurrent neural network, and a parking availability approximation module.
result SHARE outperforms state-of-the-art baselines in predicting city-wide parking availability.
Lipschitz RNNs improve stability and performance in various tasks.
problem Improving stability and performance of RNNs.
method Introduced a Lipschitz recurrent unit with a linear and Lipschitz nonlinear component for stability analysis.
result Lipschitz RNNs outperform existing units on benchmark tasks.
Neural network model predicts alternating event-free periods.
problem Dynamic prediction of alternating recurrent events with statistical nuance.
method Developed an online dynamic prediction framework using neural network theory.
result Outstanding performance in predicting alternating recurrent event-free time.
Optimal stock price prediction model using recurrent neural networks with RMSprop optimizer.
problem Stock price prediction using neural networks.
method Comparison of fully connected, convolutional, and recurrent architectures; inclusion of three optimization techniques.
result Single layer recurrent neural network with RMSprop optimizer produces optimal results with validation and test MAE of 0.0150 and 0.0148 respectively.
Study improves epidemic forecasting with a sparsified GSRNN.
problem Epidemic forecasting on real-world health data.
method Graph-structured recurrent neural network (GSRNN) with sparsification via transformed-ℓ1 penalty. result Maintained prediction accuracy with 70% of network weights being zero.
This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences. A recurrent neural network generates individual words, thus reaping the benefits of discriminatively-trained vector representati…
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…
Recurrent neural networks trained on regular languages exhibit stable states that can recover from noise.
problem Stability of internal states in recurrent neural networks trained on regular languages.
method Empirical study with analysis of network activation and transitions between states.
result Recurrent neural networks trained on regular languages can recover from random perturbations and maintain stable states.
Paper compares training and sampling schemes for character-level RNNs.
problem Optimizing training and sampling schemes for character-level RNNs.
method Tested various training and sampling schemes on different datasets and architectures.
result The choice of training and sampling scheme affects stability, performance, and implementation effort.
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
CMDRNN predicts user location using WiFi fingerprints with deep learning.
problem Predicting user activity with WiFi fingerprints is challenging due to high dimensionality.
method Combines CNN, RNN, and MDN to model high-dimensional time-series data.
result CMDRNN effectively predicts user location using WiFi fingerprints.
A novel RNN survival model predicts web user return times.
problem Predicting when web users will return.
method Developed a novel RNN survival model that combines RNN's feature learning with survival analysis's non-returning user representation.
result Successfully predicts return times with superior discrimination between returning and non-returning users.
Improved traffic forecasting model handles missing data.
problem Short-term traffic forecasting with missing values.
method Proposed SBU-LSTM architecture with bidirectional and unidirectional LSTM.
result Superior performance in accuracy and robustness for network-wide traffic prediction.
Model predicts event sequences using neural networks.
problem Predicting future events based on past patterns.
method Neural Hawkes Process with LSTM for evolving event intensities.
result Generative model achieves competitive performance.
GeoSeq2Seq integrates geometric principles into neural networks for improved predictions.
problem Improving the geometric representation of neural network embeddings.
method Proposes a new Seq2Seq network that uses Fisher information metric to encode latent embeddings as Fisher kernels of Gaussian Mixture Models.
result GeoSeq2Seq outperforms traditional methods by 10-15% in predicting shortest routes on graphs.
The paper evaluates various machine learning models for predicting industrial aging processes.
problem Accurately predicting industrial aging processes to schedule maintenance efficiently.
method Compared traditional stateless models (linear and kernel ridge regression, feed-forward neural networks) to more complex recurrent neural networks (echo state networks and LSTMs) on synthetic and real-world data.
result Recurrent models produce near perfect predictions when trained on larger datasets and maintain good performance even with domain shifts, while simpler models perform comparably on smaller datasets.
GRUs predict flu at state and city levels with lower error.
problem Influenza prediction at multiple spatial resolutions.
method Gated Recurrent Unit (GRU) neural network, real-time search data.
result GRU outperforms state-of-the-art methods for flu prediction.
Model predicts patient outcomes from EHR data by limiting feature interactions.
problem Predicting patient outcomes from noisy, missing EHR data.
method Recurrent neural network with feature interaction restriction.
result Improvement in AU-ROC for mortality prediction (1.1% improvement, p<0.01).
BiHRNN predicts inflation by leveraging hierarchical structure and bidirectional RNNs.
problem Accurate inflation forecasting is challenging due to dynamic factors and the layered structure of the Consumer Price Index.
method Bi-directional Hierarchical Recurrent Neural Network (BiHRNN) model that uses bidirectional information flow between levels and informative constraints on RNN parameters.
result BiHRNN significantly outperforms traditional RNN models in forecasting accuracy.
RNNs learn combinatorial graph problems with sample complexity bounds.
problem Learning efficient approximations for real-valued combinatorial graph problems.
method Upper bounds the sample complexity for learning real-valued RNNs.
result Real-valued RNNs can be learned with polynomial number of samples.
New neural networks with variable time constants for better time-series prediction.
problem Improving neural network performance in time-series prediction.
method Constructing networks of linear dynamical systems modulated by nonlinear gates, using numerical differential equation solvers.
result Liquid Time-Constant Networks (LTCs) yield superior performance on time-series prediction tasks.