Proposes a new Deep Recurrent Double Q-Learning model for Atari games.
problem Improving Deep Reinforcement Learning models for Atari games.
method Integrates Double Q-Learning and Recurrent Networks (LSTM, DRQN) for Atari games.
result Demonstrates improved performance in Atari games compared to existing models.
Efficient deep GNNs achieve state-of-the-art performance without training.
problem Efficiency issue in deep graph neural networks.
method Representing graphs as fixed points of dynamical systems and using small, sparse, untrained recurrent networks.
result Small deep GNNs without training can achieve or improve state-of-the-art performance.
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
CMDRNN predicts user location using WiFi fingerprints with deep learning.
problem Predicting user activity with WiFi fingerprints is challenging due to high dimensionality.
method Combines CNN, RNN, and MDN to model high-dimensional time-series data.
result CMDRNN effectively predicts user location using WiFi fingerprints.
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
Differentiable mask prunes deep networks for vision and text.
problem Efficiently compressing deep networks for edge devices.
method Introduces a differentiable mask for sparsity induction.
result Successfully prunes weights, filters, and nodes of convolutional and recurrent networks.
Recurrent-DBN models dynamic relational data with interpretable latent structures.
problem Interpreting dynamic relational data with hidden structures.
method Recurrent Dirichlet Belief Network framework with hierarchical latent structures and efficient inference strategy.
result Recurrent-DBN discovers interpretable latent structures and improves link prediction.
Deep neural networks solve stochastic control problems with delay.
problem Challenges in stochastic control problems with delay due to path-dependence and high dimensions.
method Employing recurrent neural networks (RNNs) to parameterize policies and optimize objectives.
result RNNs, especially LSTMs, efficiently capture path-dependence and outperform feedforward networks in training and performance.
This study compares RNN and CNN for predicting wave propagation using the Saint-Venant equations.
problem Predicting wave propagation over long time periods using deep learning.
method Investigated recurrent and convolutional neural networks for their performance in predicting surface waves governed by the Saint-Venant equations.
result Convolutional networks perform at least as well as recurrent networks in predicting wave propagation.
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neura…
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted ℓ1-ℓ1 minimization algorithm to design a deep recurrent neural network. result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
We analyze generalization in deep learning models using random matrix theory.
problem Understanding the generalization error in deep learning models with random feature representations.
method Applying Random Matrix Theory to derive asymptotic generalization error formulas for various architectures.
result Linear ESNs are equivalent to ridge regression with exponentially time-weighted input covariance, revealing an inductive bias towards recent inputs.
Deep RNNs excel at capturing long-term dependencies in sequential data.
problem Lack of a formal measure for RNNs' long-term memory capacity.
method Introduced a measure called Start-End separation rank to quantify RNNs' ability to model long-term dependencies.
result Deep RNNs support Start-End separation ranks that are combinatorially higher than shallow ones.
Review of neural network expressivity and architectures.
problem Understanding neural network expressivity across different architectures.
method Comprehensive overview of approximation results for various neural network types.
result Deep neural networks offer advantages over shallow ones for specific function classes.
Deep neural nets can estimate regression with dependent data without the curse of dimensionality.
problem Regression with dependent data and structural assumptions on the regression function.
method Deep recurrent neural network estimate under suitable structural assumptions.
result Deep neural nets can circumvent the curse of dimensionality for regression with dependent data.
Two deep learning models improve indoor location prediction from WiFi fingerprints.
problem Indoor location prediction from WiFi fingerprints.
method Convolutional mixture density recurrent neural network and VAE-based semi-supervised learning model.
result Proposed models outperform existing methods in real-world datasets.
Unified theory for deep and recurrent networks using Gaussian processes.
problem Understanding capabilities and limitations of different network architectures.
method Unified derivation of mean-field theory from statistical physics of disordered systems.
result Gaussian processes yield identical Gaussian kernels for both architectures at a single time point or layer.
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
problem Estimating parameters in stochastic, nonlinear dynamical models is challenging.
method DeepBayes leverages deep recurrent neural networks to learn an estimator that minimizes mean-squared error.
result DeepBayes achieves asymptotically equivalent performance to Bayesian estimation methods.
DeepRAM evaluates and selects the best deep learning architecture for DNA/RNA binding specificity prediction.
problem Selecting the best deep learning architecture for predicting DNA/RNA binding specificity.
method Systematic exploration of various deep learning architectures using deepRAM, an end-to-end deep learning tool.
result A k-mer embedding convolutional layer and recurrent layer architecture outperforms other methods.
Deep RNNs compute American option prices and deltas efficiently.
problem Computing prices and deltas of high-dimensional American options.
method Two deep RNNs, one for price and one for delta, learn over spacetime.
result Linear time and constant memory cost compared to feedforward networks.
Deep learning predicts AF recurrence from MRI images without pre-processing.
problem Estimating AF recurrence from MRI images efficiently and accurately.
method Deep learning approach to predict shape descriptors directly from MRI images.
result Deep learning method produces similar outcomes to state-of-the-art methods.
This paper improves speech synthesis using a DGP with SRU for naturalness.
problem Improving naturalness in synthetic speech.
method Deep Gaussian process with a recurrent architecture using SRU.
result SRU-DGP outperforms other models in naturalness of synthetic speech.
The study compares econometric and deep learning models for forecasting COMEX copper futures volatility.
problem Forecasting volatility of COMEX copper futures across different time intervals.
method Econometric models (GARCH, HAR) and deep learning models (RNN, LSTM, GRU) applied to daily and hourly data.
result Deep learning models outperform econometric models in hourly data, but HAR remains the best overall for daily data.
In recent years, China, the United States and other countries, Google and other high-tech companies have increased investment in artificial intelligence. Deep learning is one of the current artificial intelligence research's key areas. This paper analyzes and summarizes the latest progress and future research direction…
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points …
This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.
problem Understanding how recurrence and non-linear dynamics in cortical networks contribute to their function.
method Transformed time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels.
result Optimal time-series classifiers can be built from random reservoir networks, demonstrating significant performance gains.
Investigates scaling deep neural networks to avoid capacity issues.
problem Avoiding the shattering problem in deep neural networks.
method Formulates conjecture and studies various architectures to determine scaling relations.
result Reveals scaling relations for deep residual networks and recurrent networks.
Study shows structured reservoirs improve deep ESN performance.
problem Improving performance of deep reservoir computing networks.
method Investigated structured reservoir topologies in deep ESNs.
result Structured reservoirs significantly enhance predictive performance.
Convolutional and Recurrent, deep neural networks have been successful in machine learning systems for computer vision, reinforcement learning, and other allied fields. However, the robustness of such neural networks is seldom apprised, especially after high classification accuracy has been attained. In this paper, we …
A new deep approach to Kalman filtering integrates uncertainty estimates efficiently.
problem Integrating uncertainty estimates into deep time-series models.
method Proposes a Recurrent Kalman Network (RKN) that learns directly using backpropagation.
result RKN obtains more accurate uncertainty estimates and slightly improved prediction performance.
Proposes CVRCF for streaming recommender systems combining deep learning and probabilistic models.
problem Streaming recommendation problem with dynamic data and complexity.
method Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework integrating stochastic processes and deep factorization models.
result Favorable performance in temporal dependency modeling and predictive accuracy compared to state-of-the-art methods.
New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.
problem Memory constraints in training deep hierarchical RNNs.
method Replace gradient backpropagation with locally computable losses in deep HRNNs.
result Memory requirements reduced by a factor exponential in hierarchy depth.
This research tackles balancing exploration and exploitation in deep RL for partially observable systems.
problem Balancing exploration and exploitation in deep RL for partially observable systems.
method Deployed and tested several techniques including adaptive and deterministic exploration strategies, and a modified quadratic loss function.
result Adaptive methods better approximate the trade-off between exploration and exploitation.
IAM improves deep RL by selectively storing influential past observations.
problem Training and performance issues with recurrent neural networks in high-dimensional environments.
method Influence-aware memory architecture that restricts input to variables influencing hidden state.
result IAM outperforms standard RNNs in training speed and policy performance.
Proposes a neural network solution for queuing dynamics.
problem Modeling service system dynamics with limited parametric assumptions.
method Combines recurrent neural networks and generative adversarial networks.
result Evaluates solution on various datasets, demonstrating effectiveness.
In this paper, we propose a novel recurrent neural network architecture for speech separation. This architecture is constructed by unfolding the iterations of a sequential iterative soft-thresholding algorithm (ISTA) that solves the optimization problem for sparse nonnegative matrix factorization (NMF) of spectrograms.…
Optimal stock price prediction model using recurrent neural networks with RMSprop optimizer.
problem Stock price prediction using neural networks.
method Comparison of fully connected, convolutional, and recurrent architectures; inclusion of three optimization techniques.
result Single layer recurrent neural network with RMSprop optimizer produces optimal results with validation and test MAE of 0.0150 and 0.0148 respectively.
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis. We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multi…
Proposes a new model for time-to-event prediction with uncertainty quantification.
problem Lack of uncertainty in time-to-event predictions using recurrent neural networks.
method Deep Kernel Accelerated Failure Time models combining RNN and sparse Gaussian Process.
result Model delivers better uncertainty estimates compared to related methods.
A new memory-efficient sign language translation model reduces weight usage.
problem Memory constraints in real-time sign language translation.
method Variational Bayesian sequence-to-sequence network with Gaussian posterior and Indian Buffet Process prior.
result The proposed model achieves substantial weight compression without compromising performance.
Deep Recurrent Q-Network improves autonomous driving in urban areas with pedestrians.
problem Challenges in urban autonomous driving due to complex road structures and unpredictable pedestrian behavior.
method Combines Deep Q-Network with LSTM for long-term memory, designed a 3-D state representation, and uses a reward function.
result The proposed DRQN-based approach outperforms rule-based methods in dense urban scenarios.
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
Deep learning models predict ICU readmission with varying accuracy.
problem Predicting ICU readmission risk using deep learning architectures.
method Several deep learning architectures including attention-based models, recurrent layers, neural ODEs, and embeddings were trained on MIMIC-III data.
result Attention-based models with neural ODEs achieved highest predictive accuracy.
Much combinatorial optimisation problems constitute a non-polynomial (NP) hard optimisation problem, i.e., they can not be solved in polynomial time. One such problem is finding the shortest route between two nodes on a graph. Meta-heuristic algorithms such as A∗ along with mixed-integer programming (MIP) methods …
Deep neural networks predict electricity consumption accurately.
problem Predicting future electricity consumption for better management.
method Used Recurrent Neural Networks (RNN) and Long Short Term Memory (LSTM) networks to predict electricity consumption based on past data.
result Both RNN and LSTM achieved an average Root Mean Square error of 0.1.
Adaptive neural network adjusts layer depth based on computational needs.
problem Deciding the number of layers in deep recurrent neural networks is challenging.
method Proposes a layer flexible recurrent neural network with adaptive computation time.
result Improves performance by 7\% to 12\% on financial and language data.
Deep-learning method improves sEMG-based gesture recognition accuracy.
problem Inter-session and inter-subject variances in sEMG-based gesture recognition.
method Recurrent Neural Networks and Domain Adaptation method.
result Our approach outperforms state-of-the-art methods in recognition accuracy.