A new model captures diffusion dynamics in networks using hidden states.
problem Capturing temporal relationships and hidden content trajectories in network diffusion.
method A topological recurrent neural model that embeds diffusion history as hidden states.
result Good experimental performances for diffusion modeling and prediction.
Study enhances neural network predictions for wave height using topological features.
problem Challenges in predicting wave heights due to short-term and long-term factors.
method Hybridization of persistent homology with neural networks for feature engineering.
result Significant improvements in R2 score and reductions in errors for various neural network models. Hexagon grid patterns emerge from conformal isometry in grid cell neural networks.
problem Understanding the algebraic, geometric, and topological properties of grid cells.
method Investigating recurrent neural network models of grid cells, focusing on Lie group and Lie algebra representations, conformal isometry, and hexagon periodic patterns.
result Conformal isometry leads to hexagon periodic patterns in grid cell responses and accurate path integration.
EvoNet predicts the evolution of dynamic graphs using a graph neural network and recurrent architecture.
problem Predicting the evolution of dynamic graphs is challenging and underexplored.
method EvoNet uses a graph neural network and recurrent architecture to predict the evolution of dynamic graphs.
result EvoNet effectively predicts the evolution of dynamic graphs on both artificial and real-world datasets.
Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node …
In this paper, we study the problem of using representation learning to assist information diffusion prediction on graphs. In particular, we aim at estimating the probability of an inactive node to be activated next in a cascade. Despite the success of recent deep learning methods for diffusion, we find that they often…
This study applies EMD to MSCI World index and converts IMFs into graphs for GNN modeling.
problem Modeling financial time series with GNNs.
method EMD, CEEMDAN, graph transformations (natural visibility, horizontal visibility, recurrence, transition graphs), topological analysis.
result High-frequency IMFs yield dense, highly connected small-world graphs; low-frequency IMFs produce sparser networks.
New method reduces training time for deep hedging networks.
problem Challenges in training deep hedging networks with large batch sizes.
method Integrates topological features to reduce batch sizes.
result Practical training of deep hedging models without sacrificing performance.
Paper develops a new model for dynamic graph representation learning.
problem Learning over dynamic graphs with changing topology and node attributes.
method Hierarchical variational model with latent random variables and semi-implicit variational inference.
result SI-VGRNN and VGRNN outperform existing methods in dynamic link prediction.
We present a new recurrent neural network topology to enhance state-of-the-art machine learning systems by incorporating a broader context. Our approach overcomes recent limitations with extended narratives through a multi-layered computational approach to generate an abstract context representation. Therefore, the dev…
QRNN uses quantum neurons to learn sequences efficiently.
problem Efficiently learning sequences with quantum computing.
method Parametrized quantum neurons and amplitude amplification.
result QRNN outperforms classical RNNs on sequence learning tasks.
Reverse engineered RNNs reveal line attractor dynamics for sentiment classification.
problem Understanding how recurrent neural networks solve sequential tasks like sentiment classification.
method Dynamical systems analysis to reverse engineer trained RNNs, identifying fixed points and linearized dynamics.
result Trained RNNs converge to low-dimensional line attractor dynamics, providing interpretable solutions.
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
problem Understanding and predicting the dynamics of GRUs for neural data.
method Continuous time dynamical systems analysis of GRU networks.
result GRUs can represent stable limit cycles, multi-stable dynamics, and homoclinic bifurcations but not continuous attractors.
Delayed-RNN approximates stacked and bidirectional RNNs.
problem Improving RNN expressiveness and representational capacity.
method Weight-constrained delayed-RNN, equivalent to stacked-RNNs, with partial acausality.
result Delayed-RNN can approximate stacked and bidirectional RNNs, outperforming them in some tasks.
RecNets use RNNs to process image channels in a compact, recurrent way.
problem Creating efficient neural network architectures for computer vision.
method Introducing RecNets with CRC layers that simulate recurrent processing of image channels.
result RecNets achieve superior size-accuracy trade-off compared to other compact models.
Algorithm discovers dynamic cell structures for better neural network performance.
problem Finding optimal neural network architectures for diverse data samples and time steps.
method Combines recurrent and recursive neural networks to dynamically search for customized cell structures.
result Achieves better prediction accuracy compared to existing models.
RANP improves neural processes for sequential data.
problem Capturing temporal order and recurrent structure from sequential data.
method Incorporated ANP into a recurrent neural network.
result RANP outperforms NPs and LSTMs in 1D regression and autonomous-driving tasks.
RFN models urban mobility demand by separating temporal and spatial variability.
problem Aligning supply and demand in MoD systems for efficient transportation.
method Recurrent flow networks with latent variables and normalizing flows.
result RFN models explicitly disentangle temporal and spatial variability in urban mobility.
SIS-RNN improves model flexibility for sequential data.
problem Limited expressive power of existing stochastic RNNs.
method Semi-implicit variational inference for implicit latent representations.
result SIS-RNN outperforms existing methods in various tasks.
RNNs compute by warping neural representations over time.
problem Understanding how RNNs perform task computations.
method Developed a Riemannian geometric framework to derive the manifold topology and geometry of RNNs.
result Dynamic warping is a fundamental feature of RNN computations.
Study on how neural network weights evolve and form structures.
problem Understanding the structure and evolution of neural network weights during training.
method Applied topological data analysis to monitor and analyze the weights of neural networks.
result Weights evolve to form trees or smooth surfaces, revealing important factors of variation.
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
This work combines recurrent models with diffusion for probabilistic time series forecasting.
problem Scalability and capturing high-dimensional distributions and cross-feature dependencies in time series forecasting.
method Combines recurrent neural networks' efficiency with diffusion models' probabilistic modeling, using stochastic interpolants and conditional generation.
result Offers scalable probabilistic time series forecasting methods.
Paper presents FPGA implementation for efficient recurrent neural networks.
problem Implementing recurrent neural networks on FPGAs for low latency.
method Developed hls4ml framework to implement LSTM and GRU layers.
result Demonstrated effective designs for both small and large models.
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward …
RNN beats Lee-Carter in forecasting mortality rates.
problem Forecasting mortality rates across different demographics.
method Long Short-Term Memory (LSTM) recurrent neural network trained on multiple countries, ages, and sexes.
result RNN model outperforms the Lee-Carter model in mortality rate forecasting.
Study on recurrent neural networks' feature selection and memorization using F1B test.
problem Conflict between feature selection and memorization in sequence learning.
method Flagged-1-Bit (F1B) test, four recurrent network models studied analytically and experimentally.
result Conflict can be resolved by gating mechanism or increasing state dimension.
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis. We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multi…
New models improve language generation by sharing intermediate states.
problem Language models struggle with past mistakes in sequence generation.
method Integrate second-order terms in hidden-state update, sharing intermediate states.
result Shared parametrization improves language modeling performance.
ENRNN uses eigenvalue normalization for short-term memory in RNNs.
problem Vanishing/exploding gradient problem and long-term dependency modeling.
method Eigenvalue normalization of recurrent matrix to simulate short-term memory.
result ENRNN outperforms existing RNN variants in experiments.
End-to-end training of recurrent neural networks for biological sequences.
problem Training recurrent neural networks for biological sequence classification.
method Generalized convolutional kernel networks with recurrent structures.
result Outperforms existing methods for protein classification tasks.
The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while…
Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique …
Recurrent neural networks can learn complex transduction problems that require maintaining and actively exploiting a memory of their inputs. Such models traditionally consider memory and input-output functionalities indissolubly entangled. We introduce a novel recurrent architecture based on the conceptual separation b…
Neural network model predicts alternating event-free periods.
problem Dynamic prediction of alternating recurrent events with statistical nuance.
method Developed an online dynamic prediction framework using neural network theory.
result Outstanding performance in predicting alternating recurrent event-free time.
New kernel-based models improve on traditional neural methods in sequence modeling.
problem Sequence modeling challenges in natural language processing and neuroscience.
method Kernel-based recurrent neural networks and convolutional neural networks.
result Kernel-based models perform on par or better than traditional neural methods.
Recurrent neural networks improve time series forecasting accuracy.
problem Time series forecasting is challenging, especially for sequential data.
method A recurrent neural network framework for feature engineering, prediction, and evaluation is presented.
result The LSTM and GRU networks outperform traditional methods in forecasting accuracy.
Improved GRU model with weighted time-delay feedback for long-term dependencies.
problem Modeling long-term dependencies in sequential data.
method Introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism.
result τ-GRU outperforms state-of-the-art models on various tasks.
Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a re…
CMDRNN predicts user location using WiFi fingerprints with deep learning.
problem Predicting user activity with WiFi fingerprints is challenging due to high dimensionality.
method Combines CNN, RNN, and MDN to model high-dimensional time-series data.
result CMDRNN effectively predicts user location using WiFi fingerprints.
Lipschitz RNNs improve stability and performance in various tasks.
problem Improving stability and performance of RNNs.
method Introduced a Lipschitz recurrent unit with a linear and Lipschitz nonlinear component for stability analysis.
result Lipschitz RNNs outperform existing units on benchmark tasks.
It is often hypothesized that a crucial role for recurrent connections in the brain is to constrain the set of possible response patterns, thereby shaping the neural code. This implies the existence of neural codes that cannot arise solely from feedforward processing. We set out to find such codes in the context of one…
The paper studies how grid cell patterns emerge in neural networks.
problem Understanding how grid cells in the brain form hexagonal firing patterns.
method Training recurrent neural networks with conformal normalization of velocity inputs.
result Conformal normalization is crucial for the emergence of hexagonal grid patterns in neural networks.
Optimal stock price prediction model using recurrent neural networks with RMSprop optimizer.
problem Stock price prediction using neural networks.
method Comparison of fully connected, convolutional, and recurrent architectures; inclusion of three optimization techniques.
result Single layer recurrent neural network with RMSprop optimizer produces optimal results with validation and test MAE of 0.0150 and 0.0148 respectively.
Improved stable RNNs trained faster with less expressibility trade-off.
problem Stable recurrent neural networks are hard to train without sacrificing expressibility.
method Implicit model structure with contraction analysis for stable models.
result Significant increase in training speed and model performance.
RNF learns distinct representations for Bayesian filtering steps, improving time series prediction accuracy and uncertainty.
problem Improving time series prediction accuracy and uncertainty using distinct representations for Bayesian filtering steps.
method Introduces Recurrent Neural Filter (RNF) architecture that learns distinct representations for each Bayesian filtering step.
result RNF improves accuracy of one-step-ahead forecasts and provides realistic uncertainty estimates.