Recurrent-DBN models dynamic relational data with interpretable latent structures.
problem Interpreting dynamic relational data with hidden structures.
method Recurrent Dirichlet Belief Network framework with hierarchical latent structures and efficient inference strategy.
result Recurrent-DBN discovers interpretable latent structures and improves link prediction.
Study analyzes stock market dynamics using recurrence measures and transitions.
problem Understanding transitions in stock market dynamics during crises.
method Recurrence plots and networks from nonstationary stock market data.
result Recurrence measures capture transitions in stock market dynamics.
Enhanced speech emotion recognition using nonlinear recurrence dynamics.
problem Improving speech emotion recognition accuracy.
method Phase space reconstruction, Recurrence Plot, Recurrence Quantification Analysis, statistical functionals, feature fusion, Bidirectional Recurrent Neural Network.
result State-of-the-art performance on IEMOCAP with up to 10.7% improvement in accuracy.
This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.
problem Understanding how recurrence and non-linear dynamics in cortical networks contribute to their function.
method Transformed time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels.
result Optimal time-series classifiers can be built from random reservoir networks, demonstrating significant performance gains.
Mathematical study of learning long-term integration in linear RNNs.
problem How do linear recurrent neural networks learn to integrate over long timescales?
method Analytical study of linear RNNs trained to integrate white noise and damped oscillatory filters.
result Learning dynamics are described by low-dimensional effective equations for outlier eigenvalues.
Proposes a new model for better speech segmentation.
problem Improving speech segmentation accuracy.
method Integrates recurrent explicit duration variables into rSLDS and uses Pólya-gamma augmentation for inference.
result Demonstrates improved segmentation on various datasets.
Algorithm discovers dynamic cell structures for better neural network performance.
problem Finding optimal neural network architectures for diverse data samples and time steps.
method Combines recurrent and recursive neural networks to dynamically search for customized cell structures.
result Achieves better prediction accuracy compared to existing models.
Neural network model predicts alternating event-free periods.
problem Dynamic prediction of alternating recurrent events with statistical nuance.
method Developed an online dynamic prediction framework using neural network theory.
result Outstanding performance in predicting alternating recurrent event-free time.
Reverse engineered RNNs reveal line attractor dynamics for sentiment classification.
problem Understanding how recurrent neural networks solve sequential tasks like sentiment classification.
method Dynamical systems analysis to reverse engineer trained RNNs, identifying fixed points and linearized dynamics.
result Trained RNNs converge to low-dimensional line attractor dynamics, providing interpretable solutions.
Interneurons improve learning in neural networks by accelerating convergence.
problem Rapid adaptation to changing input statistics in neural networks.
method Two mathematically tractable recurrent linear neural networks were compared: one with direct recurrent connections and the other with interneurons that mediate recurrent communication.
result The network with interneurons converges more quickly than the network with direct recurrent connections, scaling logarithmically with initialization spectrum.
The paper tackles adversarial attacks on recurrent neural networks.
problem Adversarial attacks on recurrent neural networks are easy and lack theoretical guarantees.
method Inspired by dynamical systems theory, the paper dynamically computes adversarial perturbations for each timestep of the input sequence.
result The paper provides theoretical guarantees on the existence of adversarial examples and robustness margins.
Deep neural nets approximate random dynamical system trajectories uniformly in time.
problem Approximating trajectories of random dynamical systems over infinite time horizons.
method Recurrent neural networks with simple feedback structures.
result Certain random trajectories can be approximated uniformly in time to any desired accuracy.
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
problem Understanding and predicting the dynamics of GRUs for neural data.
method Continuous time dynamical systems analysis of GRU networks.
result GRUs can represent stable limit cycles, multi-stable dynamics, and homoclinic bifurcations but not continuous attractors.
RIMs improve generalization by specializing modular structures.
problem Improving generalization and robustness to changes in tasks.
method Recurrent Independent Mechanisms (RIMs) architecture with independent dynamics, sparing communication, and selective updates.
result RIMs lead to dramatic improvement in generalization on tasks with varying factors.
ASRNN adapts scales dynamically for better sequence modeling.
problem Fixed scales in multiscale RNNs don't match temporal patterns.
method Adaptively learns and adjusts scales based on temporal contexts.
result ASRNNs yield better performances with dynamical scaling.
HRM-Agent learns to navigate dynamic mazes using reinforcement learning.
problem Training HRM in dynamic, uncertain, partially observable environments.
method Reinforcement learning to train HRM-Agent.
result HRM-Agent successfully learns to navigate dynamic mazes.
Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during…
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
problem Estimating parameters in stochastic, nonlinear dynamical models is challenging.
method DeepBayes leverages deep recurrent neural networks to learn an estimator that minimizes mean-squared error.
result DeepBayes achieves asymptotically equivalent performance to Bayesian estimation methods.
AntisymmetricRNN improves RNN trainability without extra computation.
problem Difficulty in learning long-term dependencies in RNNs.
method Connecting RNNs to ordinary differential equations and proposing AntisymmetricRNN.
result AntisymmetricRNN captures long-term dependencies more predictably and efficiently.
RNN operators solve Newton's equations with large timesteps for molecular dynamics.
problem Solving Newton's equations of motion with large timesteps for molecular dynamics simulations.
method Recurrent Neural Networks (RNN) operators to solve Newton's equations using past trajectory data.
result Significant speedup in molecular dynamics simulations with timesteps up to 4000 times larger.
Boost-R uses gradient boosted trees for analyzing recurrence data.
problem Analyzing recurrence data with static and dynamic features.
method Gradient boosted additive trees with time-dependent functions.
result Estimates the cumulative intensity function of recurrent event processes.
Transformer models outperform recurrent ones in modeling hierarchical data.
problem Modeling hierarchical structure in data.
method Introducing Multiresolution Transformer Networks leveraging self-attention.
result Multiresolution Transformer Networks significantly outperform state-of-the-art models on query suggestion datasets.
Transformers can outperform feedforward and recurrent networks due to dynamic sparsity.
problem Understanding when and why Transformers outperform other neural network architectures.
method Analyzing a sequence-to-sequence data generating model with dynamic sparsity, proving sample complexity differences between feedforward, recurrent, and Transformers.
result Transformers can learn dynamic sparsity models with lower sample complexity than feedforward and recurrent networks.
Recurrent neural networks have gained widespread use in modeling sequence data across various domains. While many successful recurrent architectures employ a notion of gating, the exact mechanism that enables such remarkable performance is not well understood. We develop a theory for signal propagation in recurrent net…
Recommender system improves with temporal representations.
problem Improving interpretability and performance in recommender systems.
method Incorporates temporal representations via recurrent point process in continuous time.
result Characterizes effects of perception, interest, and seasonal changes on reviews.
Study predicts synchronization state of financial time series using cross-recurrence plots.
problem Predicting the state of synchronization of financial time series.
method Cross-correlation analysis and deep learning framework for predicting synchronization state based on cross-recurrence plots.
result Satisfactory performance in predicting synchronization state for certain pairs of stocks.
Improved recurrent neural networks learn long-term dependencies through multi-scale memory.
problem Capturing long-term dependencies in recurrent neural networks.
method Incremental training of a modular RNN architecture with multi-scale hidden states.
result Incremental training and multi-scale memory enhance RNNs' ability to learn long-term dependencies.
Paper develops a new model for dynamic graph representation learning.
problem Learning over dynamic graphs with changing topology and node attributes.
method Hierarchical variational model with latent random variables and semi-implicit variational inference.
result SI-VGRNN and VGRNN outperform existing methods in dynamic link prediction.
Physics-guided models improve lake temperature and quality predictions.
problem Predicting and monitoring water temperature and quality in lakes.
method Combining physics-based models and recurrent neural networks with physical constraints.
result Improved prediction accuracy and scientific consistency.
Recurrence Plot (RP) and Recurrence Quantification Analysis RQA) are signal numerical analysis methodologies able to work with non linear dynamical systems and non stationarity. Moreover they well evidence changes in the states of a dynamical system. It is shown that RP and RQA detect the critical regime in financial i…
Model captures complex neural dynamics with Gaussian process and RNN.
problem Challenges in extracting latent dynamics from noisy neural data.
method Gaussian process recurrent neural networks (GP-RNN) for nonlinear, non-Markovian dynamics.
result Outperforms state-of-the-art methods in reconstructing latent dynamics.
Study compares LSTM, Transformer, and Mamba for bladder cancer recurrence analysis.
problem Complex time-dependent data in bladder cancer recurrence analysis.
method Evaluation of LSTM, Transformer, and Mamba models using Cox proportional hazards model.
result LSTM-Cox model outperforms Transformer-Cox and Mamba-Cox models in prediction accuracy.
Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward …
SRNNs learn dynamics of physical systems from data.
problem Learning dynamics of complex, noisy Hamiltonian systems.
method SRNNs model Hamiltonian functions with neural networks, using symplectic integration and optimization.
result SRNNs reliably learn dynamics of complex and noisy Hamiltonian systems.
New insights on stability in reservoir computing for better performance.
problem Stability in reservoir computing networks.
method Using the recurrent kernel limit for large reservoir sizes.
result Quantitative characterization of stability and chaos frontier.
Non-normal RNNs outperform orthogonal ones in sequential tasks.
problem Vanishing/exploding gradients in RNNs training.
method Investigate non-normal RNNs with non-normal recurrent connectivity matrix.
result Non-normal RNNs outperform orthogonal ones in various benchmarks.
EvoNet predicts the evolution of dynamic graphs using a graph neural network and recurrent architecture.
problem Predicting the evolution of dynamic graphs is challenging and underexplored.
method EvoNet uses a graph neural network and recurrent architecture to predict the evolution of dynamic graphs.
result EvoNet effectively predicts the evolution of dynamic graphs on both artificial and real-world datasets.
Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node …
DCRNN improves LSTM for chaotic dynamical system forecasting.
problem Modeling chaotic dynamical systems with recurrent neural networks.
method DCRNN incorporates learnable skip-connections and a Lyapunov stability regularization term.
result DCRNN outperforms LSTM in 100 out of 100 experiments, reducing mean squared error by 80.0%.
Proposes a neural network solution for queuing dynamics.
problem Modeling service system dynamics with limited parametric assumptions.
method Combines recurrent neural networks and generative adversarial networks.
result Evaluates solution on various datasets, demonstrating effectiveness.
Study the dynamics and topology of random hyperbolic manifolds.
problem Understanding the dynamics and topology of random hyperbolic manifolds.
method Analyzing Delaunay graphs and capacity over point processes.
result Established conditions for recurrence and transience in random hyperbolic manifolds.
Many natural systems, such as neurons firing in the brain or basketball teams traversing a court, give rise to time series data with complex, nonlinear dynamics. We can gain insight into these systems by decomposing the data into segments that are each explained by simpler dynamic units. Building on switching linear dy…
SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.
problem Nonstationary data challenges traditional models in continuous learning.
method SORSCNs autonomously adjust network parameters and structure in real-time using adaptive algorithms.
result SORSCNs outperform other models in generalizing to nonstationary data.
We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference…
Model learns collective and individual dynamics in time series data.
problem Lack of models capturing system-level collective behavior in individual time series.
method Hierarchical switching-state model with latent system-level and entity-level Markov chains.
result Model improves interpretability and forecasting accuracy compared to larger models.
A novel deep probabilistic model for dynamic systems forecasting.
problem Probabilistic forecasting in dynamic systems.
method Combining deep generative models and state space models with recurrent neural networks and variational sequence models.
result Outperforms existing models in system identification benchmarks and real-world centrifugal compressor forecasting.
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
Develop a variational framework for statistical inference on cyclic interactions.
problem Estimating and comparing large-scale recurrent organization in directed interactions.
method Represent directed interactions as edge flows on a simplicial complex and evolve under an energy-minimizing dynamical system.
result Separate transient interaction components from persistent harmonic flows, yielding a low-dimensional cycle space.