Proposes Fusion Recurrent Neural Network for sequence data.
problem Improving sequence learning for practical applications.
method Fusion module and Transport module for sequence data.
result Fusion RNN performs comparably to state-of-the-art RNNs.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “recurrent modules”
AGCRN forecasts traffic using adaptive graph and recurrent learning.
problem Forecasting traffic dynamics with complex spatial and temporal correlations.
method Adaptive Graph Convolutional Recurrent Network (AGCRN) with Node Adaptive Parameter Learning (NAPL) and Data Adaptive Graph Generation (DAGG).
result AGCRN outperforms state-of-the-art models without pre-defined graphs.
Improved recurrent neural networks learn long-term dependencies through multi-scale memory.
problem Capturing long-term dependencies in recurrent neural networks.
method Incremental training of a modular RNN architecture with multi-scale hidden states.
result Incremental training and multi-scale memory enhance RNNs' ability to learn long-term dependencies.
Two solutions for multi-modal record linkage using Deep Learning inspired by Visual Question Answering.
problem Matching records from multiple sources representing the same entity.
method Two fusion modules: Recurrent Neural Network + Convolutional Neural Network and Stacked Attention Network. A Siamese Neural Network computes similarity.
result Recurrent Neural Network + Convolutional Neural Network fusion module outperforms a simple model.
Dynamic and Context-Dependent Stock Price Prediction Using Attention Modules and News Sentimentq-fin.CP
Improved stock price prediction using attention modules and news sentiment.
problem Predicting stock prices with non-stationary and non-parametric data.
method α_{t}-RIM architecture with attention modules and exponentially smoothed recurrent neural network.
result The -RIM outperforms state-of-the-art models in predicting unseen data.
Traditional sequential multi-object attention models rely on a recurrent mechanism to infer object relations. We propose a relational extension (R-SQAIR) of one such attention model (SQAIR) by endowing it with a module with strong relational inductive bias that computes in parallel pairwise interactions between inferre…
Learning representations that accurately capture long-range dependencies in sequential inputs -- including text, audio, and genomic data -- is a key problem in deep learning. Feed-forward convolutional models capture only feature interactions within finite receptive fields while recurrent architectures can be slow and …
Quantum theory constructs a group and skein module for knot complements.
problem Understanding the fundamental group of knot complements using quantum methods.
method Using bottom tangles, the universal space of quantum representations is constructed, then factored by the skein relation to get the skein module.
result Derives recurrence relation for the colored Jones polynomial, known as polynomial.
We propose an autoencoding sequence-based transceiver for communication over dispersive channels with intensity modulation and direct detection (IM/DD), designed as a bidirectional deep recurrent neural network (BRNN). The receiver uses a sliding window technique to allow for efficient data stream estimation. We find t…
Recurrent Networks are one of the most powerful and promising artificial neural network algorithms to processing the sequential data such as natural languages, sound, time series data. Unlike traditional feed-forward network, Recurrent Network has a inherent feed back loop that allows to store the temporal context info…
Improved speech enhancement with MNTFA using time-frequency attention.
problem Speech enhancement with limited model size and memory.
method Designing MNTFA with self-attention modules for long sequences and joint training.
result MNTFA achieves better performance with fewer parameters than DPCRN.
Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Miningcs.CV
This paper improves image super-resolution by integrating cross-scale non-local attention.
problem Improving image super-resolution by leveraging long-range and cross-scale feature correlations.
method Proposes a Cross-Scale Non-Local (CS-NL) attention module integrated into a recurrent neural network.
result Significantly improved performance on SISR benchmarks.
Paper presents a novel neural network for MIMO symbol detection.
problem Handling a variable number of users in MIMO systems.
method Recurrent and permutation equivariant neural network architecture with iterative decoding.
result The neural detector outperforms existing methods in accuracy and efficiency.
The paper improves DFA for CNN and RNN training to match BP accuracy.
problem Low accuracy in CNN and RNN training using DFA.
method Divide network into modules, apply DFA within, use sparse backward weight, and incorporate dilated convolution and sparse matrix multiplication.
result Achieves BP-level accuracy in CNN and RNN training.
PSTN improves traffic condition forecasting with deep neural networks.
problem Challenges in accurately forecasting traffic conditions due to complex spatiotemporal correlations.
method Proposes PSTN with three modules: graph convolutional network, temporal convolutional network, and gated recurrent unit framework.
result Significantly outperforms state-of-the-art benchmarks in short-term traffic conditions forecasting.
Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study the memorization subtask from the point of view of the design and training of recurrent neural networks. We propose a new model, the Linear Me…
Dynamic models learn from sparse, interacting sub-systems.
problem Learning robust models for systems with local views and spatial locations.
method Abstracting the system as a collection of sparsely interacting sub-systems, each with a learned topology informed by spatial structure.
result Models are more robust to the number of available views and generalize better to novel tasks.
Automated trading system with preprocessing and reinforcement learning.
problem Efficient portfolio trading for individual investors.
method Feature preprocessing, recurrent reinforcement learning, automated trading algorithm.
result System outperforms other strategies in profit and drawdown.
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random …
A new deep learning framework captures multi-scale spatio-temporal dependencies.
problem Designing and analyzing deep learning models for complex spatio-temporal analytics.
method Developed an IDRNN model with three modules for integrating and learning multi-scale spatio-temporal data.
result The IDRNN model outperforms classical and state-of-the-art models in capturing meaningful multi-scale spatio-temporal dependencies.
Central Pattern Generators (CPGs) are biological neural circuits capable of producing coordinated rhythmic outputs in the absence of rhythmic input. As a result, they are responsible for most rhythmic motion in living organisms. This rhythmic control is broadly applicable to fields such as locomotive robotics and medic…
We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, …
We propose a new architecture called Memory-Augmented Encoder-Solver (MAES) that enables transfer learning to solve complex working memory tasks adapted from cognitive psychology. It uses dual recurrent neural network controllers, inside the encoder and solver, respectively, that interface with a shared memory module a…
Knowledge graph reasoning is a critical task in natural language processing. The task becomes more challenging on temporal knowledge graphs, where each fact is associated with a timestamp. Most existing methods focus on reasoning at past timestamps and they are not able to predict facts happening in the future. This pa…
CoRMF uses RNNs to solve Ising models efficiently by ordering critical edges.
problem Solving Ising models efficiently and accurately.
method Criticality-ordered spin sequence and RNNs for mean-field factorization.
result Proves tighter error bounds than naive mean-field.
Many events occur in the world. Some event types are stochastically excited or inhibited---in the sense of having their probabilities elevated or decreased---by patterns in the sequence of previous events. Discovering such patterns can help us predict which type of event will happen next and when. We model streams of d…
UniPhyNet improves cognitive load classification accuracy using EEG, ECG, and EDA signals.
problem Classifying cognitive load using multimodal physiological data.
method Unified network architecture integrating multiscale parallel convolutional blocks, ResNet-type blocks, and channel block attention module. Uses bidirectional gated recurrent unit for temporal dependencies.
result Improves raw signal classification accuracy from 70% to 80% (binary) and 62% to 74% (ternary) on CL-Drive dataset.
Study long-term behavior of semi-Markov modulated processes using integral functions.
problem Analyzing long-term behavior of semi-Markov modulated processes involving integral functions.
method Using ergodic semi-Markovian environment and affine stochastic recurrence equation.
result Mixture type laws emerge in long-term limit for processes.
PAQ8 is an open source lossless data compression algorithm that currently achieves the best compression rates on many benchmarks. This report presents a detailed description of PAQ8 from a statistical machine learning perspective. It shows that it is possible to understand some of the modules of PAQ8 and use this under…
An artificial agent for financial risk and returns' prediction is built with a modular cognitive system comprised of interconnected recurrent neural networks, such that the agent learns to predict the financial returns, and learns to predict the squared deviation around these predicted returns. These two expectations a…
Recurrent convolution (RC) shares the same convolutional kernels and unrolls them multiple steps, which is originally proposed to model time-space signals. We argue that RC can be viewed as a model compression strategy for deep convolutional neural networks. RC reduces the redundancy across layers. However, the perform…
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. Fr…
The ability to predict city-wide parking availability is crucial for the successful development of Parking Guidance and Information (PGI) systems. Indeed, the effective prediction of city-wide parking availability can improve parking efficiency, help urban planning, and ultimately alleviate city congestion. However, it…
Recent dialogue approaches operate by reading each word in a conversation history, and aggregating accrued dialogue information into a single state. This fixed-size vector is not expandable and must maintain a consistent format over time. Other recent approaches exploit an attention mechanism to extract useful informat…
The principles of adaptation in organisms and machines II: Thermodynamics of the Bayesian brainq-bio.NC
The article explores how organisms and machines learn and recognize the world using Bayesian inference and thermodynamics.
problem Understanding how organisms and machines learn and recognize the world.
method Introducing a thermodynamic view of the Bayesian brain hypothesis, using a simple generative model of spiking neural populations.
result The process of Bayesian inference can be quantified using entropy, revealing the perceptual capacity of neural activity.
FHRN uses continuous-time dynamics to stabilize reentrant neural computation.
problem Stabilizing reentrant neural computation.
method Formulated as a continuous-time neural-ODE system, revealing norm-regulated reentry.
result Achieves stable oscillatory trajectories through population-level gain modulation.
The Linear Attention Recurrent Neural Network (LARNN) is a recurrent attention module derived from the Long Short-Term Memory (LSTM) cell and ideas from the consciousness Recurrent Neural Network (RNN). Yes, it LARNNs. The LARNN uses attention on its past cell state values for a limited window size . The formulas ar…
News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, fast growing number of items, accelerated item's value decay, and…
In many online applications interactions between a user and a web-service are organized in a sequential way, e.g., user browsing an e-commerce website. In this setting, recommendation system acts throughout user navigation by showing items. Previous works have addressed this recommendation setup through the task of pre…
CARRNN tackles deep learning for sporadic data, improving prediction errors in healthcare.
problem Challenges in learning temporal patterns from sporadic multivariate longitudinal data.
method Developed a novel deep learning architecture combining RNN and CAR models, using a generalized discrete-time autoregressive model.
result CARRNN achieves the lowest prediction errors in multivariate time-series regression tasks.
New neural networks with variable time constants for better time-series prediction.
problem Improving neural network performance in time-series prediction.
method Constructing networks of linear dynamical systems modulated by nonlinear gates, using numerical differential equation solvers.
result Liquid Time-Constant Networks (LTCs) yield superior performance on time-series prediction tasks.
In this paper, we present a novel approach for incorporating external knowledge in Recurrent Neural Networks (RNNs). We propose the integration of lexicon features into the self-attention mechanism of RNN-based architectures. This form of conditioning on the attention distribution, enforces the contribution of the most…
Traffic flow forecasting is hot spot research of intelligent traffic system construction. The existing traffic flow prediction methods have problems such as poor stability, high data requirements, or poor adaptability. In this paper, we define the traffic data time singularity ratio in the dropout module and propose a …
Cubic spline smoothing improves interpolation between irregularly sampled data.
problem Interpolation discontinuity in recurrent neural networks for irregularly sampled sequences.
method Cubic spline smoothing compensation module trained end-to-end with ODE-RNN.
result Improves interpolation between irregularly sampled data points.
Short-term road traffic prediction (STTP) is one of the most important modules in Intelligent Transportation Systems (ITS). However, network-level STTP still remains challenging due to the difficulties both in modeling the diverse traffic patterns and tacking high-dimensional time series with low latency. Therefore, a …
The pullback approach to global Finsler geometry is adopted. Three classes of recurrence in Finsler geometry are introduced and investigated: simple recurrence, Ricci recurrence and concircular recurrence. Each of these classes consists of four types of recurrence. The interrelationships between the different types of …
Study on biharmonic hypersurfaces with specific recurrent operators in Euclidean space.
problem Characterizing biharmonic hypersurfaces with recurrent operators.
method Analysis of various recurrent operators and their impact on biharmonic hypersurfaces.
result Some well-known recurrent operators play a significant role in making biharmonic hypersurfaces minimal.
Attention mechanism combines bottom-up and top-down signals in neural networks.
problem Combining robust perception with bottom-up and top-down signals.
method Attention mechanism over modulated recurrent neural networks.
result Bidirectional information flow leads to improved performance in various tasks.