A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new recurrent unit alleviates vanishing gradients for long-term dependencies.
New theory explains GAN's high quality but low diversity.
Bayesian units improve speech recognition with minimal parameters.
Lipschitz RNNs improve stability and performance in various tasks.
Improved GRU model with weighted time-delay feedback for long-term dependencies.
Recurrent Neural Networks (RNNs) with sophisticated units that implement a gating mechanism have emerged as powerful technique for modeling sequential signals such as speech or electroencephalography (EEG). The latter is the focus on this paper. A significant big data resource, known as the TUH EEG Corpus (TUEEG), has …
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Recurrent neural networks with various types of hidden units have been used to solve a diverse range of problems involving sequence data. Two of the most recent proposals, gated recurrent units (GRU) and minimal gated units (MGU), have shown comparable promising results on example public datasets. In this paper, we int…
The paper evaluates three variants of the Gated Recurrent Unit (GRU) in recurrent neural networks (RNN) by reducing parameters in the update and reset gates. We evaluate the three variant GRU models on MNIST and IMDB datasets and show that these GRU-RNN variant models perform as well as the original GRU RNN model while…
New eGRU unit improves keyword spotting on ultra-low-power devices.
Paper presents RGNN for better graph node representation learning.
Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SR…
ENRNN uses eigenvalue normalization for short-term memory in RNNs.
CRUs model irregular time series with continuous hidden states.
This paper improves speech synthesis using a DGP with SRU for naturalness.
Bayesian approach improves neural network recurrence.
Study shows GRU model with dropout outperforms in Bitcoin price prediction.
Analyzes convex structures in Teichmüller space unit tangent spheres.
Recurrent Neural Network (RNN) has been successfully applied in many sequence learning problems. Such as handwriting recognition, image description, natural language processing and video motion analysis. After years of development, researchers have improved the internal structure of the RNN and introduced many variants…
The Rectified Linear Unit (ReLU) is a foundational activation function in artficial neural networks. Recent literature frequently misattributes its origin to the 2018 (initial) version of this paper, which exclusively investigated ReLU at the classification layer. This paper formally corrects the citation record by tra…
Recurrent neural networks (RNNs) have been used extensively and with increasing success to model various types of sequential data. Much of this progress has been achieved through devising recurrent units and architectures with the flexibility to capture complex statistics in the data, such as long range dependency or l…
It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, th…
Cortical circuits exhibit intricate recurrent architectures that are remarkably similar across different brain areas. Such stereotyped structure suggests the existence of common computational principles. However, such principles have remained largely elusive. Inspired by gated-memory networks, namely long short-term me…
Generative adversarial networks (GANs) can be interpreted as an adversarial game between two players, a discriminator D and a generator G, in which D learns to classify real from fake data and G learns to generate realistic data by "fooling" D into thinking that fake data is actually real data. Currently, a dominating …
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
We prove quantitative recurrence and large deviations results for the Teichmuller geodesci flow on a connected component of a stratum of the moduli space of holomorphic unit-area quadratic differentials on a compact genus surface.
In this paper, we introduce the notion of liquid time-constant (LTC) recurrent neural networks (RNN)s, a subclass of continuous-time RNNs, with varying neuronal time-constant realized by their nonlinear synaptic transmission model. This feature is inspired by the communication principles in the nervous system of small …
Random feature maps improve forecasting of chaotic dynamical systems.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Neural network predicts falls in elderly people up to 10 minutes in advance.
Visual Question Answering (VQA) requires AI models to comprehend data in two domains, vision and text. Current state-of-the-art models use learned attention mechanisms to extract relevant information from the input domains to answer a certain question. Thus, robust attention mechanisms are essential for powerful VQA mo…
NS-GAN mode collapse due to sample weighting inversion, solved with MM-nsat.
Many natural systems, such as neurons firing in the brain or basketball teams traversing a court, give rise to time series data with complex, nonlinear dynamics. We can gain insight into these systems by decomposing the data into segments that are each explained by simpler dynamic units. Building on switching linear dy…
Efficient deep GNNs achieve state-of-the-art performance without training.
ARU adapts deep forecasting models in streaming data with efficient updates.
In this paper we propose and investigate a novel nonlinear unit, called unit, for deep neural networks. The proposed unit receives signals from several projections of a subset of units in the layer below and computes a normalized norm. We notice two interesting interpretations of the unit. First…
Neural network predicts cardiovascular events from EHRs with high accuracy.
QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
Theoretical analysis shows RNNs with various nonlinearities benefit from depth efficiency.
Successful recurrent models such as long short-term memories (LSTMs) and gated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these models have been found to improve the learning of medium to long term temporal dependencies and to help with vanishing gradient issues. We prove that learnable gates in a…
In this work, we present the Grounded Recurrent Neural Network (GRNN), a recurrent neural network architecture for multi-label prediction which explicitly ties labels to specific dimensions of the recurrent hidden state (we call this process "grounding"). The approach is particularly well-suited for extracting large nu…
Time-aware neural models improve system identification from unevenly sampled data.
Bedside monitors in Intensive Care Units (ICUs) frequently sound incorrectly, slowing response times and desensitising nurses to alarms (Chambrin, 2001), causing true alarms to be missed (Hug et al., 2011). We compare sliding window predictors with recurrent predictors to classify patient state-of-health from ICU multi…
Paper presents FPGA implementation for efficient recurrent neural networks.
New method infers nonlinear Granger causality from time series data.
We show that for any weakly convergent sequence of ergodic -invariant probability measures on a stratum of unit-area translation surfaces, the corresponding Siegel-Veech constants converge to the Siegel-Veech constant of the limit measure. Together with a measure equidistribution result due to Eskin-M…
A deep learning model improves pedestrian tracking accuracy.
The standard LSTM recurrent neural networks while very powerful in long-range dependency sequence applications have highly complex structure and relatively large (adaptive) parameters. In this work, we present empirical comparison between the standard LSTM recurrent neural network architecture and three new parameter-r…