A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper studies a theoretical pruning method for RNNs to reduce computational costs.
Convolutional neural networks (CNNs) attained a good performance in hyperspectral sensing image (HSI) classification, but CNNs consider spectra as orderless vectors. Therefore, considering the spectra as sequences, recurrent neural networks (RNNs) have been applied in HSI classification, for RNNs is skilled at dealing …
Lyapunov analysis improves RNN performance prediction.
Study on RNNs' generalization bounds and variants.
In this paper, we unravel a fundamental connection between weighted finite automata~(WFAs) and second-order recurrent neural networks~(2-RNNs): in the case of sequences of discrete symbols, WFAs and 2-RNNs with linear activation functions are expressively equivalent. Motivated by this result, we build upon a recent ext…
Combines Fourier methods and RNNs for efficient time series prediction.
SKOLR uses linear RNNs to approximate Koopman operators for time-series forecasting.
The paper distills a weighted automaton from RNNs for language modeling.
Algorithm extracts deterministic PDFA from probabilistic models with improved performance.
We consider the problem of training input-output recurrent neural networks (RNN) for sequence labeling tasks. We propose a novel spectral approach for learning the network parameters. It is based on decomposition of the cross-moment tensor between the output and a non-linear transformation of the input, based on score …
In this paper, we propose a fully supervised speaker diarization approach, named unbounded interleaved-state recurrent neural networks (UIS-RNN). Given extracted speaker-discriminative embeddings (a.k.a. d-vectors) from input utterances, each individual speaker is modeled by a parameter-sharing RNN, while the RNN state…
Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the …
Paper establishes generalization bounds for RNNs and improves existing results.
Proposes a new RNN structure to improve expressivity without sacrificing stability.
Vanishing and exploding gradients are two of the main obstacles in training deep neural networks, especially in capturing long range dependencies in recurrent neural networks~(RNNs). In this paper, we present an efficient parametrization of the transition matrix of an RNN that allows us to stabilize the gradients that …
This work improves LSTM and GRU training stability and generalization.
Delayed-RNN approximates stacked and bidirectional RNNs.
Recurrent Neural Networks (RNNs) are powerful sequence modeling tools. However, when dealing with high dimensional inputs, the training of RNNs becomes computational expensive due to the large number of model parameters. This hinders RNNs from solving many important computer vision tasks, such as Action Recognition in …
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points …
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Proposes Fusion Recurrent Neural Network for sequence data.
Recurrent neural networks (RNNs) are powerful and effective for processing sequential data. However, RNNs are usually considered "black box" models whose internal structure and learned parameters are not interpretable. In this paper, we propose an interpretable RNN based on the sequential iterative soft-thresholding al…
Efficient RNN algorithm guarantees convergence in online learning.
While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the case of RNNs with finite precision whose computation time is linear in the input length. Under these limitations, we show that different RNN va…
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
Paper compresses RNNs for resource-constrained devices.
RNNs struggle with in-context retrieval, while Transformers excel.
GPU-optimized ES-RNN boosts time series forecasting speed by 322x.
Paper refines RNN training by analyzing smoothness and attractors.
Novel method improves training RNNs by accelerating gradient descent.
Analyzes RNNs using ODEs to map their properties and improve stability.
Elman-type RNNs converge to globally optimal solutions in the mean-field regime.
Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational res…
Recurrent Neural Networks (RNNs) have long been recognized for their potential to model complex time series. However, it remains to be determined what optimization techniques and recurrent architectures can be used to best realize this potential. The experiments presented take a deep look into Hessian free optimization…
New kernels boost RNN performance on non-time-series data.
Lyapunov exponents help understand RNN stability.
Study proves deep narrow RNNs can approximate any function, with minimum width independent of data length.
Hamiltonian RNN controls hidden states gradient for long-term dependencies.
We introduce MinimalRNN, a new recurrent neural network architecture that achieves comparable performance as the popular gated RNNs with a simplified structure. It employs minimal updates within RNN, which not only leads to efficient learning and testing but more importantly better interpretability and trainability. We…
Frequentist method estimates uncertainty in RNNs without altering architecture.
This paper proposes structurally sparse RNNs to reduce computational and memory costs.
New model handles uneven time intervals better than traditional methods.
Multivariate time-series modeling and forecasting is an important problem with numerous applications. Traditional approaches such as VAR (vector auto-regressive) models and more recent approaches such as RNNs (recurrent neural networks) are indispensable tools in modeling time-series data. In many multivariate time ser…
RNNs struggle with chaotic dynamics due to exploding gradients, but we found a way to optimize training.
RNNs are suboptimal at compressing past sensory inputs for future prediction.
RNNs learn combinatorial graph problems with sample complexity bounds.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…