A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
ERNN improves RNN accuracy and stability with time-delayed self-feedback.
New theory shows predictive coding makes learning landscape easier to navigate.
Studies of wealth inequality often assume that an observed wealth distribution reflects a system in equilibrium. This constraint is rarely tested empirically. We introduce a simple model that allows equilibrium but does not assume it. To geometric Brownian motion (GBM) we add reallocation: all individuals contribute in…
MuonEq improves training of matrix-valued parameters by rebalancing momentum before orthogonalization.
Delayed-RNN approximates stacked and bidirectional RNNs.
Recurrent Neural Networks (RNNs) are powerful sequence modeling tools. However, when dealing with high dimensional inputs, the training of RNNs becomes computational expensive due to the large number of model parameters. This hinders RNNs from solving many important computer vision tasks, such as Action Recognition in …
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
In this paper, we explore different ways to extend a recurrent neural network (RNN) to a \textit{deep} RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points …
We provide further evidence that markets trend on the medium term (months) and mean-revert on the long term (several years). Our results bolster Black's intuition that prices tend to be off roughly by a factor of 2, and take years to equilibrate. The story behind these results fits well with the existence of two types …
This paper studies a theoretical pruning method for RNNs to reduce computational costs.
Lyapunov analysis improves RNN performance prediction.
The notion of the stationary equilibrium ensemble has played a central role in statistical mechanics. In machine learning as well, training serves as generalized equilibration that drives the probability distribution of model parameters toward stationarity. Here, we derive stationary fluctuation-dissipation relations t…
In this work, we propose a novel recurrent neural network (RNN) architecture. The proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking multiple recurrent layers by allowing and controlling signals flowing from upper recurrent layers to lower layers using a global gating unit for each pai…
Proposes Fusion Recurrent Neural Network for sequence data.
This paper proposes an alternative to the classical price-adjustment mechanism (called "tâtonnement" after Walras) that is second-order in time. The proposed mechanism, an analogue to the damped harmonic oscillator, provides a dynamic equilibration process that depends only on local information. We show how such a proc…
Recurrent neural networks (RNNs) are powerful and effective for processing sequential data. However, RNNs are usually considered "black box" models whose internal structure and learned parameters are not interpretable. In this paper, we propose an interpretable RNN based on the sequential iterative soft-thresholding al…
Novel deep learning method predicts reaction coordinates and future MD trajectories.
Efficient RNN algorithm guarantees convergence in online learning.
While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the case of RNNs with finite precision whose computation time is linear in the input length. Under these limitations, we show that different RNN va…
Paper compresses RNNs for resource-constrained devices.
RNNs struggle with in-context retrieval, while Transformers excel.
GPU-optimized ES-RNN boosts time series forecasting speed by 322x.
Paper refines RNN training by analyzing smoothness and attractors.
Novel method improves training RNNs by accelerating gradient descent.
Analyzes RNNs using ODEs to map their properties and improve stability.
AC-RNN improves RNN for sequence labeling tasks.
Elman-type RNNs converge to globally optimal solutions in the mean-field regime.
Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational res…
Recurrent Neural Networks (RNNs) have long been recognized for their potential to model complex time series. However, it remains to be determined what optimization techniques and recurrent architectures can be used to best realize this potential. The experiments presented take a deep look into Hessian free optimization…
New kernels boost RNN performance on non-time-series data.
Lyapunov exponents help understand RNN stability.
Study proves deep narrow RNNs can approximate any function, with minimum width independent of data length.
Hamiltonian RNN controls hidden states gradient for long-term dependencies.
We introduce MinimalRNN, a new recurrent neural network architecture that achieves comparable performance as the popular gated RNNs with a simplified structure. It employs minimal updates within RNN, which not only leads to efficient learning and testing but more importantly better interpretability and trainability. We…
Frequentist method estimates uncertainty in RNNs without altering architecture.
This paper proposes structurally sparse RNNs to reduce computational and memory costs.
A new model combines spatial and spectral features for HSI classification.
New model handles uneven time intervals better than traditional methods.
Multivariate time-series modeling and forecasting is an important problem with numerous applications. Traditional approaches such as VAR (vector auto-regressive) models and more recent approaches such as RNNs (recurrent neural networks) are indispensable tools in modeling time-series data. In many multivariate time ser…
RNNs struggle with chaotic dynamics due to exploding gradients, but we found a way to optimize training.
RNNs are suboptimal at compressing past sensory inputs for future prediction.
RNNs learn combinatorial graph problems with sample complexity bounds.
Recurrent Neural Networks (RNNs), which are a powerful scheme for modeling temporal and sequential data need to capture long-term dependencies on datasets and represent them in hidden layers with a powerful model to capture more information from inputs. For modeling long-term dependencies in a dataset, the gating mecha…
Study shows challenges in converting RNNs to FSMs due to computational complexity.
Recurrent neural networks (RNNs) are powerful models of sequential data. They have been successfully used in domains such as text and speech. However, RNNs are susceptible to overfitting; regularization is important. In this paper we develop Noisin, a new method for regularizing RNNs. Noisin injects random noise into t…
Mathematical methods characterize RNNs' asymptotics as hidden units and data grow.
Paper compresses RNNs using HT decomposition for better performance.
The recent adoption of recurrent neural networks (RNNs) for session modeling has yielded substantial performance gains compared to previous approaches. In terms of context-aware session modeling, however, the existing RNN-based models are limited in that they are not designed to explicitly model rich static user-side c…