A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Generative diffusion models mimic biological memory networks, encoding associative dynamics in deep neural weights.
NeuZip compresses neural network weights to save memory during training and inference.
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …
To be effective in sequential data processing, Recurrent Neural Networks (RNNs) are required to keep track of past events by creating memories. While the relation between memories and the network's hidden state dynamics was established over the last decade, previous works in this direction were of a predominantly descr…
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
Serenity optimizes neural network execution for edge devices by scheduling with optimal memory footprint.
Deep neural networks with memory learn reduced equations from partial data.
We study the problem of learning associative memory -- a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by…
Study on memory effects in RNNs learning temporal data.
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked …
This paper introduces a hierarchical associative memory model with multiple layers.
Improved recurrent neural networks learn long-term dependencies through multi-scale memory.
A central challenge faced by memory systems is the robust retrieval of a stored pattern in the presence of interference due to other stored patterns and noise. A theoretically well-founded solution to robust retrieval is given by attractor dynamics, which iteratively clean up patterns during recall. However, incorporat…
New model captures long-term memory effects in epidemic dynamics.
New method combines long-memory reservoirs for accurate dengue forecasting from short data.
Enhances option pricing with fractional order Black-Scholes-Merton model.
New neural operators model turbulence with memory and randomness.
We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in t…
Neural ARFIMA model improves exchange rate forecasting for BRIC economies.
Researchers develop a neural network that learns like humans, overcoming forgetting and structure issues.
We study the inference of a model of dynamic networks in which both communities and links keep memory of previous network states. By considering maximum likelihood inference from single snapshot observations of the network, we show that link persistence makes the inference of communities harder, decreasing the detectab…
A new theory explains large associative memory with biological plausibility.
A new algorithm reduces memory usage for deep learning models.
Develops hyperparameter transfer methods for Dense Associative Memories.
When analyzing the genome, researchers have discovered that proteins bind to DNA based on certain patterns of the DNA sequence known as "motifs". However, it is difficult to manually construct motifs due to their complexity. Recently, externally learned memory models have proven to be effective methods for reasoning ov…
New cooperative dynamics enhances retrieval performance in neural networks.
V-HMN integrates memory mechanisms for improved image recognition.
Study on neural networks with non-normal interactions reveals unique spectral properties.
Computational Fluid Dynamics (CFD) is a hugely important subject with applications in almost every engineering field, however, fluid simulations are extremely computationally and memory demanding. Towards this end, we present Lat-Net, a method for compressing both the computation time and memory usage of Lattice Boltzm…
Paper uses deep reinforcement learning for network slicing and traffic prediction.
TGNs learn from dynamic graphs efficiently and outperform previous methods.
This work proposes a novel neural network architecture, called the Dynamically Controlled Recurrent Neural Network (DCRNN), specifically designed to model dynamical systems that are governed by ordinary differential equations (ODEs). The current state vectors of these types of dynamical systems only depend on their sta…
Comprehending complex systems by simplifying and highlighting important dynamical patterns requires modeling and mapping higher-order network flows. However, complex systems come in many forms and demand a range of representations, including memory and multilayer networks, which in turn call for versatile community-det…
The well-known Mori-Zwanzig theory tells us that model reduction leads to memory effect. For a long time, modeling the memory effect accurately and efficiently has been an important but nearly impossible task in developing a good reduced model. In this work, we explore a natural analogy between recurrent neural network…
Neural SVEs model complex systems with memory, outperforming traditional methods.
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. Fr…
ActNN reduces neural network training memory by 2-bit quantization.
SSINNs learn Hamiltonian systems from data with interpretable, low-memory models.
SDA method reduces memory and time for assimilating noisy geophysical data.
Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the sta…
LEM efficiently models long-term sequences with gradients.
Derivation of reduced order representations of dynamical systems requires the modeling of the truncated dynamics on the retained dynamics. In its most general form, this so-called closure model has to account for memory effects. In this work, we present a framework of operator inference to extract the governing dynamic…
This work investigates power laws in deep neural network ensembles and predicts their performance.
The Unlearning algorithm improves neural network performance in memory tasks.
Sparse codes improve optimal control tasks with correlated inputs.
RNN-HAR model improves VaR forecasting with long-memory and non-linear dynamics.
Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing cont…
Recent empirical results on long-term dependency tasks have shown that neural networks augmented with an external memory can learn the long-term dependency tasks more easily and achieve better generalization than vanilla recurrent neural networks (RNN). We suggest that memory augmented neural networks can reduce the ef…