A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Proposes a novel SAM operator for separate item and relational memories.
Model combines long-term and short-term memory using conceptors.
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
New memory allocation scheme improves image generation performance.
Enhanced Hopfield model boosts memory retrieval capacity.
Generative diffusion models mimic biological memory networks, encoding associative dynamics in deep neural weights.
We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva's sparse distributed memory, it has a robust distributed reading and writing mechanism. The memory is analytically tractable, which enables optimal on-line compression via a Bayesian updat…
New memory models improve LSSVM's generalization and reduce time cost.
Memory is increasingly often the bottleneck when training neural network models. Despite this, techniques to lower the overall memory requirements of training have been less widely studied compared to the extensive literature on reducing the memory requirements of inference. In this paper we study a fundamental questio…
A new model for sequential memory using temporal predictive coding.
We discuss memory models which are based on tensor decompositions using latent representations of entities and events. We show how episodic memory and semantic memory can be realized and discuss how new memory traces can be generated from sensory input: Existing memories are the basis for perception and new memories ar…
Paper questions RNN and LSTM's long-term memory and introduces a new definition.
New analysis tightens memory capacity of Hopfield models using spherical codes.
Current generation of memory-augmented neural networks has limited scalability as they cannot efficiently process data that are too large to fit in the external memory storage. One example of this is lifelong learning scenario where the model receives unlimited length of data stream as an input which contains vast majo…
Augmenting a neural network with memory that can grow without growing the number of trained parameters is a recent powerful concept with many exciting applications. We propose a design of memory augmented neural networks (MANNs) called Labeled Memory Networks (LMNs) suited for tasks requiring online adaptation in class…
Paper proposes low-rank gradient approximation to save memory for deep neural network training.
It is generally accepted that many time series of practical interest exhibit strong dependence, i.e., long memory. For such series, the sample autocorrelations decay slowly and log-log periodogram plots indicate a straight-line relationship. This necessitates a class of models for describing such behavior. A popular cl…
DAM with MRL improves relational reasoning in MANNs.
DMs emerge from DenseAMs, transitioning from memorization to generalization.
Typical neural networks with external memory do not effectively separate capacity for episodic and working memory as is required for reasoning in humans. Applying knowledge gained from psychological studies, we designed a new model called Differentiable Working Memory (DWM) in order to specifically emulate human workin…
Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern…
Deep learning typically requires training a very capable architecture using large datasets. However, many important learning problems demand an ability to draw valid inferences from small size datasets, and such problems pose a particular challenge for deep learning. In this regard, various researches on "meta-learning…
Tensor models decode human perception and memory using SPO triples.
PipeDream-2BW accelerates large model training by 20x with minimal memory usage.
State-of-the-art models are now trained with billions of parameters, reaching hardware limits in terms of memory consumption. This has created a recent demand for memory-efficient optimizers. To this end, we investigate the limits and performance tradeoffs of memory-efficient adaptively preconditioned gradient methods.…
We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in t…
This study examines memory effects in S&P500 market correlations using Langevin models.
We investigate the robustness properties of image recognition models equipped with two features inspired by human vision, an explicit episodic memory and a shape bias, at the ImageNet scale. As reported in previous work, we show that an explicit episodic memory improves the robustness of image recognition models agains…
Unified model explains volatility memory in stocks and forex.
We study the long-term memory in diverse stock market indices and foreign exchange rates using the Detrended Fluctuation Analysis(DFA). For all daily and high-frequency market data studied, no significant long-term memory property is detected in the return series, while a strong long-term memory property is found in th…
GPU (graphics processing unit) has been used for many data-intensive applications. Among them, deep learning systems are one of the most important consumer systems for GPU nowadays. As deep learning applications impose deeper and larger models in order to achieve higher accuracy, memory management becomes an important …
In recent years, we have witnessed an increased interest in temporal modeling of patient records from large scale Electronic Health Records (EHR). While simpler RNN models have been used for such problems, memory networks, which in other domains were found to generalize well, are underutilized. Traditional memory netwo…
The key to a Transformer model is the self-attention mechanism, which allows the model to analyze an entire sequence in a computationally efficient manner. Recent work has suggested the possibility that general attention mechanisms used by RNNs could be replaced by active-memory mechanisms. In this work, we evaluate wh…
Transformers can perform well with less long-range memory.
We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability. To tackle this problem, we propose a novel end-to…
Research on long-range memory in financial and social systems using various models.
Accurate volatility modelling is paramount for optimal risk management practices. One stylized feature of financial volatility that impacts the modelling process is long memory explored in this paper for alternative risk measures, observed absolute and squared returns for high frequency intraday UK futures. Volatility …
We augment recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning. We conceptualize this memory as a rapidly adaptable function that we parameterize as a deep neural network. Reading from the neural memory function amounts to pushing an input (the key vector) throug…
SliceOut speeds up deep learning training without sacrificing accuracy.
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that…
New model captures long-term memory effects in epidemic dynamics.
The paper studies scaling laws for associative memory mechanisms.
Global Memory Augmentation (GMAT) improves Transformer performance on long documents.
The Mike-Farmer (MF) model was constructed empirically based on the continuous double auction mechanism in an order-driven market, which can successfully reproduce the cubic law of returns and the diffusive behavior of stock prices at the transaction level. However, the volatility (defined by absolute return) in the MF…
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the sam…
Extended LSTM improves language modeling performance.
NEST optimizes deep learning training by placing devices efficiently across networks and memory.
Enhances sequence memory capacity in neural networks.