A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A new memory replay mechanism improves reinforcement learning stability and speed.
We discuss memory models which are based on tensor decompositions using latent representations of entities and events. We show how episodic memory and semantic memory can be realized and discuss how new memory traces can be generated from sensory input: Existing memories are the basis for perception and new memories ar…
The brain optimizes memory by forgetting what's predictable, improving generalization.
Technological progress is leading to proliferation and diversification of trading venues, thus increasing the relevance of the long-standing question of market fragmentation versus consolidation. To address this issue quantitatively, we analyse systems of adaptive traders that choose where to trade based on their previ…
Proposes a CL technique to improve accuracy and reduce forgetting.
ANPyC combats forgetting by pruning and consolidating neural parameters.
Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current understanding of biological memory transfer to arrive at AI algorithms for memory consolidation and repl…
Analyzes self-attention in recurrent networks, proving it mitigates vanishing gradients.
TMNs model brain memory systems for continual learning.
Humans do not acquire perceptual abilities in the way we train machines. While machine learning algorithms typically operate on large collections of randomly-chosen, explicitly-labeled examples, human acquisition relies more heavily on multimodal unsupervised learning (as infants) and active learning (as children). Wit…
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
Active-memory mechanisms can replace self-attention in Transformers, but optimal results often require both.
SPARC improves continual learning with minimal memory and computational overhead.
Sequential learning of tasks using gradient descent leads to an unremitting decline in the accuracy of tasks for which training data is no longer available, termed catastrophic forgetting. Generative models have been explored as a means to approximate the distribution of old tasks and bypass storage of real data. Here …
PARMESAN learns from memory without parameters for fast, efficient continual learning.
ActNN reduces neural network training memory by 2-bit quantization.
Improved EXACT strategy reduces GNN memory consumption and runtime.
This research proposes a CL model for RNNs to handle sequential data without forgetting.
Study reduces memory needs for active learning with enriched queries.
Personalized predictive medicine necessitates the modeling of patient illness and care processes, which inherently have long-term temporal dependencies. Healthcare observations, recorded in electronic medical records, are episodic and irregular in time. We introduce DeepCare, an end-to-end deep dynamic neural network t…
Both the scientific community and the popular press have paid much attention to the speed of the Securities Information Processor, the data feed consolidating all trades and quotes across the US stock market. Rather than the speed of the Securities Information Processor, or SIP, we focus here on its accuracy. Relying o…
Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers. This presents a practical concern when seeking to deploy very deep architectures in production, especially when models need to be frequently re-trained on updated datasets. In…
Sleep-based regularization stabilizes STDP in recurrent neural networks.
Unified framework for adaptive learning systems using consolidation and expansion operations.
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of mo…
Using more than 6.7 billions of trades, we explore how the tick-by-tick dynamics of limit order books depends on the aggregate actions of large investment funds on a much larger (quarterly) timescale. In particular, we find that the well-established long memory of market order signs is markedly weaker when large invest…
We introduce a conceptually simple and scalable framework for continual learning domains where tasks are learned sequentially. Our method is constant in the number of parameters and is designed to preserve performance on previously encountered tasks while accelerating learning progress on subsequent problems. This is a…
We propose a model of fractal point process driven by the nonlinear stochastic differential equation. The model is adjusted to the empirical data of trading activity in financial markets. This reproduces the probability distribution function and power spectral density of trading activity observed in the stock markets. …
New proof shows neural networks can memorize training data with high accuracy.
A method to improve sequential learning by keeping past data errors in check.
EVCL combines VCL and EWC to prevent forgetting new tasks.
EWC helps prevent forgetting in neural networks by adjusting weights dynamically.
The paper explores tensor decompositions in deep learning models.
We propose a novel active learning framework for activity recognition using wearable sensors. Our work is unique in that it takes physical and cognitive limitations of the oracle into account when selecting sensor data to be annotated by the oracle. Our approach is inspired by human-beings' limited capacity to respond …
Compact DNNs increase memory footprint and reduce energy efficiency.
A model retains learned knowledge for longer by adding a plastic component to neural networks.
Intelligence emerges from stabilizing invariant cycles in memory.
We propose a new algorithm for training neural networks with binary activations and multi-level weights, which enables efficient processing-in-memory circuits with embedded nonvolatile memories (eNVM). Binary activations obviate costly DACs and ADCs. Multi-level weights leverage multi-level eNVM cells. Compared to exis…
We propose two optimization techniques to minimize memory usage and computation while meeting system timing constraints for real-time classification in wearable systems. Our method derives a hierarchical classifier structure for Support Vector Machine (SVM) in order to reduce the amount of computations, based on the pr…
A federated model learns shared archetypes from heterogeneous clients in continual learning.
In this paper, a novel architecture of Recurrent Neural Network (RNN) is designed and experimented. The proposed RNN adopts a computational memory based on the concept of stigmergy. The basic principle of a Stigmergic Memory (SM) is that the activity of deposit/removal of a quantity in the SM stimulates the next activi…
Algorithm improves model performance on shifted concepts without retraining.
Earlier we proposed the stochastic point process model, which reproduces a variety of self-affine time series exhibiting power spectral density S(f) scaling as power of the frequency f and derived a stochastic differential equation with the same long range memory properties. Here we present a stochastic differential eq…
Recurrent neural networks (RNNs) provide state-of-the-art performance in processing sequential data but are memory intensive to train, limiting the flexibility of RNN models which can be trained. Reversible RNNs---RNNs for which the hidden-to-hidden transition can be reversed---offer a path to reduce the memory require…
Collecting the large datasets needed to train deep neural networks can be very difficult, particularly for the many applications for which sharing and pooling data is complicated by practical, ethical, or legal concerns. However, it may be the case that derivative datasets or predictive models developed within individu…
Paper compresses ViTs models by 60% with minimal accuracy loss.
Sequential learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby previously learned knowledge is erased during learning of new, disjoint knowledge. Here, we propose a new approach to sequential learning which leverages the recent discovery of adversari…
New activation function BrownianReLU improves LSTM network performance on financial time series.