A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New approach reduces malware detection memory requirements and speeds up training.
Service-induced congestion in memory-constrained LLM serving
Global Memory Augmentation (GMAT) improves Transformer performance on long documents.
New algorithm reduces costs and latency for large language model inference.
Algorithm learns linear systems from partial observations with near-optimal rate.
Adaptive beamforming collapses in highly non-stationary environments, but the Universal Switching Beamformer resolves this by dynamically adjusting memory length.
We propose a prediction model based on the minority game in which traders continuously evaluate a complete set of trading strategies with different memory lengths using the strategies' past performance. Based on the chosen trading strategy they determine their prediction of the movement for the following time period of…
To understand the fundamental trade-offs between training stability, temporal dynamics and architectural complexity of recurrent neural networks~(RNNs), we directly analyze RNN architectures using numerical methods of ordinary differential equations~(ODEs). We define a general family of RNNs--the ODERNNs--by relating t…
In mix-game which is an extension of minority game, there are two groups of agents; group1 plays the majority game, but the group2 plays the minority game. This paper studies the change of the average winnings of agents and volatilities vs. the change of mixture of agents in mix-game model. It finds that the correlatio…
New methods improve neural network extrapolation to long sequences.
Compact Recurrent Transformer (CRT) improves Transformer efficiency for long sequences.
Current generation of memory-augmented neural networks has limited scalability as they cannot efficiently process data that are too large to fit in the external memory storage. One example of this is lifelong learning scenario where the model receives unlimited length of data stream as an input which contains vast majo…
AI beats 95% of humans in Rock-Paper-Scissors.
Deep neural networks with memory learn reduced equations from partial data.
Enhances sequence memory capacity in neural networks.
This paper studies the correlations of the average winnings of agents and the volatilities of systems based on mix-game model which is an extension of minority game (MG). In mix-game, there are two groups of agents; group1 plays the majority game, but the group2 plays the minority game. The results show that the correl…
A new algorithm reduces memory usage for long token attention in streaming applications.
Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length. In this paper we introduce sparse factorizations of the attention matrix which reduce this to . We also introduce a) a variation on architecture and initialization to train deeper net…
One of the core tasks in multi-view learning is to capture relations among views. For sequential data, the relations not only span across views, but also extend throughout the view length to form long-term intra-view and inter-view interactions. In this paper, we present a new memory augmented neural network model that…
This paper explores using a Long short-term memory (LSTM) based sequence autoencoder to learn interesting features for detecting surveillance aircraft using ADS-B flight data. An aircraft periodically broadcasts ADS-B (Automatic Dependent Surveillance - Broadcast) data to ground receivers. The ability of LSTM networks …
We analyze computational limits of modern Hopfield models based on pattern norms.
Express improves causal attention guarantees for language models.
An associative memory is a framework of content-addressable memory that stores a collection of message vectors (or a dataset) over a neural network while enabling a neurally feasible mechanism to recover any message in the dataset from its noisy version. Designing an associative memory requires addressing two main task…
Music relies heavily on repetition to build structure and meaning. Self-reference occurs on multiple timescales, from motifs to phrases to reusing of entire sections of music, such as in pieces with ABA structure. The Transformer (Vaswani et al., 2017), a sequence model based on self-attention, has achieved compelling …
Paper proposes new method for time series confidence intervals using LSTM.
We study finite sample properties of estimators of power-law cross-correlations -- detrended cross-correlation analysis (DCCA), height cross-correlation analysis (HXA) and detrending moving-average cross-correlation analysis (DMCA) -- with a special focus on short-term memory bias as well as power-law coherency. Presen…
Efficient online kernel CUSUM detects changes quickly and accurately.
In this paper we use a time-evolving graph which consists of a sequence of graph snapshots over time to model many real-world networks. We study the path classification problem in a time-evolving graph, which has many applications in real-world scenarios, for example, predicting path failure in a telecommunication netw…
Study shows memory needs grow with task sequence length in continual learning.
Paper presents methods to create stock price confidence intervals using LSTM models.
We investigated distributions of short term price trends for high frequency stock market data. A number of trends as a function of their lengths was measured. We found that such a distribution does not fit to results following from an uncorrelated stochastic process. We proposed a simple model with a memory that gives …
Given a system of equations in a "random" finitely generated subgroup of the braid group, we show how to find a small ordered list of elements in the subgroup, which contains a solution to the equations with a significant probability. Moreover, with a significant probability, the solution will be the first in the list.…
We introduce for the first time the utilization of Long short-term memory (LSTM) neural network architectures for the compensation of fiber nonlinearities in digital coherent systems. We conduct numerical simulations considering either C-band or O-band transmission systems for single channel and multi-channel 16-QAM mo…
Paper introduces a new framework to improve sample efficiency in POMDPs learning.
Truncated backpropagation through time (TBPTT) is a popular method for learning in recurrent neural networks (RNNs) that saves computation and memory at the cost of bias by truncating backpropagation after a fixed number of lags. In practice, choosing the optimal truncation length is difficult: TBPTT will not converge …
Time series forecasting is an important problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. In this paper, we propose to tackle such forecasting problem with Transformer [1]. Although impressed by its performance in our preliminary study, …
Efficiently infers latent SDEs with scalable memory and time costs.
Kernel networks' stability edge linked to Fisher Information singularity.
New method improves FO-BLO convergence without increasing memory or time complexity.
Efficient sparse attention reduces self-attention complexity and improves model performance.
New algorithm reduces decision switching in dynamic environments.
New linear models improve time series classification efficiency and interpretability.
Proposes a neural network for learning step-size policies for L-BFGS optimization.
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures lik…
We examine the scaling regime for the detrended fluctuation analysis (DFA) - the most popular method used to detect the presence of long memory in data and the fractal structure of time series. First, the scaling range for DFA is studied for uncorrelated data as a function of length of time series and regression li…
New model captures long-range patterns in sequences efficiently.
Revisiting Trade-sign Long-memory and Square-root Law price impact
Linformer reduces transformer complexity to linear, improving efficiency.