A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A method for learning complex functions from data with reduced memory usage.
New memory-query tradeoffs for convex optimization algorithms.
Memory-constrained algorithms need superlinear memory for efficient convex optimization.
New method improves FO-BLO convergence without increasing memory or time complexity.
Paper tackles online optimization with memory and competitive control.
PySAD offers a unified Python framework for efficient streaming anomaly detection.
New algorithm learns and unlearns from streaming data efficiently.
New algorithms solve convex optimization problems with limited memory.
GPA improves LLM training speed by 8.71% for Llama-160M models.
New algorithm optimizes auction prices in real-time.
Efficiently infers latent SDEs with scalable memory and time costs.
A new algorithm reduces memory usage for long token attention in streaming applications.
Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A n…
In this paper, we revisit the recurrent back-propagation (RBP) algorithm, discuss the conditions under which it applies as well as how to satisfy them in deep neural networks. We show that RBP can be unstable and propose two variants based on conjugate gradient on the normal equations (CG-RBP) and Neumann series (Neuma…
Tail averaging consists in averaging the last examples in a stream. Common techniques either have a memory requirement which grows with the number of samples to average, are not available at every timestep or do not accomodate growing windows. We propose two techniques with a low constant memory cost that perform tail …
We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, -dimensional samples are presented sequentially, and the goal is to produce the -dimensional subspace that best approximates these points. Standard algorithms require memory; mea…
The computation of the sparse principal component of a matrix is equivalent to the identification of its principal submatrix with the largest maximum eigenvalue. Finding this optimal submatrix is what renders the problem -hard. In this work, we prove that, if the matrix is positive semidefinite and its …
Gradient descent is efficient for solving feasibility problems with minimal memory and queries.
Understanding the statistical properties of recurrence intervals of extreme events is crucial to risk assessment and management of complex systems. The probability distributions and correlations of recurrence intervals for many systems have been extensively investigated. However, the impacts of microscopic rules of a c…
We note that known methods achieving the optimal oracle complexity for first order convex optimization require quadratic memory, and ask whether this is necessary, and more broadly seek to characterize the minimax number of first order queries required to optimize a convex Lipschitz function subject to a memory constra…
In this paper we introduce a novel online time series forecasting model we refer to as the pM-GP filter. We show that our model is equivalent to Gaussian process regression, with the advantage that both online forecasting and online learning of the hyper-parameters have a constant (rather than cubic) time complexity an…
Quadratic memory is essential for optimal convex optimization queries.
New technique reduces memory usage and boosts neural network training speed.
We study the problem of identifying correlations in multivariate data, under information constraints: Either on the amount of memory that can be used by the algorithm, or the amount of communication when the data is distributed across several machines. We prove a tight trade-off between the memory/communication complex…
Exponential smoothers are a simple and memory efficient way to compute running averages of time series. Here we define and describe practical properties of exponential smoothers for signals observed at constant and variable intervals.
Study on deleting user data in linear regression models to maintain limited memory.
Extended LSTM improves language modeling performance.
Study shows how to balance memory and learning efficiency in continual learning.
Comprehending complex systems by simplifying and highlighting important dynamical patterns requires modeling and mapping higher-order network flows. However, complex systems come in many forms and demand a range of representations, including memory and multilayer networks, which in turn call for versatile community-det…
We give explicit representation formulas for marginally trapped submanifolds of co-dimension two in pseudo-Riemannian spaces with arbitrary signature and constant sectional curvature. This paper is dedicated to the memory of Franki Dillen, 1963-2013.
Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.
In this paper we prove that the only algebraic constant mean curvature (cmc) surfaces in R^3 of order less than four are the planes, the spheres and the cylinders. The method used heavily depends on the efficiency of algorithms to compute Groebner Bases and also on the memory capacity of the computer used to do the com…
Model predicts EMF of Ni-Mn-Ga MSMA, improved with GRNN.
We consider the infinite-horizon discounted optimal control problem formalized by Markov Decision Processes. We focus on several approximate variations of the Policy Iteration algorithm: Approximate Policy Iteration, Conservative Policy Iteration (CPI), a natural adaptation of the Policy Search by Dynamic Programming a…
Implicit models can match or exceed explicit models with more test-time compute.
FPG uses fractional calculus for efficient reinforcement learning with long-term memory.
New neural networks model complex phenomena with fewer parameters.
Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern…
Recent empirical studies have demonstrated long-memory in the signs of orders to buy or sell in financial markets [2, 19]. We show how this can be caused by delays in market clearing. Under the common practice of order splitting, large orders are broken up into pieces and executed incrementally. If the size of such lar…
We analyze computational limits of modern Hopfield models based on pattern norms.
mGRN improves multivariate time series prediction by managing marginal and joint memories.
Efficient online kernel CUSUM detects changes quickly and accurately.
Efficient sampling reduces memory usage for Minimax distance analysis.
Learning to remember long sequences remains a challenging task for recurrent neural networks. Register memory and attention mechanisms were both proposed to resolve the issue with either high computational cost to retain memory differentiability, or by discounting the RNN representation learning towards encoding shorte…
In recent years, memory-augmented neural networks(MANNs) have shown promising power to enhance the memory ability of neural networks for sequential processing tasks. However, previous MANNs suffer from complex memory addressing mechanism, making them relatively hard to train and causing computational overheads. Moreove…
A new algorithm combines SGD and Thompson Sampling for contextual bandits.
Iterative learning to infer approaches have become popular solvers for inverse problems. However, their memory requirements during training grow linearly with model depth, limiting in practice model expressiveness. In this work, we propose an iterative inverse model with constant memory that relies on invertible networ…
New neural operators model turbulence with memory and randomness.