arXiv research

Quadratic memory needed for linear prediction in streaming model.

Paper studies statistical-computational trade-offs in tensor PCA and related problems.

New framework captures long-term decision dependence in online learning.

Efficiently tests discrete distributions with limited memory and communication.

We derive upper and lower bounds for the policy regret of $T$-round online learning problems with graph-structured feedback, where the adversary is nonoblivious but assumed to have a bounded memory. We obtain upper bounds of $\widetilde O(T^{2/3})$ and $\widetilde O(T^{3/4})$ for strongly-observable and weakly-observab…

Survey on statistical inference under memory constraints.

The paper aims to develop new combinatorial dimensions for bounded memory learning.

Study reduces memory needs for active learning with enriched queries.

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

Quadratic memory is essential for optimal convex optimization queries.

A new method stabilizes deep reinforcement learning by using QGraphs to retain replay memory information.

Gradient descent is efficient for solving feasibility problems with minimal memory and queries.

New algorithm reduces memory usage for multi-pass bandit problems.

Improved single-pass streaming MAB regret bound to O(K^1/3T^2/3).

We analyze computational limits of modern Hopfield models based on pattern norms.

New algorithms for streaming bandits with limited memory.

Study on bandits with fading memory, improving regret bounds.

We propose a new method to study the internal memory used by reinforcement learning policies. We estimate the amount of relevant past information by estimating mutual information between behavior histories and the current action of an agent. We perform this estimation in the passive setting, that is, we do not interven…

This work optimizes quantization of linear models to reduce memory usage.

New algorithms provide long-term memory guarantees for online learning.

New analysis tightens memory capacity of Hopfield models using spherical codes.

A central challenge faced by memory systems is the robust retrieval of a stored pattern in the presence of interference due to other stored patterns and noise. A theoretically well-founded solution to robust retrieval is given by attractor dynamics, which iteratively clean up patterns during recall. However, incorporat…

The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…

Transformers recall from long distributions with statistical guarantees.

Nonlinear RNNs' memory capacity varies widely, making it impractical.

We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness…

This paper provides a mathematical framework for time-delay reservoir computing.

ECPv2 optimizes Lipschitz functions efficiently and scalably.

We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word level generative model for natural language. It uses a recurrent neural network with a dynamic attention and canvas memory mechanism to iteratively construct sentences. By viewing the state of the memory at intermediate stages and where the m…

Global convergence of an online (stochastic) limited memory version of the Broyden-Fletcher- Goldfarb-Shanno (BFGS) quasi-Newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established. Lower and upper bounds on the Hessian eigenvalues of the sample …

Study shows how to balance memory and learning efficiency in continual learning.

Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an emphasis on biological plausibility of memory dynamics and learning. We find that …

We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice. It adds elements from a stream either to a solution set $S$ or to an extra buffer $B$ based on two adaptive thresholds, and improves $S$ by a final…

Memory constraints slow linear regression convergence.

The study examines how investor protection and past information affect stock returns and interest rates.

New algorithm reduces control error in systems with changing dynamics.

New algorithm reduces memory-regret trade-off for bandits.

Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine learning, representing the optimization backbone for training several classic models, from regression to neural networks. Given the recent practical focus on distributed machine learning, significant work has been dedicated to the convergence prope…

This paper introduces a hierarchical associative memory model with multiple layers.

The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.

Compact DNNs increase memory footprint and reduce energy efficiency.

Study identifies three quantization regimes for ReLU networks.

It has been known for a long time that the classical spherical perceptrons can be used as storage memories. Seminal work of Gardner, \cite{Gar88}, started an analytical study of perceptrons storage abilities. Many of the Gardner's predictions obtained through statistical mechanics tools have been rigorously justified. …

New constraints on space and adaptivity in bandits force more batches and memory use.

New algorithms solve convex optimization problems with limited memory.

ARMIN improves memory efficiency and lightness in neural networks.

A method models nonlinear dynamics from data using barycentric coordinates and memory.

LDAdam optimizes large models with low memory by adapting to lower-dimensional subspaces.