arXiv research

FibQuant improves KV-cache compression for long-context inference.

KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.

Express improves causal attention guarantees for language models.

WildCat efficiently compresses neural network attention mechanisms.

New algorithm reduces costs and latency for large language model inference.

Deep analysis of KV-Poisson structures and new invariant computed.

For a compact oriented surface $Σ$ of genus $g$ with $n+1$ boundary components, the space $\mathfrak{g}(Σ)$ spanned by free homotopy classes of loops in $Σ$ carries the structure of a Lie bialgebra equipped with a natural decreasing filtration, whose structure morphisms are called the Goldman bracket and the (framed) T…

We define a family ${\rm KV}^{(g,n)}$ of Kashiwara-Vergne problems associated with compact connected oriented 2-manifolds of genus $g$ with $n+1$ boundary components. The problem ${\rm KV}^{(0,3)}$ is the classical Kashiwara-Vergne problem from Lie theory. We show the existence of solutions of ${\rm KV}^{(g,n)}$ for ar…

The paper computes KV cochain differentials and their geometric implications.

The paper quantizes Hessian structures on R^2 using KV-algebras.

MSD removes dequantization bottleneck in LLM inference by approximating high-precision activations.

This study investigates the use of reinforcement learning to guide a general purpose cache manager decisions. Cache managers directly impact the overall performance of computer systems. They govern decisions about which objects should be cached, the duration they should be cached for, and decides on which objects to ev…

A deep learning model predicts future cache accesses with high accuracy.

Parrot learns optimal cache replacement policies using imitation learning.

As the industry deploys increasingly large and complex neural networks to mobile devices, more pressure is put on the memory and compute resources of those devices. Deep compression, or compression of deep neural network weight matrices, is a technique to stretch resources for such scenarios. Existing compression metho…

DEAP Cache learns prefetching, eviction, and admission using machine learning.

Support Vector Machines (SVMs) can solve structured multi-output learning problems such as multi-label classification, multiclass classification and vector regression. SVM training is expensive especially for large and high dimensional datasets. The bottleneck of the SVM training often lies in the kernel value computat…

Transformer is a popularly used neural network architecture, especially for language understanding. We introduce an extended and unified architecture that can be used for tasks involving a variety of modalities like image, text, videos, etc. We propose a spatio-temporal cache mechanism that enables learning spatial dim…

New MAB model for online caching costs.

We developed a caching method to speed up concept learning in complex knowledge bases.

The cohomology of affine structures on Lie groups is compared with that of Koszul-Vinberg algebras.

Training large-scale image recognition models is computationally expensive. This raises the question of whether there might be simple ways to improve the test performance of an already trained model without having to re-train or fine-tune it with new data. Here, we show that, surprisingly, this is indeed possible. The …

New algorithm improves cache management with delayed feedback and decaying costs.

New method speeds up diffusion models without sacrificing quality.

In this paper, we describe a surprising link between the theory of the Goldman-Turaev Lie bialgebra on surfaces of genus zero and the Kashiwara-Vergne (KV) problem in Lie theory. Let $Σ$ be an oriented 2-dimensional manifold with non-empty boundary and $\mathbb{K}$ a field of characteristic zero. The Goldman-Turaev Lie…

Due to the drastic increase of mobile traffic, wireless caching is proposed to serve repeated requests for content download. To determine the caching scheme for decentralized caching networks, the content preference learning problem based on mobility prediction is studied. We first formulate preference prediction as a …

This paper optimizes caching and model multiplexing for large model inference.

A new method for energy-efficient file delivery in small cell networks.

Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary i…

Over the last few years, Deep Neural Networks (DNNs) have become ubiquitous owing to their high accuracy on real-world tasks. However, this increase in accuracy comes at the cost of computationally expensive models leading to higher prediction latencies. Prior efforts to reduce this latency such as quantization, model …

LATM framework uses LLMs to create and reuse tools for efficient problem-solving.

Generative compression technique reduces neural network size and improves performance on microcontrollers.

In this paper, the distributed edge caching problem in fog radio access networks (F-RANs) is investigated. By considering the unknown spatio-temporal content popularity and user preference, a user request model based on hidden Markov process is proposed to characterize the fluctuant spatio-temporal traffic demands in F…

In the framework of the theory of differential coverings \cite{KV}, we discuss a general geometric construction that serves the base for the so-called Lax pairs containing differentiation with respect to the spectral parameter \cite{OS}. Such kind of objects arise, for example, when studying integrability properties of…

Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the required memory. We present Bandana, a storage system that reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as the prim…

KATA improves associative recall by optimizing feature maps derived from nonnegative attention weights.

Study on deformation of affine structures on Lie groups using cohomology.

Study on existence of harmonic metrics for non-Hermitian Yang-Mills bundles.

TabPFN-3 scales tabular prediction models to large datasets and improves performance and speed.

We address challenges of active learning under scarce informational resources in non-stationary environments. In real-world settings, data labeled and integrated into a predictive model may become invalid over time. However, the data can become informative again with switches in context and such changes may indicate un…

Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. Unfortunately, using large batches is slow and expensive on conventional hardware. Thus, it would be nice if we could generate batches that were effectively large though actual…

Research optimizes C++ patterns for HFT, reducing latency and improving profitability.

This paper is about metric data structures in high-dimensional or non-Euclidean space that permit cached sufficient statistics accelerations of learning algorithms. It has recently been shown that for less than about 10 dimensions, decorating kd-trees with additional "cached sufficient statistics" such as first and sec…

This work is devoted to an intrinsic cohomology theory of Koszul-Vinberg algebras and their modules. Our results may be regarded as improvements of the attempt by Albert Nijenhuis in [NA]. The relationships between the cohomology theory developed here and some classical problems are pointed out, e.g. extensions of alge…

Low power digital signal processors (DSPs) typically have a very limited amount of memory in which to cache data. In this paper we develop efficient bottleneck feature (BNF) extractors that can be run on a DSP, and retrain a baseline large-vocabulary continuous speech recognition (LVCSR) system to use these BNFs with o…

Generating logical form equivalents of human language is a fresh way to employ neural architectures where long short-term memory effectively captures dependencies in both encoder and decoder units. The logical form of the sequence usually preserves information from the natural language side in the form of similar token…

In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our i…

In this paper we analyze, evaluate, and improve the performance of training generalized linear models on modern CPUs. We start with a state-of-the-art asynchronous parallel training algorithm, identify system-level performance bottlenecks, and apply optimizations that improve data parallelism, cache line locality, and …