Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “pipeline caching”
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscs.MA
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
Paper proposes EEIPU, a memoization-aware BO algorithm to reduce hyperparameter tuning costs.
problem High costs in GPU-days for training and fine-tuning language models.
method Memoization-aware Bayesian Optimization (EEIPU) algorithm in tandem with pipeline caching.
result EEIPU produces 103% more hyperparameter candidates and 108% more validation metric improvement.
DEAP Cache learns prefetching, eviction, and admission using machine learning.
problem Improving cache performance through better prefetching, eviction, and admission strategies.
method End-to-end pipeline using machine learning, inspired by pretraining on large corpora and online reinforcement learning.
result Optimal policy distribution between two orthogonal eviction strategies based on frequency and recency.
This study investigates the use of reinforcement learning to guide a general purpose cache manager decisions. Cache managers directly impact the overall performance of computer systems. They govern decisions about which objects should be cached, the duration they should be cached for, and decides on which objects to ev…
A deep learning model predicts future cache accesses with high accuracy.
problem Improving cache management and performance through better prediction of future data accesses.
method Proposed a LSTM-based recurrent neural network model to predict future cache accesses using only a cache trace as input.
result The proposed model achieves high prediction accuracy and outperforms state-of-the-art practical policies.
Parrot learns optimal cache replacement policies using imitation learning.
problem Improving cache hit rates in complex access patterns.
method Imitation learning approach using Belady's oracle policy.
result Parrot increases cache hit rates by 61% on a web search benchmark.
Support Vector Machines (SVMs) can solve structured multi-output learning problems such as multi-label classification, multiclass classification and vector regression. SVM training is expensive especially for large and high dimensional datasets. The bottleneck of the SVM training often lies in the kernel value computat…
We developed a caching method to speed up concept learning in complex knowledge bases.
problem Complex concept learning requires many instance retrieval calls, increasing runtime.
method Semantics-aware caching that links concepts to instances via crisp set operations.
result Our cache reduces concept retrieval and learning runtime by an order of magnitude.
Express improves causal attention guarantees for language models.
problem Improving causal attention guarantees for language models.
method Introducing Express, a tool for converting non-causal attention into causal with matching guarantees.
result Express improves causal attention guarantees to log^(3/2)(n)/s with minimal memory and compression overhead.
Training large-scale image recognition models is computationally expensive. This raises the question of whether there might be simple ways to improve the test performance of an already trained model without having to re-train or fine-tune it with new data. Here, we show that, surprisingly, this is indeed possible. The …
SIVF k-means algorithm speeds up sparse data clustering.
problem Efficiently clustering large-scale high-dimensional sparse data.
method Inverted-file structure for centroids, filter-based similarity reduction.
result SIVF achieves higher speed and lower memory consumption.
New algorithm improves cache management with delayed feedback and decaying costs.
problem Improving cache replacement policies with delayed and decaying costs feedback.
method Proposed EXP4-DFDC adaptive reinforcement learning algorithm.
result Expected regret is a vanishing quantity as a function of time.
AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reusestat.ML
New method speeds up diffusion models without sacrificing quality.
problem Slow inference in diffusion models.
method Adams-Bashforth method for caching and acceleration.
result Achieved nearly 3x speedup with maintained quality.
Due to the drastic increase of mobile traffic, wireless caching is proposed to serve repeated requests for content download. To determine the caching scheme for decentralized caching networks, the content preference learning problem based on mobility prediction is studied. We first formulate preference prediction as a …
This paper optimizes caching and model multiplexing for large model inference.
problem Resource consumption and latency challenges in large model deployment.
method Jointly optimizing a caching algorithm (GDSF or LEC) and a model multiplexer for large model inference.
result Achieves optimal rates in offline and online settings with up to 50x improvement over baseline.
A new method for energy-efficient file delivery in small cell networks.
problem Efficient resource management in femto-caching with time-variant statistical properties.
method Formulates a resource allocation problem as a stochastic knapsack problem and a multi-armed bandit problem, developing solutions for each.
result The proposed method maximizes the accumulated utility over the horizon, especially suitable for networks with time-variant statistical properties.
Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary i…
Over the last few years, Deep Neural Networks (DNNs) have become ubiquitous owing to their high accuracy on real-world tasks. However, this increase in accuracy comes at the cost of computationally expensive models leading to higher prediction latencies. Prior efforts to reduce this latency such as quantization, model …
Existing multi-armed bandit (MAB) models make two implicit assumptions: an arm generates a payoff only when it is played, and the agent observes every payoff that is generated. This paper introduces synchronization bandits, a MAB variant where all arms generate costs at all times, but the agent observes an arm's instan…
LATM framework uses LLMs to create and reuse tools for efficient problem-solving.
problem Improving problem-solving capabilities of LLMs with external tools.
method Closed-loop framework with two phases: tool making and tool using. LLMs act as both tool makers and users. Resource-intensive model for tool making, lightweight model for tool using.
result LATM framework achieves performance equivalent to using a powerful LLM for both roles but with significantly reduced costs.
In this paper, the distributed edge caching problem in fog radio access networks (F-RANs) is investigated. By considering the unknown spatio-temporal content popularity and user preference, a user request model based on hidden Markov process is proposed to characterize the fluctuant spatio-temporal traffic demands in F…
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the required memory. We present Bandana, a storage system that reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as the prim…
Deep Neural Networks (DNNs) have revolutionized numerous applications, but the demand for ever more performance remains unabated. Scaling DNN computations to larger clusters is generally done by distributing tasks in batch mode using methods such as distributed synchronous SGD. Among the issues with this approach is th…
FibQuant improves KV-cache compression for long-context inference.
problem Memory traffic bottleneck in long-context inference due to KV cache growth.
method Introduces FibQuant, a universal vector quantizer that combines Beta-quantile radii, Fibonacci/Roberts-Kronecker directions, and Lloyd-Max refinement.
result FibQuant achieves high compression rates with minimal loss in attention cosine similarity.
We address challenges of active learning under scarce informational resources in non-stationary environments. In real-world settings, data labeled and integrated into a predictive model may become invalid over time. However, the data can become informative again with switches in context and such changes may indicate un…
Research optimizes C++ patterns for HFT, reducing latency and improving profitability.
problem Optimizing latency-critical code for high-frequency trading systems.
method Creation of a Low-Latency Programming Repository, optimisation of trading strategy, implementation of Disruptor pattern.
result Significant performance improvements in speed and profitability.
This paper is about metric data structures in high-dimensional or non-Euclidean space that permit cached sufficient statistics accelerations of learning algorithms. It has recently been shown that for less than about 10 dimensions, decorating kd-trees with additional "cached sufficient statistics" such as first and sec…
Low power digital signal processors (DSPs) typically have a very limited amount of memory in which to cache data. In this paper we develop efficient bottleneck feature (BNF) extractors that can be run on a DSP, and retrain a baseline large-vocabulary continuous speech recognition (LVCSR) system to use these BNFs with o…
Generating logical form equivalents of human language is a fresh way to employ neural architectures where long short-term memory effectively captures dependencies in both encoder and decoder units. The logical form of the sequence usually preserves information from the natural language side in the form of similar token…
In this paper, we propose several improvements on the block-coordinate Frank-Wolfe (BCFW) algorithm from Lacoste-Julien et al. (2013) recently used to optimize the structured support vector machine (SSVM) objective in the context of structured prediction, though it has wider applications. The key intuition behind our i…
In this paper we analyze, evaluate, and improve the performance of training generalized linear models on modern CPUs. We start with a state-of-the-art asynchronous parallel training algorithm, identify system-level performance bottlenecks, and apply optimizations that improve data parallelism, cache line locality, and …
The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline c…
Paper accelerates K-means clustering for large sparse document data.
problem Efficiently clustering large-scale sparse document data.
method Designs an AFM algorithm leveraging UCs and inverted-index structure.
result Significantly improves clustering speed for large-scale documents.
Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML…
Study proposes a statistical testing framework for evaluating clustering pipelines.
problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.
Investigates fairness in pipeline models where individuals may drop out.
problem Fairness in pipeline models where individuals may drop out and subsequent stages depend on remaining individuals.
method Rigorous framework for evaluating fairness guarantees, showing that naïve auditing is insufficient and dependence must exist between stages.
result Fairness in pipelines can be arbitrary, even with just two stages, and requires dependence between stages.
Transformer is a popularly used neural network architecture, especially for language understanding. We introduce an extended and unified architecture that can be used for tasks involving a variety of modalities like image, text, videos, etc. We propose a spatio-temporal cache mechanism that enables learning spatial dim…
Developing efficient and scalable algorithms for Latent Dirichlet Allocation (LDA) is of wide interest for many applications. Previous work has developed an O(1) Metropolis-Hastings sampling method for each token. However, the performance is far from being optimal due to random accesses to the parameter matrices and fr…
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
Automates supervised learning pipeline design with matrix and tensor factorization.
problem Designing effective supervised learning pipelines with many choices.
method Uses matrix and tensor factorization to model pipeline search space and develops greedy experiment design protocols.
result Demonstrates the effectiveness of the approach on real-world classification problems.
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
A new method corrects bias in machine learning for trading by filtering out non-executable prices.
problem Contaminated data from price limits causes misleading performance metrics in trading models.
method Mask-first design with tradability filters, GPU optimization, and penalized losses.
result The system achieves Sharpe ratios of 1.63 on real data, significantly outperforming existing methods.
New model optimizes oil product distribution via pipelines.
problem Optimizing oil product distribution via pipelines.
method Discrete-time mixed integer linear programming model.
result Significant reductions in pipeline operational cost.
Pipeline decomposes portfolio optimization problems into smaller, solvable subproblems.
problem Large-scale portfolio optimization with constraints.
method Decomposition pipeline with preprocessing, clustering, and risk rebalancing.
result Pipeline reduces problem size by 80% and computation time.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.