Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all s…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “effective reuse rate”
Bayesian Experience Reuse improves learning from multiple experts.
problem Learning from multiple experts with conflicting goals.
method Bayesian neural networks with shared features to model uncertainty and derive a probability distribution over expert models.
result BERS method effectively samples demonstrations from the derived distribution to reuse them in new tasks.
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscs.MA
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature. We propose a novel and effective approach to handle concept drift via model reuse, leveraging previous knowledge by reusing models. Each…
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
Although reinforcement learning has made great strides recently, a continuing limitation is that it requires an extremely high number of interactions with the environment. In this paper, we explore the effectiveness of reusing experience from the experience replay buffer in the Deep Q-Learning algorithm. We test the ef…
Improves reinforcement learning stability and efficiency.
problem Combining stability and efficiency in reinforcement learning.
method Combines on-policy stability with off-policy sample reuse.
result Demonstrates improved performance in both theory and practice.
VRER selectively reuses samples to improve policy optimization in complex systems.
problem Lack of effective reuse of historical samples in reinforcement learning.
method Variance reduction based experience replay (VRER) framework.
result VRER accelerates policy optimization and enhances performance.
The paper analyzes how repeating epochs affects data scaling in linear regression.
problem Understanding how to scale data for multi-epoch training in linear regression.
method Theoretical analysis of effective reuse rate (E(K, N)) under strong convexity or Zipf-distributed data.
result The effective reuse rate E(K, N) plateaus at a problem-dependent value that grows with N, indicating diminishing marginal gains.
An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, f…
New method uses rank-conditioned Horvitz-Thompson estimation for unbiased sample reuse in Plackett-Luce best-of-K objective.
problem Estimating the expected maximum reward in Plackett-Luce draws without replacement.
method Rank-conditioned Horvitz-Thompson estimation with joint-score REINFORCE for unbiased sample reuse.
result Unbiased estimation of the Plackett-Luce best-of-K objective with finite second moment guarantees.
New learning rate approach reveals phase transitions in SGD performance.
problem Understanding feature learning dynamics in neural networks.
method Characterizing the relationship between learning rate(s) and sample complexity for gradient-based algorithms.
result Phase transition from information exponent to generative exponent regime with different learning rates.
Study confirms eurozone interbank market stability but finds higher collateral reuse.
problem Analyzing eurozone interbank market behavior and stability.
method Examined secured transactions data from ECB, tested stylized facts, measured network properties.
result Observed higher collateral reuse and network symmetry compared to unsecured markets.
The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
Transfer learning improves image classifier performance in data-starved regimes.
problem Deploying image classifiers in domains with limited labeled data.
method Transfer learning with deep neural networks, focusing on feature reuse and overparameterization.
result Transfer learning enhances CNN performance in data-starved regimes.
This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation. Simple rules are designed to detect slowly-varying gradients and, therefore, trigger the reuse of outdated gradients. The resultant gra…
The paper investigates what enables successful transfer learning and separates feature reuse from data statistics.
problem Understanding what enables successful transfer learning and identifying the responsible parts of the network.
method Analyzes transfer learning on block-shuffled images to distinguish feature reuse from data statistics.
result Some benefit of transfer learning comes from learning low-level statistics of data, not just feature reuse.
This paper develops a communication-efficient algorithm to solve the stochastic optimization problem defined over a distributed network, aiming at reducing the burdensome communication in applications such as distributed machine learning.Different from the existing works based on quantization and sparsification, we int…
We propose several ways of reusing subword embeddings and other weights in subword-aware neural language models. The proposed techniques do not benefit a competitive character-aware model, but some of them improve the performance of syllable- and morpheme-aware models while showing significant reductions in model sizes…
New RL algorithms improve control tasks with data reuse.
problem Real-world control requires performance guarantees and data efficiency.
method Generalized Policy Improvement combining on-policy guarantees and sample reuse.
result Extensive experimental analysis shows benefits of new algorithms.
Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…
VRER selectively reuses past observations to reduce variance in policy optimization.
problem Lack of effective experience replay for accelerating policy optimization in complex systems.
method Variance Reduction Experience Replay (VRER) framework that selectively reuses informative samples.
result VRER reduces gradient variance and improves policy learning over state-of-the-art algorithms.
When concept drift is detected during classification in a data stream, a common remedy is to retrain a framework's classifier. However, this loses useful information if the classifier has learnt the current concept well, and this concept will recur again in the future. Some frameworks retain and reuse classifiers, but …
Excessive reuse of holdout data can lead to overfitting. However, there is little concrete evidence of significant overfitting due to holdout reuse in popular multiclass benchmarks today. Known results show that, in the worst-case, revealing the accuracy of adaptively chosen classifiers on a data set of size al…
Generalized Hindsight improves RL by reusing data from one task for another.
problem High sample complexity in reinforcement learning due to wasted uninformative data.
method Approximate inverse reinforcement learning to relabel behaviors with better-suited tasks.
result Efficient reuse of samples, reducing sample complexity on multi-task RL tasks.
ACS is an interactive framework for model-free selection with guaranteed error control.
problem Model-free selection with rigorous error control.
method Adaptive conformal selection with human-in-the-loop data exploration and new information incorporation.
result ACS provides concrete selection algorithms for various goals, including model update/selection, diversified selection, and incorporating new data.
AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reusestat.ML
New method speeds up diffusion models without sacrificing quality.
problem Slow inference in diffusion models.
method Adams-Bashforth method for caching and acceleration.
result Achieved nearly 3x speedup with maintained quality.
The policy gradient approach is a flexible and powerful reinforcement learning method particularly for problems with continuous actions such as robot control. A common challenge in this scenario is how to reduce the variance of policy gradient estimates for reliable policy updates. In this paper, we combine the followi…
Develops a multilevel Monte Carlo framework with dropout for efficient uncertainty quantification.
problem Efficiently quantify uncertainty in complex models using dropout.
method Integrates multilevel Monte Carlo with Monte Carlo dropout, creating coupled estimators to reduce variance.
result Demonstrates significant variance reduction and efficiency gains over single-level Monte Carlo dropout.
Nested Mini-Batch K-Meansstat.ML
A new algorithm is proposed which accelerates the mini-batch k-means algorithm of Sculley (2010) by using the distance bounding approach of Elkan (2003). We argue that, when incorporating distance bounds into a mini-batch algorithm, already used data should preferentially be reused. To this end we propose using nested …
Clapping reduces memory usage in distributed optimization by reusing data samples.
problem Significant communication overhead and impractical memory overhead in pipeline-parallel distributed optimization.
method Lazy sampling strategy to reuse data samples across steps, supporting convergence without unbiased gradient assumptions.
result Clapping achieves convergence in few-epoch or online training regimes without sample-size memory overhead.
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show that a program synthesis approach that combines gradient descent w…
The study addresses biases in evaluating molecular optimization methods and proposes methods to reduce these biases.
problem Biases in in silico evaluation of molecular optimization methods.
method Discussion and empirical investigation of bias reduction methods for predictor misspecification and sample reuse.
result Empirical investigation of bias reduction methods for predictor misspecification and sample reuse.
Given a publicly available pool of machine learning models constructed for various tasks, when a user plans to build a model for her own machine learning application, is it possible to build upon models in the pool such that the previous efforts on these existing models can be reused rather than starting from scratch? …
We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing successful architectures that improves sample efficiency. We aim to address the limitation of current network transformation operations that…
Improved scaling laws in linear regression using data reuse.
problem Sustainability of neural scaling laws when running out of new data.
method Data reuse in multi-pass stochastic gradient descent (multi-pass SGD) for -dimensional linear models trained on data with sketched features.
result Multi-pass SGD achieves a test error of with , improving scaling laws in data-constrained regimes.
Transfer learning from natural image datasets, particularly ImageNet, using standard large models and corresponding pretrained weights has become a de-facto method for deep learning applications to medical imaging. However, there are fundamental differences in data sizes, features and task specifications between natura…
This paper proposes reusing CNN layers to speed up hyperparameters tuning.
problem Time-consuming hyperparameters tuning in CNNs.
method Reuse trained convolutional layers among different trainings.
result Reduces training time and increases accuracy of neural networks.
Recently, Transformer has gained success in automatic speech recognition (ASR) field. However, it is challenging to deploy a Transformer-based end-to-end (E2E) model for online speech recognition. In this paper, we propose the Transformer-based online CTC/attention E2E ASR architecture, which contains the chunk self-at…
WSqD extends learning rate schedules for large model training without fixed horizons.
problem Fixed learning rate schedules limit training horizon extension.
method WSqD replaces constant stable phase with a shifted inverse-square-root base, retaining linear cooldown.
result WSqD achieves minimax-optimal convergence rate and horizon-independence.
Method predicts rarity of image features to support research integrity investigations.
problem Difficulty in determining if image reuse is by chance or intentional.
method Statistical estimation of ORB features' chance occurrence across PubMed Open Access Subset dataset.
result The method produces decreasingly smaller p-values for more complex imagery, supporting null hypothesis.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
QuantaAlpha uses evolutionary algorithms to mine financial alpha robustly across market distributions.
problem Challenges in alpha mining due to market noise and regime shifts.
method Evolutionary framework treating each mining run as a trajectory, mutation, crossover, targeted revision, and reuse of effective patterns.
result Consistent gains over strong baselines and prior systems, achieving high IC and ARR.
Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all involve test data reuse beyond guidance by statistical confidence bounds. Nonetheless, recent replication studies give evidence that popular…
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
Two-layer networks learn faster with batch reuse, overcoming information and leap exponents.
problem Limitations of gradient flow and single-pass GD in learning multi-index target functions.
method Multi-pass gradient descent that reuses batches, analyzed using Dynamical Mean-Field Theory.
result Two-time-step overlap with target subspace for non-staircase functions, overcoming information and leap exponents.
New method shows how order of gradient updates impacts stability and convergence in deep learning.
problem Training deep learning models can be unstable and computationally expensive.
method Theoretical analysis and experiments with backward-SGD.
result The order of gradient updates affects stability and convergence, leading to improved performance.