ECPF improves classification accuracy and speed for evolving data streams.
problem Reusing classifiers trained on recurring concepts to maintain accuracy and speed.
method ECPF uses similarity of classifications to quickly identify the best classifier to reuse.
result ECPF significantly outperforms state-of-the-art frameworks on synthetic and real-world datasets.
New bounds show multiclass problems reduce overfitting from test set reuse.
problem Overfitting from test set reuse in multiclass problems.
method Upper and lower bounds on bias of attacks, practical and inefficient attacks.
result Multiclass problems mitigate overfitting from test set reuse.
Transfer learning improves image classifier performance in data-starved regimes.
problem Deploying image classifiers in domains with limited labeled data.
method Transfer learning with deep neural networks, focusing on feature reuse and overparameterization.
result Transfer learning enhances CNN performance in data-starved regimes.
Entity resolution (ER) is one of the fundamental problems in data integration, where machine learning (ML) based classifiers often provide the state-of-the-art results. Considerable human effort goes into feature engineering and training data creation. In this paper, we investigate a new problem: Given a dataset D_T fo…
New samplers improve compositional generation with diffusion models.
problem Improving compositional generation with diffusion models.
method Score-based interpretation, energy-based parameterization, Metropolis-corrected samplers.
result New samplers enable successful compositional generation across various tasks.
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
We propose several ways of reusing subword embeddings and other weights in subword-aware neural language models. The proposed techniques do not benefit a competitive character-aware model, but some of them improve the performance of syllable- and morpheme-aware models while showing significant reductions in model sizes…
New RL algorithms improve control tasks with data reuse.
problem Real-world control requires performance guarantees and data efficiency.
method Generalized Policy Improvement combining on-policy guarantees and sample reuse.
result Extensive experimental analysis shows benefits of new algorithms.
Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all s…
Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…
Bayesian Experience Reuse improves learning from multiple experts.
problem Learning from multiple experts with conflicting goals.
method Bayesian neural networks with shared features to model uncertainty and derive a probability distribution over expert models.
result BERS method effectively samples demonstrations from the derived distribution to reuse them in new tasks.
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature. We propose a novel and effective approach to handle concept drift via model reuse, leveraging previous knowledge by reusing models. Each…
Paper proposes PRR network for better experience reuse in reinforcement learning.
problem Efficient experience reuse in reinforcement learning across multiple granularities.
method Proposes PRR network trained on multi-level architecture to extract and store experience.
result PRR network leads to better experience reuse and improved performance.
Improves reinforcement learning stability and efficiency.
problem Combining stability and efficiency in reinforcement learning.
method Combines on-policy stability with off-policy sample reuse.
result Demonstrates improved performance in both theory and practice.
MAML's success is due to feature reuse, not rapid learning.
problem Understanding the effectiveness of MAML in few-shot learning.
method Ablation studies and analysis of latent representations.
result Feature reuse is the dominant factor in MAML's success.
Similar models predict similarly, reducing overfitting risk.
problem Excessive reuse of test data in machine learning.
method Proved model similarity mitigates overfitting and provided a generalization bound.
result Model similarity reduces the risk of overfitting, even when accuracy levels suggest otherwise.
VRER selectively reuses samples to improve policy optimization in complex systems.
problem Lack of effective reuse of historical samples in reinforcement learning.
method Variance reduction based experience replay (VRER) framework.
result VRER accelerates policy optimization and enhances performance.
Improved scaling laws in linear regression using data reuse.
problem Sustainability of neural scaling laws when running out of new data.
method Data reuse in multi-pass stochastic gradient descent (multi-pass SGD) for M-dimensional linear models trained on N data with sketched features. result Multi-pass SGD achieves a test error of Θ(M1−b+L(1−b)/a) with L>N, improving scaling laws in data-constrained regimes. This paper proposes reusing CNN layers to speed up hyperparameters tuning.
problem Time-consuming hyperparameters tuning in CNNs.
method Reuse trained convolutional layers among different trainings.
result Reduces training time and increases accuracy of neural networks.
Paper proposes a method to reuse models without raw data.
problem Reuse existing models without accessing raw data.
method Two-phase framework: Upload and Deployment phases.
result Theoretical and experimental validation of approach effectiveness.
New method uses rank-conditioned Horvitz-Thompson estimation for unbiased sample reuse in Plackett-Luce best-of-K objective.
problem Estimating the expected maximum reward in Plackett-Luce draws without replacement.
method Rank-conditioned Horvitz-Thompson estimation with joint-score REINFORCE for unbiased sample reuse.
result Unbiased estimation of the Plackett-Luce best-of-K objective with finite second moment guarantees.
Method predicts rarity of image features to support research integrity investigations.
problem Difficulty in determining if image reuse is by chance or intentional.
method Statistical estimation of ORB features' chance occurrence across PubMed Open Access Subset dataset.
result The method produces decreasingly smaller p-values for more complex imagery, supporting null hypothesis.
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
Two-layer networks learn faster with batch reuse, overcoming information and leap exponents.
problem Limitations of gradient flow and single-pass GD in learning multi-index target functions.
method Multi-pass gradient descent that reuses batches, analyzed using Dynamical Mean-Field Theory.
result Two-time-step overlap with target subspace for non-staircase functions, overcoming information and leap exponents.
BOIL updates model body only, showing better few-shot learning performance.
problem Few-shot learning efficiency with model reuse vs. change.
method Proposes BOIL, updating only model body, freezing head.
result Significantly outperforms MAML on cross-domain tasks.
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
Paper proposes a classifier using fuzzy sets for data simplification.
problem Data simplification and handling uncertainty in real-world applications.
method Multi-resolution hierarchical granular representation (MRHGRC) using hyperbox fuzzy sets.
result High accuracy at low granularity with reduced data size.
Stabilizes policy optimization with off-policy data using divergence augmentation.
problem Premature convergence and instability in policy optimization with off-policy data.
method Incorporates Bregman divergence between behavior and current policies to ensure safe policy updates.
result Empirically shows better performance in data-scarce scenarios compared to other algorithms.
The paper investigates what enables successful transfer learning and separates feature reuse from data statistics.
problem Understanding what enables successful transfer learning and identifying the responsible parts of the network.
method Analyzes transfer learning on block-shuffled images to distinguish feature reuse from data statistics.
result Some benefit of transfer learning comes from learning low-level statistics of data, not just feature reuse.
A new method, Multi-Label Deep Forest, tackles multi-label learning problems.
problem Leveraging label correlations in multi-label learning models.
method Designs a deep forest framework with two mechanisms: measure-aware feature reuse and measure-aware layer growth.
result Outperforms compared methods on six measures across benchmark datasets.
VRER selectively reuses past observations to reduce variance in policy optimization.
problem Lack of effective experience replay for accelerating policy optimization in complex systems.
method Variance Reduction Experience Replay (VRER) framework that selectively reuses informative samples.
result VRER reduces gradient variance and improves policy learning over state-of-the-art algorithms.
Efficiently processes dynamic inputs in AI writing assistants with incremental computation.
problem Efficiently updating AI models in real-time with dynamic inputs.
method Incremental computing using vector quantization to filter and reuse intermediate values in neural networks.
result Comparable accuracy with 12.1X fewer operations for processing dynamic inputs.
Generalized Hindsight improves RL by reusing data from one task for another.
problem High sample complexity in reinforcement learning due to wasted uninformative data.
method Approximate inverse reinforcement learning to relabel behaviors with better-suited tasks.
result Efficient reuse of samples, reducing sample complexity on multi-task RL tasks.
Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we invest…
New method speeds up diffusion models without sacrificing quality.
problem Slow inference in diffusion models.
method Adams-Bashforth method for caching and acceleration.
result Achieved nearly 3x speedup with maintained quality.
Robotics learns new skills faster by reusing past movements.
problem Learning new motor skills is time-consuming and requires exploration of a large space of motor configurations.
method Combines probabilistic movement primitives with relative entropy policy search for skill initialization and adaptation.
result Quality of learned skills improves and the number of required iterations to learn a new task can be reduced by more than 60%.
This paper enhances ML algorithms by improving data locality and reducing redundancy.
problem Improving performance of machine learning algorithms with complex data.
method Exploiting data locality and reuse in memory hierarchies of modern processors.
result Efficient implementation of machine learning algorithms can be achieved by reusing computation results.
The paper proposes a method to adapt machine learning models to changing conditions.
problem Machine learning models need to adapt to new conditions in a constantly changing environment.
method Reuse knowledge from existing models to train future generations.
result The proposed method allows machine learning models to adapt and survive in a dynamic environment.
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
Paper learns a versatile model from diverse networks without annotations.
problem Combining knowledge from different specialized networks without access to their training data.
method Transforms features of diverse networks into a common space and forces a student model to mimic them.
result The student model outperforms individual teacher models on various benchmarks.
Study confirms eurozone interbank market stability but finds higher collateral reuse.
problem Analyzing eurozone interbank market behavior and stability.
method Examined secured transactions data from ECB, tested stylized facts, measured network properties.
result Observed higher collateral reuse and network symmetry compared to unsecured markets.
MSNet uses high frequency residual learning for efficient multi-scale image classification.
problem Efficient multi-scale image classification for mobile and embedded devices.
method Two network architecture: low resolution for low frequency, high resolution for high frequency residuals.
result MSNet achieves significant accuracy improvements over different base networks.
This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation. Simple rules are designed to detect slowly-varying gradients and, therefore, trigger the reuse of outdated gradients. The resultant gra…
An efficient learner is one who reuses what they already know to tackle a new problem. For a machine learner, this means understanding the similarities amongst datasets. In order to do this, one must take seriously the idea of working with datasets, rather than datapoints, as the key objects to model. Towards this goal…
The policy gradient approach is a flexible and powerful reinforcement learning method particularly for problems with continuous actions such as robot control. A common challenge in this scenario is how to reduce the variance of policy gradient estimates for reliable policy updates. In this paper, we combine the followi…
Transfer learning improves sparse, interpretable probabilistic classification.
problem Sparse and interpretable models in transfer learning.
method Two transfer learning extensions integrated into sparse and interpretable probabilistic classification vector machine.
result Transfer learning extensions improve sparsity and performance.
A new algorithm is proposed which accelerates the mini-batch k-means algorithm of Sculley (2010) by using the distance bounding approach of Elkan (2003). We argue that, when incorporating distance bounds into a mini-batch algorithm, already used data should preferentially be reused. To this end we propose using nested …
The paper develops a theory linking pretraining and fine-tuning in neural networks.
problem Understanding how initialization choices impact feature learning and generalization in neural networks.
method Analytical theory of diagonal linear networks, deriving generalization error as a function of initialization parameters and task statistics.
result Different initialization choices place networks into four fine-tuning regimes with varying abilities to support feature learning and generalization.