POLO simplifies parallel optimization research with a flexible library.
problem Ease of implementing and evaluating parallel optimization algorithms.
method Uses policies and template programming for algorithm decomposition and code reuse.
result Allows implementation of state-of-the-art algorithms in minimal code.
Convolutional neural network summarizes code comments across multiple languages.
problem Insufficient or missing comments in source code.
method Language-agnostic encoder-decoder model with open vocabulary.
result Comparable results to state-of-the-art on single-language data; first results on multi-language data.
New method uses rank-conditioned Horvitz-Thompson estimation for unbiased sample reuse in Plackett-Luce best-of-K objective.
problem Estimating the expected maximum reward in Plackett-Luce draws without replacement.
method Rank-conditioned Horvitz-Thompson estimation with joint-score REINFORCE for unbiased sample reuse.
result Unbiased estimation of the Plackett-Luce best-of-K objective with finite second moment guarantees.
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
Generalized Hindsight improves RL by reusing data from one task for another.
problem High sample complexity in reinforcement learning due to wasted uninformative data.
method Approximate inverse reinforcement learning to relabel behaviors with better-suited tasks.
result Efficient reuse of samples, reducing sample complexity on multi-task RL tasks.
Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions. For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as question…
Study on reliability of latent reuse in diffusion models under distribution shift.
problem When can latent spaces from a source dataset be reused for a target dataset with different distributions?
method Considered a source-target setting with approximately low-dimensional datasets near different subspaces. Analyzed the target-domain score error due to principal-angle misalignment and target ambient noise.
result Latent reuse is reliable only if the source and target subspaces are close and the target ambient noise is not too amplified.
Enhances DenseNets with multi-scale convolutions and stochastic feature reuse.
problem Overfitting in DenseNets with dense feature reuse.
method Multi-scale Convolution Aggregation module and Stochastic Feature Reuse.
result Significant improvement in model accuracy with fewer parameters.
Pipeline-aware hyperparameter tuning speeds up machine learning pipelines by reusing intermediate computations.
problem High computational burden in hyperparameter tuning of multi-stage pipelines.
method Proposes a hybrid hyperparameter tuning method and a caching problem formulated as an ILP to maximize reuse.
result Pipeline-aware approach offers over an order-of-magnitude speedup over independent evaluations.
The simulator is an R package that streamlines the process of performing simulations by creating a common infrastructure that can be easily used and reused across projects. Methodological statisticians routinely write simulations to compare their methods to preexisting ones. While developing ideas, there is a temptatio…
Ludwig simplifies deep learning for non-experts.
problem Making deep learning accessible to non-experts.
method Type-based data abstraction and declarative configuration files.
result Ludwig democratizes deep learning, making it accessible to a broader audience.
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
Verified numerics prove existence of a curvature solution with known symmetries.
problem Existence of a curvature solution for the Nirenberg problem.
method Verified numerics and computer assistance.
result Existence of a genuine solution with known symmetry groups.
We propose several ways of reusing subword embeddings and other weights in subword-aware neural language models. The proposed techniques do not benefit a competitive character-aware model, but some of them improve the performance of syllable- and morpheme-aware models while showing significant reductions in model sizes…
New RL algorithms improve control tasks with data reuse.
problem Real-world control requires performance guarantees and data efficiency.
method Generalized Policy Improvement combining on-policy guarantees and sample reuse.
result Extensive experimental analysis shows benefits of new algorithms.
ECPF improves classification accuracy and speed for evolving data streams.
problem Reusing classifiers trained on recurring concepts to maintain accuracy and speed.
method ECPF uses similarity of classifications to quickly identify the best classifier to reuse.
result ECPF significantly outperforms state-of-the-art frameworks on synthetic and real-world datasets.
Simplifies efficient estimation via automatic differentiation and probabilistic programming.
problem Constructing efficient estimators for complex statistical models.
method Automatic differentiation applied to statistical functionals, avoiding the need to derive efficient influence functions.
result Users can generate efficient estimators with minimal code, simplifying the process for non-experts.
Bayesian Experience Reuse improves learning from multiple experts.
problem Learning from multiple experts with conflicting goals.
method Bayesian neural networks with shared features to model uncertainty and derive a probability distribution over expert models.
result BERS method effectively samples demonstrations from the derived distribution to reuse them in new tasks.
QuantaAlpha uses evolutionary algorithms to mine financial alpha robustly across market distributions.
problem Challenges in alpha mining due to market noise and regime shifts.
method Evolutionary framework treating each mining run as a trajectory, mutation, crossover, targeted revision, and reuse of effective patterns.
result Consistent gains over strong baselines and prior systems, achieving high IC and ARR.
Deep learning compresses L-values for QAM symbols, reducing memory and improving performance.
problem Efficiently storing log-likelihood ratios (L-values) for QAM modulation.
method A deep autoencoder that jointly compresses and reconstructs L-values with a weighted loss function.
result Reduces memory footprint by up to two times with less than 0.1 dB performance loss.
TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection
problem Kernel learning and model selection
method GPU acceleration
result Competitive predictive performance with speedups
Averaging recent model checkpoints speeds up training time.
problem Training large vision or language models is time-consuming.
method Average the weights of the k latest checkpoints.
result Speeds up training by dozens of epochs, saving up to 68 GPU hours.
In many real-world applications, data are often collected in the form of stream, and thus the distribution usually changes in nature, which is referred as concept drift in literature. We propose a novel and effective approach to handle concept drift via model reuse, leveraging previous knowledge by reusing models. Each…
Paper proposes PRR network for better experience reuse in reinforcement learning.
problem Efficient experience reuse in reinforcement learning across multiple granularities.
method Proposes PRR network trained on multi-level architecture to extract and store experience.
result PRR network leads to better experience reuse and improved performance.
Improves reinforcement learning stability and efficiency.
problem Combining stability and efficiency in reinforcement learning.
method Combines on-policy stability with off-policy sample reuse.
result Demonstrates improved performance in both theory and practice.
MAML's success is due to feature reuse, not rapid learning.
problem Understanding the effectiveness of MAML in few-shot learning.
method Ablation studies and analysis of latent representations.
result Feature reuse is the dominant factor in MAML's success.
Similar models predict similarly, reducing overfitting risk.
problem Excessive reuse of test data in machine learning.
method Proved model similarity mitigates overfitting and provided a generalization bound.
result Model similarity reduces the risk of overfitting, even when accuracy levels suggest otherwise.
VRER selectively reuses samples to improve policy optimization in complex systems.
problem Lack of effective reuse of historical samples in reinforcement learning.
method Variance reduction based experience replay (VRER) framework.
result VRER accelerates policy optimization and enhances performance.
Improved scaling laws in linear regression using data reuse.
problem Sustainability of neural scaling laws when running out of new data.
method Data reuse in multi-pass stochastic gradient descent (multi-pass SGD) for M-dimensional linear models trained on N data with sketched features. result Multi-pass SGD achieves a test error of Θ(M1−b+L(1−b)/a) with L>N, improving scaling laws in data-constrained regimes. New bounds show multiclass problems reduce overfitting from test set reuse.
problem Overfitting from test set reuse in multiclass problems.
method Upper and lower bounds on bias of attacks, practical and inefficient attacks.
result Multiclass problems mitigate overfitting from test set reuse.
This paper proposes reusing CNN layers to speed up hyperparameters tuning.
problem Time-consuming hyperparameters tuning in CNNs.
method Reuse trained convolutional layers among different trainings.
result Reduces training time and increases accuracy of neural networks.
Paper proposes a method to reuse models without raw data.
problem Reuse existing models without accessing raw data.
method Two-phase framework: Upload and Deployment phases.
result Theoretical and experimental validation of approach effectiveness.
New algorithm learns abstractions from experience to simplify complex tasks.
problem Solving complex problems in environments with continuous state spaces.
method Uses MDP homomorphisms to find abstract MDPs and guides exploration.
result Demonstrates task transfer method outperforms deep Q-networks.
Method predicts rarity of image features to support research integrity investigations.
problem Difficulty in determining if image reuse is by chance or intentional.
method Statistical estimation of ORB features' chance occurrence across PubMed Open Access Subset dataset.
result The method produces decreasingly smaller p-values for more complex imagery, supporting null hypothesis.
Paper explores reusing and adapting training data for entity resolution.
problem Entity resolution with limited training data.
method Distributed representation and five algorithms for reuse scenarios.
result Significant performance improvements with reused training data.
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
Two-layer networks learn faster with batch reuse, overcoming information and leap exponents.
problem Limitations of gradient flow and single-pass GD in learning multi-index target functions.
method Multi-pass gradient descent that reuses batches, analyzed using Dynamical Mean-Field Theory.
result Two-time-step overlap with target subspace for non-staircase functions, overcoming information and leap exponents.
Multivariate analysis of fMRI data has benefited substantially from advances in machine learning. Most recently, a range of probabilistic latent variable models applied to fMRI data have been successful in a variety of tasks, including identifying similarity patterns in neural data (Representational Similarity Analysis…
Transfer learning improves image classifier performance in data-starved regimes.
problem Deploying image classifiers in domains with limited labeled data.
method Transfer learning with deep neural networks, focusing on feature reuse and overparameterization.
result Transfer learning enhances CNN performance in data-starved regimes.
Stabilizes policy optimization with off-policy data using divergence augmentation.
problem Premature convergence and instability in policy optimization with off-policy data.
method Incorporates Bregman divergence between behavior and current policies to ensure safe policy updates.
result Empirically shows better performance in data-scarce scenarios compared to other algorithms.
The paper investigates what enables successful transfer learning and separates feature reuse from data statistics.
problem Understanding what enables successful transfer learning and identifying the responsible parts of the network.
method Analyzes transfer learning on block-shuffled images to distinguish feature reuse from data statistics.
result Some benefit of transfer learning comes from learning low-level statistics of data, not just feature reuse.
A new method, Multi-Label Deep Forest, tackles multi-label learning problems.
problem Leveraging label correlations in multi-label learning models.
method Designs a deep forest framework with two mechanisms: measure-aware feature reuse and measure-aware layer growth.
result Outperforms compared methods on six measures across benchmark datasets.
E2GC optimizes energy efficiency in DNNs by balancing computational and data movement costs.
problem Imbalance between computational complexity and data reuse in GConv leads to suboptimal energy efficiency.
method Developed an optimum group size model and proposed E2GC module with constant group size.
result E2GC modules improve energy efficiency by 10.8% and 4.73% on P100 and P4000 GPUs, respectively.
SlicStan improves Stan's usability and efficiency.
problem Stan's block syntax sacrifices usability for scalability.
method Formalized Stan, introduced SlicStan with compositional syntax and flexible functions.
result SlicStan facilitates better code reuse and abstraction.
Transfer learning offers little benefit in medical imaging tasks.
problem Understanding transfer learning's impact on medical imaging tasks.
method Evaluation of transfer learning on two large-scale medical imaging tasks.
result Simple, lightweight models perform comparably to ImageNet architectures.
VRER selectively reuses past observations to reduce variance in policy optimization.
problem Lack of effective experience replay for accelerating policy optimization in complex systems.
method Variance Reduction Experience Replay (VRER) framework that selectively reuses informative samples.
result VRER reduces gradient variance and improves policy learning over state-of-the-art algorithms.
ARMIN improves memory efficiency and lightness in neural networks.
problem Complex memory addressing and inefficient memory information exploitation in previous MANNs.
method ARMIN uses hidden state for automatic memory addressing and a novel RNN cell for memory integration.
result ARMIN achieves lower computational overhead and similar performances compared to vanilla LSTM.
Efficiently processes dynamic inputs in AI writing assistants with incremental computation.
problem Efficiently updating AI models in real-time with dynamic inputs.
method Incremental computing using vector quantization to filter and reuse intermediate values in neural networks.
result Comparable accuracy with 12.1X fewer operations for processing dynamic inputs.