Less frequent retraining improves forecast accuracy in retail demand forecasting.
problem Balancing forecast accuracy and computational efficiency in global models.
method Analysis of ten machine learning and deep learning models across two large retail datasets with various retraining scenarios.
result Less frequent retraining strategies maintain forecast accuracy while reducing computational costs.
Deep model predicts Bitcoin price movements without retraining.
problem Stationary modelling of high-frequency Bitcoin price movements.
method Deep recurrent model based on order flow.
result Model maintains stability during volatile periods.
Proposes a method to retrain neural networks incrementally for continuous data flow.
problem Continuous data flow and the challenges of catastrophic forgetting and efficient retraining.
method Incremental retraining using multi-armed bandits to select important samples and weights, and a new regularization term for synapse and neuron importance.
result Mitigates catastrophic forgetting and boosts model performance.
Retraining with predicted labels improves model accuracy in noisy settings.
problem Improving model accuracy with noisy or corrupted labels.
method Retraining with predicted hard labels in a linearly separable binary classification setting.
result Retraining with predicted labels can increase model accuracy, as proven theoretically.
DeltaGrad rapidly retrain models with minimal data changes.
problem Rapid retraining of machine learning models with minimal data changes.
method DeltaGrad algorithm based on cached training information.
result DeltaGrad compares favorably to state-of-the-art methods.
A monitoring procedure improves machine learning forecasts for digital platforms.
problem Maintaining accurate and stable forecasts for data streams at digital platforms.
method Developed a monitoring procedure to determine when to retrain machine learning algorithms.
result Monitor-based retraining produces accurate forecasts compared to benchmarks.
This paper optimizes retraining models using their own predictions and noisy labels.
problem Improving model performance through optimal retraining of noisy labels.
method Developed a principled framework based on approximate message passing (AMP) to analyze iterative retraining procedures.
result Derivation of the Bayes optimal aggregator function to minimize prediction error.
Proposes a new method to unlearn from specific data points in conformal predictors.
problem Challenges of existing unlearning methods in conformal predictors.
method Formalizes conformal unlearning, introduces practical metrics, and presents an optimization algorithm.
result Demonstrates effective removal of targeted information while preserving utility.
MetaDVFS uses device and application metadata to improve DVFS efficiency.
problem Improving energy efficiency in mobile platforms with diverse applications and hardware.
method Formulates DVFS as a multi-task reinforcement learning problem and introduces MetaDVFS, leveraging metadata for knowledge transfer.
result MetaDVFS achieves up to 26% improvement in Quality of Experience and up to 17% improvement in Performance-Power Ratio.
K-priors enable quick adaptation with minimal retraining.
problem Machine learning models struggle to adapt to changes efficiently.
method Combines weight and function-space priors to reconstruct past gradients.
result Adaptation with K-priors achieves similar performance to full retraining with less data.
Two retraining techniques outperform fine-tuning in neural network pruning.
problem Improving accuracy and compression in neural network pruning.
method Weight rewinding and learning rate rewinding compared to fine-tuning.
result Rewinding techniques outperform fine-tuning in accuracy and compression.
Traditional classification algorithms assume that training and test data come from similar distributions. This assumption is violated in adversarial settings, where malicious actors modify instances to evade detection. A number of custom methods have been developed for both adversarial evasion attacks and robust learni…
Retraining stabilizes model influence on data.
problem Performativity in predictive models leads to feedback loops.
method Developed the stable signal principle to address retraining dynamics.
result Repeated risk minimization converges geometrically to stable signal direction.
This research shows loss weighting remains effective in last layer retraining despite model overparameterization.
problem Overcoming biases in machine learning models at scale.
method Theoretical and practical exploration of last layer retraining in an overparameterized setting.
result Loss weighting is still effective in last layer retraining, but weights must account for model overparameterization.
New method detects data distribution changes and retraining is advised.
problem Detecting when data distribution changes for retraining prediction algorithms.
method Exchangeable martingales and conformal prediction.
result Guaranteed validity of the method, with efficiency explored.
Paper tackles model collapse in synthetic data retraining.
problem Iterative retraining of generative models on synthetic data can lead to performance deterioration.
method Integrates an external synthetic data verifier to prevent model collapse.
result Synthetic retraining with a verifier can improve model performance initially but may lead to convergence to the verifier's knowledge center.
Data augmentation methods improve worst-case model performance.
problem Ensuring fair predictions across subpopulations in large models.
method Linear last layer retraining with data augmentation techniques.
result Optimal worst-group accuracy achieved for Gaussian latent representation distribution.
The most common method for DNN pruning is hard thresholding of network weights, followed by retraining to recover any lost accuracy. Recently developed smart pruning algorithms use the DNN response over the training set for a variety of cost functions to determine redundant network weights, leading to less accuracy deg…
Classifier learns to ignore unreliable feedback from end users.
problem Improving classifier performance by filtering unreliable feedback.
method Modeling end users as autonomous agents, periodically retraining classifier with filtered feedback.
result Classifier can identify and filter out unreliable feedback, improving performance.
Last layer retraining improves robustness to spurious correlations without high computational costs.
problem Neural networks can rely on spurious features like backgrounds for predictions.
method Simple last layer retraining on large models.
result Last layer retraining matches or outperforms state-of-the-art approaches on spurious correlation benchmarks.
Quantum computing speeds up neural network training and retraining.
problem Inefficient classical training and retraining of neural networks.
method Adiabatic quantum computing to optimize Kolmogorov-Arnold Networks using Bezier curves.
result Quantum optimization achieves 100x faster retraining compared to classical methods.
Recurrent neural networks (RNNs) are types of artificial neural networks (ANNs) that are well suited to forecasting and sequence classification. They have been applied extensively to forecasting univariate financial time series, however their application to high frequency trading has not been previously considered. Thi…
New method uses neural tangent kernel for efficient active learning.
problem Efficiently approximating deep learning's look-ahead selection criteria.
method Approximates retraining with neural tangent kernel for active learning.
result Approximation works asymptotically and enables sequential active learning.
The paper shows how curated synthetic data can optimize human preferences in generative models.
problem Contamination of web-scale datasets by synthetic data affects future model training.
method Theoretical study of iterated retraining of generative models with curated synthetic data.
result Data curation can be seen as an implicit preference optimization mechanism, maximizing expected reward.
Improved method for efficient black-box optimization in latent space.
problem Efficiently optimize expensive black-box functions over complex input spaces.
method Optimize in latent space of deep generative models, retrain model periodically and weight data points.
result Significantly improved efficiency and performance on synthetic and real-world problems.
Efficiently removes specific data subsets without retraining for GDPR compliance.
problem Efficiently removing specific data subsets to comply with GDPR regulations.
method Statistical framework for machine unlearning with minimax optimality for squared loss.
result Developed Unlearning Least Squares (ULS) achieving minimax optimality for estimating model parameters.
New method protects neural networks from adversarial attacks without generating adversarial examples.
problem Vulnerability of neural networks to adversarial examples.
method Entropic retraining, inspired by information theory.
result Significant increase in NNs' security and robustness.
Bayesian method refines surrogate models for accurate full waveform inversion.
problem Complex input/output relations in full waveform inversion make accurate surrogate models difficult.
method Iterative refinement of surrogate models using MCMC samples and progressively expanding frequency bandwidth.
result Highly accurate surrogate model across full bandwidth enables accurate final MCMC inversion.
Many active learning methods belong to the retraining-based approaches, which select one unlabeled instance, add it to the training set with its possible labels, retrain the classification model, and evaluate the criteria that we base our selection on. However, since the true label of the selected instance is unknown, …
The paper proposes a method to monitor deep learning predictions for retraining, reducing costs.
problem Reducing computational costs in deep learning by detecting when predictions are no longer valid.
method Sequential monitoring of network predictions based on projected second moments monitoring.
result The proposed method can drastically reduce computational costs in deep learning.
Study on self-consuming generative models with diverse human curation, focusing on convergence and stability.
problem Analyzing self-consuming generative models with heterogeneous human curation.
method Investigates the asymptotic behavior of retraining dynamics using nonlinear Perron--Frobenius theory and Banach contraction mapping.
result Improves convergence results and provides stability and non-stability analyses for the model.
When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far been neglected in supervised learning. When ignored, performativity surfaces as…
Wide and Deep GNN learns from distributed graphs and retrain online.
problem Decentralized graph support changes over time, causing mismatch between training and testing graphs.
method Wide and Deep GNN architecture with distributed online learning.
result Convergence guarantees for online retraining of the wide part of the GNN.
Dynamic pruning during training reduces deep network complexity without significant accuracy loss.
problem High memory and computational requirements of deep networks during training and inference.
method Dynamic pruning of convolutional filters during training, using L1 normalization for optimization.
result L1 normalization-based pruning yields up to 50% reduction in filters with minimal accuracy loss.
Improved heart rate and activity recognition with low-power wrist sensors.
problem Challenges in battery life, cost, and sensor performance in wrist-worn sensing applications.
method Used photoplethysmography (PPG) for heart rate and activity recognition, applying transfer learning and CNNs.
result Low sampling frequencies (5 Hz and 10 Hz) achieved good performance in heart rate and activity recognition.
Active learning (AL) concerns itself with learning a model from as few labelled data as possible through actively and iteratively querying an oracle with selected unlabelled samples. In this paper, we focus on analyzing a popular type of AL in which the utility of a sample is measured by a specified goal achieved by th…
Improved time series forecasting with expert loss integration.
problem Enhancing time series forecasting accuracy and efficiency.
method Adaptive Mixture-of-Experts framework with expert-specific loss integration and online learning.
result Significantly improved forecasting accuracy and computational efficiency.
Residual Networks with convolutional layers are widely used in the field of machine learning. Since they effectively extract features from input data by stacking multiple layers, they can achieve high accuracy in many applications. However, the stacking of many layers raises their computation costs. To address this pro…
Timber targets decision trees, outperforming existing attacks.
problem Poisoning decision trees to manipulate model predictions.
method Greedy attack strategy using sub-tree retraining for efficiency.
result Timber outperforms existing attacks in effectiveness and efficiency.
Conformal prediction fails under severe feature turnover in COVID-19 supply chain tasks.
problem Dealing with distribution shift in conformal prediction models.
method Using COVID-19 as a natural experiment across 8 supply chain tasks, analyzing SHAP explanations.
result Coverage drops vary widely (0% to 86.7%) and correlate with single-feature dependence.
PARIS reduces imbalanced regression datasets by pruning uninformative samples.
problem Imbalanced regression where models focus on high-frequency regions, ignoring rare but impactful events.
method PARIS uses the representer theorem to compute a closed-form representer deletion residual for iterative pruning of the training set.
result PARIS reduces training set by up to 75% while preserving or improving overall performance, outperforming other methods.
Gradient ascent method successfully removes specific data points from neural networks without retraining.
problem Addressing privacy and ethical concerns by removing specific data points from trained models.
method Gradient ascent approach to unlearning, leveraging the implicit bias of gradient descent towards margin maximization conditions.
result Gradient ascent method can successfully unlearn specific data points from two-layer ReLU neural networks without retraining.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.
Algorithm improves model performance on shifted concepts without retraining.
problem Improving model performance on shifted concepts with limited source data.
method Model consolidation of intermediate internal distributions after adaptation.
result Effective improvement in model performance on shifted concepts.
Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage re…
VerifAI toolkit improves neural network-based aircraft taxiing system safety.
problem Improving safety of autonomous aircraft taxiing systems using neural networks.
method Unified approach to formal analysis and retraining of AI systems, including falsification, debugging, and retraining.
result Improved neural network performance and reduced failure cases in aircraft taxiing system.
HyFAD improves time series imputation by combining time and frequency diffusion.
problem Improve time series imputation by handling frequency-sensitive denoising and balancing global and local dynamics.
method HyFAD is a hybrid time-frequency diffusion model with frequency-aware embedding, built on DDPM paradigm.
result HyFAD achieves state-of-the-art performance in time series imputation.
New method quantifies variable importance in causal forests for treatment effect heterogeneity.
problem Lack of understanding how input variables affect treatment effect heterogeneity in causal forests.
method Developed a new importance variable algorithm for causal forests based on the drop and relearn principle.
result Shows how to handle forest retraining without a confounding variable and introduces a corrective term for confounders.