We analyze how batch learning impacts bandit problems.
problem Impact of batch learning in stochastic bandits.
method Policy-agnostic regret analysis, upper and lower bounds demonstration.
result The impact of batch learning can be measured in terms of online behavior.
Adaptive batch sizes improve active learning efficiency and flexibility.
problem Fixed batch sizes in active learning are inefficient due to dynamic cost-speed trade-offs.
method Probabilistic Numerics framework that adaptively changes batch sizes based on integration error and precision objectives.
result Significant enhancement in learning efficiency and flexibility across various applications.
BatchGFN uses generative flow networks for efficient batch active learning.
problem Efficiently selecting informative batches for active learning.
method Generative flow networks to sample batches proportional to a batch reward.
result Constructs highly informative batches with a single forward pass per point.
The paper tackles fast rates in batch active learning with pool-based data.
problem Reduced adaptivity in batch active learning leads to suboptimal results.
method Proposes a stage-wise greedy algorithm that balances informativeness and diversity.
result The algorithm's excess risk matches minimax rates in standard statistical learning settings.
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
When using active learning, smaller batch sizes are typically more efficient from a learning efficiency perspective. However, in practice due to speed and human annotator considerations, the use of larger batch sizes is necessary. While past work has shown that larger batch sizes decrease learning efficiency from a lea…
Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.
problem Optimizing training efficiency for large language models with adaptive optimizers.
method Develops a principled framework for batch-size scheduling, introducing Seesaw which multiplies learning rate by 1/√2 and doubles batch size.
result Empirically, Seesaw reduces wall-clock time by approximately 36% compared to cosine decay, matching theoretical limits.
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
problem Diminishing returns with large batch sizes in RL due to non-stationary data.
method Adaptive Batch Scaling (ABS) with Behavioral Divergence metric.
result Larger batch sizes can improve RL performance, contrary to conventional wisdom.
TaskNorm improves meta-learning performance by rethinking batch normalization.
problem Challenges in batch normalization for meta-learning with deep networks.
method Developed TaskNorm, a novel approach to batch normalization for meta-learning.
result TaskNorm consistently improves meta-learning performance across various datasets and meta-learning approaches.
New acquisition function improves batch Bayesian active learning.
problem BatchBALD conflates epistemic and aleatoric uncertainty, leading to suboptimal performance.
method Focus on predictive probabilities to separate epistemic uncertainty, leading to better performance and faster evaluation.
result The new acquisition function performs better and allows for larger batches.
Paper explains contrastive learning using cosine similarity and proposes mitigations for batch size effects.
problem Understanding and improving contrastive learning through batch size effects.
method Unified framework of cosine similarity, theoretical insights, and auxiliary loss.
result Performance improvement in small-batch settings through proposed auxiliary loss.
Study batch learning in linear bandits with context, achieving near-optimal performance.
problem Sequential batch learning in linear contextual bandits with finite actions.
method Established regret bounds and provided algorithms for two settings: arbitrary contexts and i.i.d. contexts.
result Regret upper bound nearly matches lower bound, showing polynomial and logarithmic batch requirements.
Mini-batch stochastic gradient descent and variants thereof have become standard for large-scale empirical risk minimization like the training of neural networks. These methods are usually used with a constant batch size chosen by simple empirical inspection. The batch size significantly influences the behavior of the …
SGD's performance improves with critical batch size, minimizing SFO complexity.
problem Optimizing SGD's performance with batch size and learning rate.
method Analysis of SGD using constant and decaying learning rates, focusing on batch size effects.
result SGD with critical batch size minimizes SFO complexity.
Empirical study on SGD hyperparameters and adversarial robustness.
problem Effect of SGD hyperparameters on adversarial robustness and generalization.
method Empirical observation of learning rate, batch size, and momentum effects on adversarial robustness and generalization.
result Constant learning rate to batch size ratio leads to good generalization and almost constant adversarial robustness.
Extends batch active learning to non-differentiable models.
problem Efficiently training machine learning models on large, initially unlabelled datasets.
method Black-box batch active learning for regression tasks that relies solely on model predictions.
result Achieves strong performance on regression datasets compared to white-box approaches for deep learning models.
Batch Active Learning uses derivative information for Gaussian Process regression.
problem Efficiently selecting data batches in Gaussian Process regression models.
method Proposes using the predictive covariance matrix to select data batches, exploiting full correlation.
result Demonstrates the effectiveness of incorporating derivative information across diverse applications.
Training deep neural networks with Stochastic Gradient Descent, or its variants, requires careful choice of both learning rate and batch size. While smaller batch sizes generally converge in fewer training epochs, larger batch sizes offer more parallelism and hence better computational efficiency. We have developed a n…
AdAdaGrad optimizes batch sizes for deep learning models, reducing the generalization gap.
problem The generalization gap between large-batch and small-batch training in deep learning.
method AdAdaGrad introduces adaptive batch size strategies derived from adaptive sampling methods.
result AdAdaGradNorm converges to a first-order stationary point with a rate of O(1/K) in K iterations.
Adaptive SGD learns optimal batch size for strong convex functions.
problem Finding optimal batch size for SGD in practice.
method Adaptive SGD method that learns optimal batch size.
result Adaptive SGD exhibits nearly optimal performance in experiments.
AdaScale SGD adapts learning rates for large-batch training efficiently.
problem Adapting learning rates for large-batch training to balance speed-ups and model quality.
method Adaptive learning rate adaptation based on gradient variance.
result AdaScale achieves reliable speed-ups for a wide range of batch sizes without degrading model quality.
New method for robust learning from batches, even adversarial ones.
problem Learning from batches that may be corrupt or adversarial.
method General framework for robust learning, derived from optimal robust algorithms.
result First robust agnostic learning algorithms for various distributions.
A simple method improves batch active learning without high compute.
problem Efficient batch active learning in machine learning.
method Adapting standard single-point acquisition strategies to batch.
result Simple strategy performs as well as advanced batch methods.
BYOL learns useful representations without batch statistics.
problem BYOL's reliance on batch statistics for representation learning.
method Training BYOL without batch normalization and using alternative normalization schemes.
result BYOL performance comparable to vanilla BYOL without batch normalization.
Riemannian stochastic gradient descent converges faster with increasing batch size.
problem Improving convergence rate of Riemannian stochastic gradient descent.
method Theoretical analysis and numerical investigation of increasing batch size effects.
result Riemannian stochastic gradient descent converges faster with increasing batch size.
Improved TD learning reduces batch sampling error.
problem TD learning can be inaccurate in batch settings due to action weighting.
method Introduced PSEC-TD(0) to correct action weighting using importance sampling.
result PSEC-TD(0) produces more accurate value function estimates.
We design a new algorithm for batch active learning with deep neural network models. Our algorithm, Batch Active learning by Diverse Gradient Embeddings (BADGE), samples groups of points that are disparate and high-magnitude when represented in a hallucinated gradient space, a strategy designed to incorporate both pred…
A new method SEBS optimizes SGD batch size for better performance.
problem Optimizing batch size for SGD to balance training speed and generalization.
method SEBS method uses a multi-stage geometric batch size enlargement scheme.
result SEBS reduces parameter updates without increasing generalization error.
SNGM improves large-batch training accuracy.
problem Improving generalization in large-batch training.
method Stochastic Normalized Gradient Descent with Momentum.
result SNGM achieves better test accuracy than MSGD and other large-batch methods.
This paper explores the limits of large batch sizes in deep learning.
problem Understanding the optimal batch size for deep learning models.
method Detailed numerical optimization and experimental analysis of large batch sizes.
result Improvement of 18% top-1 test accuracy with an optimized batch size recipe.
ICAL improves deep learning model accuracy and NLL with optimized batch labeling.
problem Deep Bayesian Active Learning for efficient model training.
method ICAL uses HSIC to measure dependency and optimizes batch size scaling.
result Significant improvements in model accuracy and NLL on image datasets.
Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational parallelism, small batch training has been shown to provide improved generalization performance and allows a significantly smaller memory footp…
There has recently been a surge in research in batch Deep Reinforcement Learning (DRL), which aims for learning a high-performing policy from a given dataset without additional interactions with the environment. We propose a new algorithm, Best-Action Imitation Learning (BAIL), which strives for both simplicity and per…
ADS filters data points for efficient batch active learning.
problem Efficiently selecting data points for annotation in parallel settings.
method Active Data Shapley (ADS) using the Shapley value of data.
result Significantly increases efficiency of active learning by 6x.
Batch normalization biases linear models towards uniform margins, improving performance in binary classification.
problem Understanding the implicit bias of batch normalization in linear models and neural networks.
method Analyzing gradient descent convergence on linear models and two-layer CNNs with batch normalization.
result Gradient descent with batch normalization in linear models converges to a uniform margin classifier with an exponential convergence rate.
A new method improves active learning for large batch sizes.
problem Challenges in scaling Bayesian active learning to large batch sizes.
method Derives Partial Batch Label Sampling (ParBaLS) for EPIG algorithm.
result ParBaLS EPIG outperforms top-B selection and BatchBALD. Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment. Following this result, there have been several papers showing reasonable performances under a variety of environments and batch settings. In this pape…
Determining the appropriate batch size for mini-batch gradient descent is always time consuming as it often relies on grid search. This paper considers a resizable mini-batch gradient descent (RMGD) algorithm based on a multi-armed bandit for achieving best performance in grid search by selecting an appropriate batch s…
The paper proposes an iterative approach to batch reinforcement learning for safer and more informative data collection.
problem Learning policies that are too rigid and do not adapt to new data.
method Safe diversified model-based policy search in an iterative batch reinforcement learning framework.
result Improved learned policies through continuous data collection and adaptation.
Many applications, including natural language processing, sensor networks, collaborative filtering, and federated learning, call for estimating discrete distributions from data collected in batches, some of which may be untrustworthy, erroneous, faulty, or even adversarial. Previous estimators for this setting ran in e…
SGD batch size affects autoencoder global minima sparsity and sharpness.
problem Investigating how batch size impacts autoencoder learning.
method Non-convex autoencoder training with SGD, varying batch sizes.
result SGD batch size influences global minimum sparsity and sharpness.
Adaptive batching improves Gaussian process surrogates for noisy level set estimation.
problem Learning the level set of noisy simulator responses.
method Developed four novel adaptive batching schemes for Gaussian process metamodels.
result Adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.
The paper improves Gaussian process models for efficient batch optimization.
problem Poor scaling and optimization loop issues in Gaussian process models.
method Dual GP parameterization for linear scaling and non-Gaussian likelihood updates.
result Extends sparse models to greedy batch fantasizing acquisition functions.
This paper unifies three regularization methods in batch reinforcement learning.
problem Learning overly-complex models in batch reinforcement learning.
method Unified weighted average transition matrix framework for three regularization methods.
result Empirical evaluation confirms intuitions about regularization methods' performance.
The paper studies batch decompositions of random datasets with probabilistic similarity constraints.
problem Understanding how to optimally split large datasets into batches for better model learning.
method Assumes independent data points from a space, defines similarity, and uses probabilistic and martingale methods to find bounds on batch sizes.
result Demonstrates an inherent tradeoff between relaxing similarity constraints and batch size, and provides bounds for maximum similarity subsets.
In an increasing number of domains it has been demonstrated that deep learning models can be trained using relatively large batch sizes without sacrificing data efficiency. However the limits of this massive data parallelism seem to differ from domain to domain, ranging from batches of tens of thousands in ImageNet to …
A new algorithm learns policies from batch data in hierarchical RL.
problem Learning policies from fixed batches of data without full exploration.
method Modeling RL as a two-player game with a leader-follower structure, proposing StackelbergLearner.
result StackelbergLearner achieves competitive performance in batch RL and real-world datasets.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.