arXiv research

Two new algorithms improve Q* approximation in batch RL with linear error propagation.

To deal with very large datasets a mini-batch version of the Monte Carlo Markov Chain Stochastic Approximation Expectation-Maximization algorithm for general latent variable models is proposed. For exponential models the algorithm is shown to be convergent under classicalconditions as the number of iterations increases…

LF-IBIS learns optimal policies online without explicit likelihood.

Optimal batch size minimizes training time for neural networks.

Efficient momentum-based methods for reinforcement learning with improved sample complexity.

Enhanced DFO using adaptive batch-based FD estimates.

IWeS selects examples by entropy-based importance sampling for subset selection.

The study analyzes batched methods for early stopping in stochastic multi-armed bandits.

We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. More importantly, we incorporate model uncertainty into the sampling probability to com…

Overfitting & underfitting and stable training are an important challenges in machine learning. Current approaches for these issues are mixup, SamplePairing and BC learning. In our work, we state the hypothesis that mixing many images together can be more effective than just two. Batchboost pipeline has three stages: (…

DAIS improves AIS for differentiable marginal likelihood estimation.

A new algorithm is proposed which accelerates the mini-batch k-means algorithm of Sculley (2010) by using the distance bounding approach of Elkan (2003). We argue that, when incorporating distance bounds into a mini-batch algorithm, already used data should preferentially be reused. To this end we propose using nested …

SBA improves neural network generalization by dynamically augmenting data.

New algorithms minimize MMD to approximate probability measures efficiently.

Despite the increasing interest in multi-agent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch MARL with networked agents…

Adaptive SGD learns optimal batch size for strong convex functions.

Improved TD learning reduces batch sampling error.

Online method learns sparse models efficiently in large scale settings.

Framework improves gradient estimation for faster training convergence.

We revisit the stochastic variance-reduced policy gradient (SVRPG) method proposed by Papini et al. (2018) for reinforcement learning. We provide an improved convergence analysis of SVRPG and show that it can find an $ε$-approximate stationary point of the performance function within $O(1/ε^{5/3})$ trajectories. This s…

A well-known issue of Batch Normalization is its significantly reduced effectiveness in the case of small mini-batch sizes. When a mini-batch contains few examples, the statistics upon which the normalization is defined cannot be reliably estimated from it during a training iteration. To address this problem, we presen…

Adambs adapts Adam to prioritize important training examples.

This paper improves sample complexity for AC and NAC algorithms under Markovian sampling.

The accuracy of deep neural networks is significantly affected by how well mini-batches are constructed during the training step. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical lab…

Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the reduction operations. In this paper, we propose alleviating this problem through sampling only a small fraction of data for normalization at each i…

Data collection and labeling is one of the main challenges in employing machine learning algorithms in a variety of real-world applications with limited data. While active learning methods attempt to tackle this issue by labeling only the data samples that give high information, they generally suffer from large computa…

Unified framework for solving linear systems with improved convergence rates.

Sparse feature selection improves batch RL efficiency.

(Mini-batch) Stochastic Gradient Descent is a popular optimization method which has been applied to many machine learning applications. But a rather high variance introduced by the stochastic gradient in each step may slow down the convergence. In this paper, we propose the antithetic sampling strategy to reduce the va…

Adaptive batch sizes improve local gradient methods in distributed training.

We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence i…

Improves deep learning training by matching mini-batch distributions.

Improved generalization with iterate averaging and adaptive algorithms.

SGD's performance improves with critical batch size, minimizing SFO complexity.

New analysis shows SGD with noise doesn't leak more privacy with more iterations.

New method shows how order of gradient updates impacts stability and convergence in deep learning.

U-statistics improve gradient estimation in importance-weighted variational inference.

Paper develops Monte-Carlo estimators for CoVaR, a key risk measure.

Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.

We consider the transfer of experience samples (i.e., tuples < s, a, s', r >) in reinforcement learning (RL), collected from a set of source tasks to improve the learning process in a given target task. Most of the related approaches focus on selecting the most relevant source samples for solving the target task, but t…

The paper proposes an iterative approach to batch reinforcement learning for safer and more informative data collection.

Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …

This paper improves self-play learning in games by manipulating experience distributions.

Large-batch SGD is important for scaling training of deep neural networks. However, without fine-tuning hyperparameter schedules, the generalization of the model may be hampered. We propose to use batch augmentation: replicating instances of samples within the same batch with different data augmentations. Batch augment…

This paper introduces Tree-Pyramidal Adaptive Importance Sampling (TP-AIS), a novel iterated sampling method that outperforms state-of-the-art approaches like deterministic mixture population Monte Carlo (DM-PMC), mixture population Monte Carlo (M-PMC) and layered adaptive importance sampling (LAIS). TP-AIS iteratively…

This paper introduces a neural sampler for scalable sampling from complex distributions.

Full-batch GD achieves generalization close to any stationary point with fewer assumptions.

Study mini-batch SGD noise and its limits, proving complexity guarantees.