arXiv research

The paper extends consistency results for sequential design strategies to vector-valued Gaussian processes.

The paper develops sampling methods for ocean phenomena based on temperature and salinity measurements.

Develops Thompson Sampling algorithms for mean-variance bandits.

Optimizes ranking from click feedback in a bandit setting.

The paper analyzes how to combine self-protection and self-insurance for risk reduction.

Data balancing reduces variance in machine learning models.

Proposes LBD for more robust deep learning models.

Neural SDEs reduce variance in stochastic simulations.

MARS optimizes large model training by reducing variance, outperforming AdamW.

Normal distributions ensure asymptotic variance reduction in moment matching Monte Carlo.

In this paper we suggest a modification of the regression-based variance reduction approach recently proposed in Belomestny et al. This modification is based on the stratification technique and allows for a further significant variance reduction. The performance of the proposed approach is illustrated by several numeri…

New algorithm reduces optimization complexity in adaptive mirror descent.

SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.

Optimizes MCMC chains with neural control variates.

Paper proposes a method to estimate variance reduction in DNN training using importance sampling.

Optimizes variance reduction in Heston model using large and moderate deviations.

In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce th…

Unified framework for structure learning via conditional independence testing.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

New optimizer MARS-M combines variance reduction with Muon for faster LLM training.

Develops a new SPP algorithm with variance reduction for weakly convex optimization.

Improved time complexity for parallel stochastic optimization in heterogeneous systems.

Paper develops momentum schemes with variance reduction for non-convex composition optimization.

This paper develops a new theory for ensemble learning beyond variance reduction.

Paper improves tree probability estimation using stochastic optimization and variance reduction.

New algorithms reduce variance in solving complex mathematical problems.

Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.

We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved incl…

Improved TD learning reduces variance and bias errors.

RMDA trains structured neural networks with regularization and variance reduction.

A new gradient estimator reduces variance near boundaries for binary latent variables.

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high vari…

New sparsity operator reduces variance reduction methods' computational cost.

A new method using energy distance for ensemble and scenario reduction.

Variance reduction has emerged in recent years as a strong competitor to stochastic gradient descent in non-convex problems, providing the first algorithms to improve upon the converge rate of stochastic gradient descent for finding first-order critical points. However, variance reduction techniques typically require c…

Paper improves efficiency in matrix computations for Gaussian processes.

In this paper, we are concerned with obtaining distribution-free concentration inequalities for mixture of independent Bernoulli variables that incorporate a notion of variance. Missing mass is the total probability mass associated to the outcomes that have not been seen in a given sample which is an important quantity…

Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for example by data augmentation. In such cases, the objective is no longer a finite su…

Characterizes symmetric Bernoulli distributions with minimal convex sums.

Survey of SDR methods for high-dimensional regression and embedding.

Improves gradient estimation for discrete distributions with variance reduction techniques.

U-statistics improve gradient estimation in importance-weighted variational inference.

We study optimization algorithms based on variance reduction for stochastic gradient descent (SGD). Remarkable recent progress has been made in this direction through development of algorithms like SAG, SVRG, SAGA. These algorithms have been shown to outperform SGD, both theoretically and empirically. However, asynchro…

Ensembles improve classifier performance by reducing bias, not variance.

The key idea of Bayesian optimization is replacing an expensive target function with a cheap surrogate model. By selection of an acquisition function for Bayesian optimization, we trade off between exploration and exploitation. The acquisition function typically depends on the mean and the variance of the surrogate mod…

New method reduces variance in complex probabilistic model optimization.

New algorithm reduces variance in Monte Carlo simulations using deep neural networks and policy gradients.

We show that on-policy policy gradient (PG) and its variance reduction variants can be derived by taking finite difference of function evaluations supplied by estimators from the importance sampling (IS) family for off-policy evaluation (OPE). Starting from the doubly robust (DR) estimator (Jiang & Li, 2016), we provid…