arXiv research

New sparsity operator reduces variance reduction methods' computational cost.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

Improves gradient estimation for discrete distributions with variance reduction techniques.

New method improves scalability of SGD for large datasets.

TrIM improves gradient-based dimension reduction and regression.

Identifies a gradient flow to solve kernel learning problems with noise reduction.

Extends dimension reduction to data-driven settings without gradients.

Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…

We generalize stochastic smoothing for gradient estimation of non-differentiable functions.

Evolution Strategies (ES) are a powerful class of blackbox optimization techniques that recently became a competitive alternative to state-of-the-art policy gradient (PG) algorithms for reinforcement learning (RL). We propose a new method for improving accuracy of the ES algorithms, that as opposed to recent approaches…

Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.

Unified framework for decentralized optimization combining gradient tracking and variance reduction.

Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high vari…

FPG uses fractional calculus for efficient reinforcement learning with long-term memory.

Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.

MARS optimizes large model training by reducing variance, outperforming AdamW.

U-statistics improve gradient estimation in importance-weighted variational inference.

New method improves online covariance estimation for SGD.

Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the "reparameterization trick," represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimator…

Improves MARS for nonparametric multivariate regression with dimension reduction.

We show a connection between the Fourier spectrum of Boolean functions and the REINFORCE gradient estimator for binary latent variable models. We show that REINFORCE estimates (up to a factor) the degree-1 Fourier coefficients of a Boolean function. Using this connection we offer a new perspective on variance reduction…

New algorithm reduces variance in Monte Carlo simulations using deep neural networks and policy gradients.

In the paper, we study the stochastic alternating direction method of multipliers (ADMM) for the nonconvex optimizations, and propose three classes of the nonconvex stochastic ADMM with variance reduction, based on different reduced variance stochastic gradients. Specifically, the first class called the nonconvex stoch…

We show that on-policy policy gradient (PG) and its variance reduction variants can be derived by taking finite difference of function evaluations supplied by estimators from the importance sampling (IS) family for off-policy evaluation (OPE). Starting from the doubly robust (DR) estimator (Jiang & Li, 2016), we provid…

Variance reduction has emerged in recent years as a strong competitor to stochastic gradient descent in non-convex problems, providing the first algorithms to improve upon the converge rate of stochastic gradient descent for finding first-order critical points. However, variance reduction techniques typically require c…

A new algorithm reduces bias and variance in distributionally robust optimization.

Paper introduces a new multi-kernel algorithm for better gradient approximation.

AdaSVRG combines adaptive gradient with SVRG for robust optimization.

Stochastic gradient Markov Chain Monte Carlo (SG-MCMC) has been developed as a flexible family of scalable Bayesian sampling algorithms. However, there has been little theoretical analysis of the impact of minibatch size to the algorithm's convergence rate. In this paper, we prove that under a limited computational bud…

A new algorithm SRG-DQN reduces variance in deep Q-learning.

SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.

New neural network method simplifies high-dimensional data.

This paper proposes a novel kernel approach to linear dimension reduction for supervised learning. The purpose of the dimension reduction is to find directions in the input space to explain the output as effectively as possible. The proposed method uses an estimator for the gradient of regression function, based on the…

Study on semistable points and convexity of gradient maps for group actions.

A neural network approach for feature selection using mutual information.

New deep network derived from rate reduction principles, explaining features and efficiency.

Paper improves Gumbel-Softmax estimator variance reduction.

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance reduction and we prove its linear convergence with the Fletcher and Reeves method for strongly convex and smooth functi…

A new method reduces complexity and uncertainty in neural networks.

To address the challenge of backpropagating the gradient through categorical variables, we propose the augment-REINFORCE-swap-merge (ARSM) gradient estimator that is unbiased and has low variance. ARSM first uses variable augmentation, REINFORCE, and Rao-Blackwellization to re-express the gradient as an expectation und…

Policy gradient methods have demonstrated success in reinforcement learning tasks that have high-dimensional continuous state and action spaces. However, policy gradient methods are also notoriously sample inefficient. This can be attributed, at least in part, to the high variance in estimating the gradient of the task…

Stochastic optimization algorithms with variance reduction have proven successful for minimizing large finite sums of functions. Unfortunately, these techniques are unable to deal with stochastic perturbations of input data, induced for example by data augmentation. In such cases, the objective is no longer a finite su…

GT-SARAH optimizes decentralized non-convex problems with recursive variance reduction.

Stochastic particle-optimization sampling (SPOS) is a recently-developed scalable Bayesian sampling framework that unifies stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on Wasserstein gradient flows. With a rigorous non-asymptotic convergence theory developed recently…

FGD reduces noisy gradient variance in SGD for neural networks.

MSTGD optimizes gradient descent with stratified sampling for faster convergence.

Parametric t-SNE improves generalization for streaming data.