arXiv research

A new algorithm SRG-DQN reduces variance in deep Q-learning.

STORM-PG uses momentum for faster policy gradient updates.

MARS optimizes large model training by reducing variance, outperforming AdamW.

GT-SARAH optimizes decentralized non-convex problems with recursive variance reduction.

FPG uses fractional calculus for efficient reinforcement learning with long-term memory.

Policy gradient methods have demonstrated success in reinforcement learning tasks that have high-dimensional continuous state and action spaces. However, policy gradient methods are also notoriously sample inefficient. This can be attributed, at least in part, to the high variance in estimating the gradient of the task…

In this work, we revisit fast dimension reduction approaches, as with random projections and random sampling. Our goal is to summarize the data to decrease computational costs and memory footprint of subsequent analysis. Such dimension reduction can be very efficient when the signals of interest have a strong structure…

Improved sample complexity for actor-critic algorithms in MDPs.

Paper improves convergence rate of Langevin Dynamics algorithms.

In this paper we analyze a dynamic recursive extension of the (static) notion of a deviation measure and its properties. We study distribution invariant deviation measures and show that the only dynamic deviation measure which is law invariant and recursive is the variance. We also solve the problem of optimal risk-sha…

We develop a new Monte Carlo variance reduction method to estimate the expectation of two commonly encountered path-dependent functionals: first-passage times and occupation times of sets. The method is based on a recursive approximation of the first-passage time probability and expected occupation time of sets of a Le…

Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.

Explains gradient descent methods and their convergence, focusing on simple analysis.

Stochastic compositional optimization arises in many important machine learning tasks such as value function evaluation in reinforcement learning and portfolio management. The objective function is the composition of two expectations of stochastic functions, and is more challenging to optimize than vanilla stochastic o…

Improving the sample efficiency in reinforcement learning has been a long-standing research problem. In this work, we aim to reduce the sample complexity of existing policy gradient methods. We propose a novel policy gradient algorithm called SRVR-PG, which only requires $O(1/ε^{3/2})$ episodes to find an $ε$-approxima…

A new method optimizes diffusion models with recursive likelihood ratios.

Improved real-time UAV terrain following with RVM-RLS filter.

New methods reduce constraint violations to certainty in stochastic optimization.

AdaVol adapts QML for real-time GARCH volatility prediction.

Data balancing reduces variance in machine learning models.

Neural SDEs reduce variance in stochastic simulations.

Normal distributions ensure asymptotic variance reduction in moment matching Monte Carlo.

In this paper we suggest a modification of the regression-based variance reduction approach recently proposed in Belomestny et al. This modification is based on the stratification technique and allows for a further significant variance reduction. The performance of the proposed approach is illustrated by several numeri…

New algorithm reduces optimization complexity in adaptive mirror descent.

SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.

We consider nonconvex-concave minimax optimization problems of the form $\min_{\bf x}\max_{\bf y\in{\mathcal Y}} f({\bf x},{\bf y})$, where $f$ is strongly-concave in $\bf y$ but possibly nonconvex in $\bf x$ and ${\mathcal Y}$ is a convex and compact set. We focus on the stochastic setting, where we can only access an…

Optimizes MCMC chains with neural control variates.

Paper proposes a method to estimate variance reduction in DNN training using importance sampling.

Optimizes variance reduction in Heston model using large and moderate deviations.

In this paper we propose a novel variance reduction approach for additive functionals of Markov chains based on minimization of an estimate for the asymptotic variance of these functionals over suitable classes of control variates. A distinctive feature of the proposed approach is its ability to significantly reduce th…

We propose a novel algorithm which allows to sample paths from an underlying price process in a local volatility model and to achieve a substantial variance reduction when pricing exotic options. The new algorithm relies on the construction of a discrete multinomial tree. The crucial feature of our approach is that -- …

This paper identifies knot projections with reductivity two.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

New optimizer MARS-M combines variance reduction with Muon for faster LLM training.

Value-at-Risk (VaR) and Conditional Value-at-Risk (CVaR) are two risk measures which are widely used in the practice of risk management. This paper deals with the problem of computing both VaR and CVaR using stochastic approximation (with decreasing steps): we propose a first Robbins-Monro procedure based on Rockaffela…

Develops a new SPP algorithm with variance reduction for weakly convex optimization.

Fibonacci Ensembles use Fibonacci weights to improve ensemble learning, inspired by natural growth patterns.

Stochastic Variance-Reduced Cubic regularization (SVRC) algorithms have received increasing attention due to its improved gradient/Hessian complexities (i.e., number of queries to stochastic gradient/Hessian oracles) to find local minima for nonconvex finite-sum optimization. However, it is unclear whether existing SVR…

Improved time complexity for parallel stochastic optimization in heterogeneous systems.

Paper develops momentum schemes with variance reduction for non-convex composition optimization.

The dichotomous coordinate descent (DCD) algorithm has been successfully used for significant reduction in the complexity of recursive least squares (RLS) algorithms. In this work, we generalize the application of the DCD algorithm to RLS adaptive filtering in impulsive noise scenarios and derive a unified update formu…

This paper develops a new theory for ensemble learning beyond variance reduction.

Paper improves tree probability estimation using stochastic optimization and variance reduction.

New algorithms reduce variance in solving complex mathematical problems.

Graph convolutional networks (GCNs) are powerful deep neural networks for graph-structured data. However, GCN computes the representation of a node recursively from its neighbors, making the receptive field size grow exponentially with the number of layers. Previous attempts on reducing the receptive field size by subs…

Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.

We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved incl…

Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classification problems with functional data. In this context, variable selection techniques are especially a…