arXiv research

The paper analyzes deep neural networks using control theory to set a time limit for their convergence.

Study on SA with heavy-tailed and LRD noise, establishing finite-time bounds.

Paper analyzes finite-time performance of SA in RL with Markovian noise.

This work analyzes actor-critic methods for faster convergence.

Paper analyzes Q-learning with neural networks, proving a fast convergence rate.

New analysis of stochastic approximation with non-expansive mappings.

Paper analyzes finite-time convergence of double Q-learning.

Estimates for VPMCF show ancient MCF solutions and finite-time behavior.

Improved TD learning with tail averaging and regularization achieves optimal convergence rates.

Paper analyzes convergence of dynamic policy gradient for MDPs, improving performance in finite-time problems.

Logarithmic regret achieved in continuous-time linear-quadratic reinforcement learning.

We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.

I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…

The paper analyzes finite-time singularities in Spin(7)-structure flows using Shi-type estimates.

The question of the optimality of Thompson Sampling for solving the stochastic multi-armed bandit problem had been open since 1933. In this paper we answer it positively for the case of Bernoulli rewards by providing the first finite-time analysis that matches the asymptotic rate given in the Lai and Robbins lower boun…

We study finite sample properties of estimators of power-law cross-correlations -- detrended cross-correlation analysis (DCCA), height cross-correlation analysis (HXA) and detrending moving-average cross-correlation analysis (DMCA) -- with a special focus on short-term memory bias as well as power-law coherency. Presen…

I analyse the frequentist regret of the famous Gittins index strategy for multi-armed bandits with Gaussian noise and a finite horizon. Remarkably it turns out that this approach leads to finite-time regret guarantees comparable to those available for the popular UCB algorithm. Along the way I derive finite-time bounds…

First-order method solves stochastic bilevel optimization with linear constraints.

The OLS estimator optimally identifies stable linear systems with a finite number of samples.

Novel analysis of EFP for finite-sum problems in neural networks.

Constructing eigenfunctions for finite-time singularities in Lagrangian mean curvature flow

Minimal assumptions analysis of Q-learning with time-varying policies.

New analysis shows actor-critic method converges efficiently in practical settings.

Paper analyzes finite-time guarantees for preference-based RL.

Paper analyzes SVGD algorithm for non-asymptotic convergence.

In this paper, we analyze the finite sample complexity of stochastic system identification using modern tools from machine learning and statistics. An unknown discrete-time linear system evolves over time under Gaussian noise without external inputs. The objective is to recover the system parameters as well as the Kalm…

We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward…

New bounds derived for KG algorithm's performance in finite time.

Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process. Although TD is one of the most widely used algorithms in reinforcement learning, its theoretical analysis has proved challenging and few guarantees on its s…

Paper analyzes NAC with neural networks for efficient policy optimization.

Bitcoin volatility analysis shows decreasing HE with longer sampling periods.

Study on improving the linear two-time-scale stochastic approximation method with a restarting scheme.

Theoretical justification for asymmetric actor-critic algorithms in reinforcement learning.

Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the …

Paper analyzes PSGLD for adaptive IRL with finite-sample bounds.

Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.

Despite the increasing interest in multi-agent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch MARL with networked agents…

The paper provides bounds on estimation error in a distributed online learning setting.

The paper analyzes stability and asymptotic behavior of hedging strategies in binomial and trinomial models.

We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on t…

The paper offers precise bounds for averaged LSA iterates in linear systems.

This work analyzes $Q$-learning with adaptive stepsizes for finite-time convergence.

Study optimal adaptive allocation for multi-armed bandits with Markovian rewards.

SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.\ data, where a single sample trajectory is available. With a Lipschitz continuous policy improvement operator that is smooth eno…

Study on eigenvalue distribution of correlated time series, showing deformation of Marchenko-Pastur distribution.

New Thompson sampling algorithm reduces regret for exponential family bandits.

This study analyzes LTS in sparse models with finite sample error bounds.

Proposes a TS approach for Bayesian optimization with preferential feedback.