arXiv research

Study noise-free kernel bandits, finding upper bounds on regret.

Balances the regret of different algorithms in bandit and RL problems.

New algorithms reduce regret in online MDPs by adapting to data and variance.

New algorithm reduces regret in infinite MDPs with optimal variance-dependent bounds.

Bayesian optimization algorithm reduces regret with efficient region pruning.

New algorithm reduces regret in bandit optimization for high-dimensional data.

Bandit algorithms have been predominantly analyzed in the convex setting with function-value based stationary regret as the performance measure. In this paper, motivated by online reinforcement learning problems, we propose and analyze bandit algorithms for both general and structured nonconvex problems with nonstation…

We demonstrate that, in the classical non-stochastic regret minimization problem with $d$ decisions, gains and losses to be respectively maximized or minimized are fundamentally different. Indeed, by considering the additional sparsity assumption (at each stage, at most $s$ decisions incur a nonzero outcome), we derive…

Random exploration optimizes Bayesian optimization with optimal error rates and computational efficiency.

GN algorithm solves batched bandit for nondegenerate functions near-optimally.

New RL method handles large state-action spaces with complex models.

Unified framework for analyzing online convex optimization across various settings.

Optimal simple regret bound for Gaussian Process bandits.

New algorithm reduces regret in stochastic bandit convex optimization.

New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.

This paper achieves first-order regret bounds in reinforcement learning with large state spaces.

Saddle-point optimization problems are an important class of optimization problems with applications to game theory, multi-agent reinforcement learning and machine learning. A majority of the rich literature available for saddle-point optimization has focused on the offline setting. In this paper, we study nonstationar…

Optimal algorithm for minimizing regret in heavy-tailed bandits.

Optimal algorithm found for collaborative learning in bandits with optimal regret bounds.

Kernel online convex optimization (KOCO) is a framework combining the expressiveness of non-parametric kernel models with the regret guarantees of online learning. First-order KOCO methods such as functional gradient descent require only $\mathcal{O}(t)$ time and space per iteration, and, when the only information on t…

We consider $K$-armed stochastic bandits and consider cumulative regret bounds up to time $T$. We are interested in strategies achieving simultaneously a distribution-free regret bound of optimal order $\sqrt{KT}$ and a distribution-dependent regret that is asymptotically optimal, that is, matching the $κ\ln T$ lower b…

We study the problem of regret minimization for distributed bandits learning, in which $M$ agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…

New framework for decentralized optimization of upper-linearizable functions with improved regret and complexity.

We consider the adversarial convex bandit problem and we build the first $\mathrm{poly}(T)$-time algorithm with $\mathrm{poly}(n) \sqrt{T}$-regret for this problem. To do so we introduce three new ideas in the derivative-free optimization literature: (i) kernel methods, (ii) a generalization of Bernoulli convolutions, …

Randomized exploration in linear bandits achieves optimal regret bounds.

Efficient algorithm for zeroth-order bandit convex optimization with bounds on regret.

We consider stochastic multi-armed bandits where the expected reward is a unimodal function over partially ordered arms. This important class of problems has been recently investigated in (Cope 2009, Yu 2011). The set of arms is either discrete, in which case arms correspond to the vertices of a finite graph whose stru…

New framework for Adam-type algorithms with constant β1, improving regret analysis.

The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.

A new metric, Weighted Regret, unifies FDR and power evaluation in online multiple testing.

We address the online linear optimization problem with bandit feedback. Our contribution is twofold. First, we provide an algorithm (based on exponential weights) with a regret of order $\sqrt{d n \log N}$ for any finite action set with $N$ actions, under the assumption that the instantaneous loss is bounded by 1. This…

New method reduces ensemble size for linear bandits, achieving near optimal regret.

Improves policy optimization with polylog(T) regret bounds for stochastic losses.

Algorithm reduces regret in distributed kernel bandits with shared randomness.

Study optimizes zero-order strongly convex function minimization with higher order smoothness.

The problem of distributed learning and channel access is considered in a cognitive network with multiple secondary users. The availability statistics of the channels are initially unknown to the secondary users and are estimated using sensing decisions. There is no explicit information exchange or prior agreement amon…

New algorithm reduces regret in private online learning with optimal gap-dependent rate.

New algorithm learns optimal policies with minimal memory and time.

Continuous-time algorithms improve online learning performance.

New RL algorithm reduces sample complexity for optimal learning.

We provide the first algorithm for online bandit linear optimization whose regret after T rounds is of order sqrt{Td ln N} on any finite class X of N actions in d dimensions, and of order d*sqrt{T} (up to log factors) when X is infinite. These bounds are not improvable in general. The basic idea utilizes tools from con…

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

We study the stochastic multi-armed bandit problem when one knows the value $μ^{(\star)}$ of an optimal arm, as a well as a positive lower bound on the smallest positive gap $Δ$. We propose a new randomized policy that attains a regret {\em uniformly bounded over time} in this setting. We also prove several lower bound…

New algorithm reduces dynamic regret in time-varying movement costs.

Algorithm optimizes spectrum access for dynamic multi-user environments.

Algorithm learns to bid optimally in repeated first-price auctions with censored feedback.

New algorithm minimizes expert selection regret in partial bandit feedback.

We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute. Specifically, we study two regret minimisation problems over subsets of a finite ground set $[n]$, with subset-wise relative prefe…