arXiv research

New algorithms minimize simple and cumulative regret in contextual bandits.

New algorithm optimizes best arm identification with minimal regret.

There are two variants of the classical multi-armed bandit (MAB) problem that have received considerable attention from machine learning researchers in recent years: contextual bandits and simple regret minimization. Contextual bandits are a sub-class of MABs where, at every time step, the learner has access to side in…

New meta-learning framework for minimizing simple regret in bandits.

Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.

Optimal simple regret bound for Gaussian Process bandits.

Optimal algorithm for identifying best-arm with minimal regret.

In this paper, we formalise order-robust optimisation as an instance of online learning minimising simple regret, and propose Vroom, a zero'th order optimisation algorithm capable of achieving vanishing regret in non-stationary environments, while recovering favorable rates under stochastic reward-generating processes.…

Optimal strategy identified for minimizing regret in fixed-budget best arm selection.

We consider a stochastic bandit problem with infinitely many arms. In this setting, the learner has no chance of trying all the arms even once and has to dedicate its limited number of samples only to a certain number of arms. All previous algorithms for this setting were designed for minimizing the cumulative regret o…

We propose minimum regret search (MRS), a novel acquisition function for Bayesian optimization. MRS bears similarities with information-theoretic approaches such as entropy search (ES). However, while ES aims in each query at maximizing the information gain with respect to the global maximum, MRS aims at minimizing the…

Study optimal stopping for diffusion processes using data-driven methods.

Efficient algorithm for global optimization of multivariate Lipschitz functions.

qEUBO optimizes decision-making with noisy feedback.

RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.

Algorithm optimizes cascaded functions with known structure.

In this paper, we present simple algorithms for Dueling Bandits. We prove that the algorithms have regret bounds for time horizon T of order O(T^rho ) with 1/2 <= rho <= 3/4, which importantly do not depend on any preference gap between actions, Delta. Dueling Bandits is an important extension of the Multi-Armed Bandit…

A simple algorithm reduces federated contextual linear bandits' regret efficiently.

I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…

The paper provides a method to minimize regret in estimate-then-optimize decision-making.

New adaptive learning rate for FTRL reduces regret to Θ(T^2/3).

The paper improves evolutionary computation by optimizing selection rates.

We develop a new theoretical framework, the \emph{envelope complexity}, to analyze the minimax regret with logarithmic loss functions and derive a Bayesian predictor that adaptively achieves the minimax regret over high-dimensional $\ell_1$-balls within a factor of two. The prior is newly derived for achieving the mini…

In this paper, we consider the problem of sequentially optimizing a black-box function $f$ based on noisy samples and bandit feedback. We assume that $f$ is smooth in the sense of having a bounded norm in some reproducing kernel Hilbert space (RKHS), yielding a commonly-considered non-Bayesian form of Gaussian process …

Unified proof for various bandit algorithms with logarithmic regret.

We prove a new minimax theorem connecting the worst-case Bayesian regret and minimax regret under partial monitoring with no assumptions on the space of signals or decisions of the adversary. We then generalise the information-theoretic tools of Russo and Van Roy (2016) for proving Bayesian regret bounds and combine th…

We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter $t_{\rm mix}$. These bounds are the first regret bounds in the general, non-epi…

Study uses contextual bandits to optimize charity exposure in donation solicitation.

We consider a variation on the problem of prediction with expert advice, where new forecasters that were unknown until then may appear at each round. As often in prediction with expert advice, designing an algorithm that achieves near-optimal regret guarantees is straightforward, using aggregation of experts. However, …

New algorithm reduces individual regret and communication costs in cooperative bandits.

This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given…

We argue that the existing regret matchings for Nash equilibrium approximation conduct "jumpy" strategy updating when the probabilities of future plays are set to be proportional to positive regret measures. We propose a geometrical regret matching which features "smooth" strategy updating. Our approach is simple, intu…

Unified framework for high-dimensional bandit problems with low-dimensional structures.

Paper analyzes nonconvex bandit problems with improved adaptive methods.

New algorithm reduces dynamic regret without prior function change knowledge.

Stochastic Rank-One Bandits (Katarya et al, (2017a,b)) are a simple framework for regret minimization problems over rank-one matrices of arms. The initially proposed algorithms are proved to have logarithmic regret, but do not match the existing lower bound for this problem. We close this gap by first proving that rank…

We consider the problem of strongly-convex online optimization in presence of adversarial delays; in a T-iteration online game, the feedback of the player's query at time t is arbitrarily delayed by an adversary for d_t rounds and delivered before the game ends, at iteration t+d_t-1. Specifically for \algo{online-gradi…

We provide a simple and efficient algorithm for adversarial $k$-action $d$-outcome non-degenerate locally observable partial monitoring game for which the $n$-round minimax regret is bounded by $6(d+1) k^{3/2} \sqrt{n \log(k)}$, matching the best known information-theoretic upper bound. The same algorithm also achieves…

Unified framework for distributional regret in bandits and reinforcement learning.

A new exploration strategy for contextual bandits reduces regret and is computationally efficient.

New algorithm learns LQR with $O(\sqrt{T})$ regret using Langevin dynamics and excitation.

OMD and DA perform similarly in static settings but OMD is inferior under dynamic learning rates.

EQO uses a simple bonus term for efficient exploration in tabular RL.

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

New findings show increased exploration needed in non-stationary RL tasks.

Bandit learning algorithms typically involve the balance of exploration and exploitation. However, in many practical applications, worst-case scenarios needing systematic exploration are seldom encountered. In this work, we consider a smoothed setting for structured linear contextual bandits where the adversarial conte…

Two algorithms optimize assortment selection for user choices in unknown MNL models.

New algorithm learns FMDP structure while minimizing regret.