arXiv research

Proposes a method to compute valid lower confidence bounds for multiple models selected based on their performance.

Thompson Sampling with bilateral uncertainty improves performance in Bayesian Optimization.

The betting CI outperforms classical methods in constructing confidence intervals for bounded means.

The paper offers efficient algorithms for combinatorial and linear bandits using empirical process theory.

A new method for evaluating and selecting policies in contextual bandits improves confidence intervals and policy quality.

The paper proposes a novel upper confidence bound (UCB) procedure for identifying the arm with the largest mean in a multi-armed bandit game in the fixed confidence setting using a small number of total samples. The procedure cannot be improved in the sense that the number of samples required to identify the best arm i…

New algorithm identifies best arm in semiparametric bandits with near optimal efficiency.

For an autonomous agent, executing a poor policy may be costly or even dangerous. For such agents, it is desirable to determine confidence interval lower bounds on the performance of any given policy without executing said policy. Current methods for exact high confidence off-policy evaluation that use importance sampl…

We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood techniques to formulate our estimator and confidence interval as simple convex optimization problems. Using the lower bound of our confiden…

The stochastic multi-armed bandit model is a simple abstraction that has proven useful in many different contexts in statistics and machine learning. Whereas the achievable limit in terms of regret minimization is now well known, our aim is to contribute to a better understanding of the performance in terms of identify…

We study model-based reinforcement learning in an unknown finite communicating Markov decision process. We propose a simple algorithm that leverages a variance based confidence interval. We show that the proposed algorithm, UCRL-V, achieves the optimal regret $\tilde{\mathcal{O}}(\sqrt{DSAT})$ up to logarithmic factors…

Berry et al. (1997) initiated the development of the infinite arms bandit problem. They derived a regret lower bound of all allocation strategies for Bernoulli rewards with uniform priors, and proposed strategies based on success runs. Bonald and Proutière (2013) proposed a two-target algorithm that achieves the regret…

The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.

The paper extends confidence sequences for infinite variance data.

This work creates a CS for non-negative heavy-tailed data with bounded mean.

Optimal best-arm identification with known number of optimal arms.

A bandit algorithm reduces regret in noisy, communication-constrained feedback.

The paper studies privacy-protected BAI with fixed confidence, deriving lower bounds and proposing an adaptive algorithm.

Study optimizes dynamic product selection and pricing using censored preference feedback.

Improved confidence bounds for linear logistic model with applications to bandits.

BILBO optimizes bilevel problems without repeated lower-level optimizations.

BestChanID identifies the channel with maximal capacity using training sequences.

Paper presents robust confidence sequences for means with known moment bounds and arbitrary corruption.

This paper is devoted to regret lower bounds in the classical model of stochastic multi-armed bandit. A well-known result of Lai and Robbins, which has then been extended by Burnetas and Katehakis, has established the presence of a logarithmic bound for all consistent policies. We relax the notion of consistence, and e…

New algorithms identify best policies in discounted linear MDPs efficiently.

We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search. It consists in identifying the best action in a game, when the player may sample random outcomes of sequentially chosen pairs of actions. We propose two strategies for the fixed-confidence setting: Maximin…

TS-UCB improves Thompson Sampling with minimal extra computation.

New algorithm optimizes online decision-making with dynamically generated actions.

Optimal best arm identification for multi-objective bandits with fixed error probability.

New algorithm improves best arm identification in Bayesian settings.

Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.

I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…

Proposes online debiasing estimators for adaptive linear regression.

We consider the problem of undirected graphical model inference. In many applications, instead of perfectly recovering the unknown graph structure, a more realistic goal is to infer some graph invariants (e.g., the maximum degree, the number of connected subgraphs, the number of isolated nodes). In this paper, we propo…

A bandit problem with filtered Poisson process data.

A method identifies abrupt changes in functions with fixed confidence under noisy feedback.

We study the linear contextual bandit problem with finite action sets. When the problem dimension is $d$, the time horizon is $T$, and there are $n \leq 2^{d/2}$ candidate actions per time period, we (1) show that the minimax expected regret is $Ω(\sqrt{dT (\log T) (\log n)})$ for every algorithm, and (2) introduce a V…

GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.

Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confide…

Upper bound on withdrawal success for geometric Levy alpha-stable wealth process.

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

New RL method tackles dynamic MDPs with evolving rewards and states.

Paper solves robust convex problems with heavy-tailed noise.

We consider the problem of near-optimal arm identification in the fixed confidence setting of the infinitely armed bandit problem when nothing is known about the arm reservoir distribution. We (1) introduce a PAC-like framework within which to derive and cast results; (2) derive a sample complexity lower bound for near…

New bounds for Bayesian bandits show prior improves performance.

The study optimizes distribution estimation from samples with relative entropy error, adapting to sparse distributions.

We consider the Lipschitz bandit optimization problem with an emphasis on practical efficiency. Although there is rich literature on regret analysis of this type of problem, e.g., [Kleinberg et al. 2008, Bubeck et al. 2011, Slivkins 2014], their proposed algorithms suffer from serious practical problems including extre…

Prove non-asymptotic bounds for minimal risk in statistical learning