New algorithm optimizes multi-armed bandit performance in stochastic and adversarial settings.
problem Optimizing multi-armed bandit performance in both stochastic and adversarial environments.
method Follow-the-regularized-leader method with adaptive learning rates.
result First BOBW algorithm with gap-variance-dependent regret bounds in adversarial settings.
New algorithm identifies best arm in non-stationary linear bandits with improved complexity.
problem Best arm identification in non-stationary linear bandits with adversarial parameters.
method Proposed Adjacent-optimal design and extsfAdjacent−BAI algorithm. result Error probability matches arm-set-dependent lower bound up to constants.
Optimal algorithms identify non-dominated arms in multi-output linear bandit models.
problem Identifying the Pareto Set in multi-output linear bandit models.
method Design-based algorithms for Pareto Set Identification (PSI) in a structured multi-output linear bandit model.
result Nearly optimal guarantees in both fixed-budget and fixed-confidence settings.
We consider a stochastic bandit problem with infinitely many arms. In this setting, the learner has no chance of trying all the arms even once and has to dedicate its limited number of samples only to a certain number of arms. All previous algorithms for this setting were designed for minimizing the cumulative regret o…
Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.
problem Identify the best arm in a multi-armed bandit with time-varying states.
method Propose a sequential policy to select arms without knowing their exact TPMs.
result Upper and lower bounds on expected time to find the best arm match in a special case.
Study optimal arms in combinatorial bandits with semi-bandit feedback and finite budget.
problem Finding optimal arms in combinatorial bandits with semi-bandit feedback and finite budget constraints.
method Proposes a generic algorithm covering various arm elimination strategies and derives lower bounds.
result Demonstrates sufficient and necessary budget requirements for finding the best arm.
Optimal resource allocation in censored semi-bandits with unknown thresholds.
problem Sequential resource allocation with unknown thresholds and hidden parameters.
method Established equivalence to MP-MAB and Combinatorial Semi-Bandits, derived optimal algorithms.
result Validated performance of proposed algorithms on synthetic data.
New algorithm tackles stochastic bandits with varying arm-dependent delays.
problem Applying existing algorithms to stochastic delayed bandit settings is restricted by strong assumptions on delay distributions.
method Proposes a simple UCB-based algorithm called PatientBandits that weakens assumptions on delay distributions.
result Provides bounds on regret and performance lower bounds for the PatientBandits algorithm.
Open problem: fixed-budget best arm identification complexity.
problem Understanding the complexity of identifying the best arm in a fixed budget setting.
method Analyzing existing results and conjectures in the fixed-confidence setting.
result Open questions remain about the fixed-budget setting.
Investigates probability of error in structured thresholding bandit problems.
problem Probability of misclassifying arms in structured thresholding bandit problems.
method Analyzes two shape constraints: monotonic increasing and concave sequences of arm means.
result Upper and lower bounds for the probability of error match up to constants in the problem dependent regime.
New algorithm for shareable arms with load-dependent rewards in stochastic bandits.
problem Learning optimal play strategy with shareable finite-capacity arms in stochastic bandits.
method Developed a capacity estimator and online learning algorithm for MP-MAB with shareable arms.
result Regret upper bound matches the lower bound, validating the algorithm's performance.
Sequential screening and dynamic regret in multi-armed bandits with arriving arms
problem Sequential experimentation with expanding arm set
method UCB-AA with preliminary screening
result Regret bounds depend on arrival process
Unified framework controls false discovery rate in bandit multiple testing.
problem Designing adaptive algorithms to identify true discoveries in multiple hypothesis testing.
method Unified modular framework using e-processes for FDR control in arbitrary settings.
result Unified framework ensures FDR control for dependent and simultaneous arm queries.
In this paper, we study censored Semi-Bandits, a novel variant of the semi-bandits problem. The learner is assumed to have a fixed amount of resources, which it allocates to the arms at each time step. The loss observed from an arm is random and depends on the amount of resources allocated to it. More specifically, the…
New algorithms identify best arm with less pulls, adapting to arm covariances.
problem Best arm identification under dependent and correlated arm distributions.
method Adaptive algorithms estimating arm covariances to minimize pulls.
result Substantial improvement in best arm identification over standard setting.
Algorithm identifies best arm with prior info in structured bandits.
problem Bayesian fixed-budget best-arm identification in structured bandits.
method Prior-dependent allocations based on structure and prior information.
result Improved theoretical bounds and robust performance across diverse models.
Improved best-arm identification in correlated multi-armed bandits.
problem Best-arm identification in multi-armed bandits with correlated rewards.
method Proposed C-LUCB algorithm that exploits upper bounds on conditional rewards.
result Significant reduction in sample complexity for best-arm identification.
We consider the problem of near-optimal arm identification in the fixed confidence setting of the infinitely armed bandit problem when nothing is known about the arm reservoir distribution. We (1) introduce a PAC-like framework within which to derive and cast results; (2) derive a sample complexity lower bound for near…
Algorithm selects k arms from context-dependent options using Plackett-Luce model.
problem Selecting k arms from context-dependent options with Plackett-Luce feedback.
method Proposes CPPL algorithm inspired by UCB, evaluated on synthetic and real data.
result Demonstrates effectiveness of CPPL algorithm in online algorithm selection.
New algorithm identifies optimal actions in large reward spaces efficiently.
problem Finding the best action from a large set of options with minimal trials.
method GenTS-Explore algorithm for real-valued combinatorial pure exploration.
result Achieves optimal sample complexity for large action sets.
Optimal algorithm for selecting high-quality arms from infinite bandit arms.
problem Efficiently choosing the best arm from an infinite set of options.
method Developed algorithms for both fixed confidence and fixed budget settings, achieving optimal or near-optimal sample complexities.
result Optimal sample complexity results for both fixed confidence and fixed budget settings, resolving open questions in the field.
Paper improves CMAB regret bounds by reducing batch-size dependency.
problem Reducing batch-size dependency in combinatorial semi-bandits.
method Developed BCUCB-T and SESCB algorithms with new TPVM conditions.
result Significantly improved regret bounds for various applications.
New models for bandit problems with fidelity rewards are introduced and analyzed.
problem Fidelity rewards in bandit problems to incentivize loyalty.
method Two models (loyalty-points and subscription) for fidelity rewards; stochastic and adversarial settings considered.
result Sublinear regret bounds for some models, worst case lower bounds for others.
New algorithm finds best feasible arm in grouped bandits.
problem Finding the best arm with all attributes above a threshold.
method Feasibility Constrained Successive Rejects (FCSR) algorithm.
result FCSR identifies the best feasible arm with optimal dependence on problem parameters.
Bayesian algorithm improves best-arm identification within fixed budget.
problem Maximizing probability of identifying optimal arm within fixed budget.
method Proposes Bayesian elimination algorithm and derives upper bound on misidentification probability.
result Upper bound on misidentification probability reflects prior quality and matches lower bound.
We develop a novel and generic algorithm for the adversarial multi-armed bandit problem (or more generally the combinatorial semi-bandit problem). When instantiated differently, our algorithm achieves various new data-dependent regret bounds improving previous work. Examples include: 1) a regret bound depending on the …
Optimal best-arm identification in linear bandits reduces sampling budget.
problem Identifying the best arm with fixed confidence in stochastic linear bandits.
method A simple algorithm that tracks an optimal proportion of arm draws, updated as rarely as desired.
result The algorithm's sampling complexity matches known lower bounds, asymptotically almost surely and in expectation.
New algorithm for countable bandits with optimal regret.
problem Stochastic bandit problem with countably many arms.
method Fully adaptive online learning algorithm with O(log n) expected cumulative regret.
result Achieves optimal regret of O(log n) after any number of plays n.
Consider a nonparametric contextual multi-arm bandit problem where each arm a∈[K] is associated to a nonparametric reward function fa:[0,1]→R mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward…
Algorithm aggregates rewards from multiple players to learn related tasks in online bandit learning.
problem Learning related but slightly different tasks in an online setting with heterogeneous feedback.
method RobustAgg(ε) algorithm that aggregates rewards from different players. result Achieves instance-dependent regret guarantees and nearly matching lower bounds.
We consider the problem of best arm identification in a variant of multi-armed bandits called linked bandits. In a single interaction with linked bandits, multiple arms are played sequentially until one of them receives a positive reward. Since each interaction provides feedback about more than one arm, the sample comp…
Given a vector of probability distributions, or arms, each of which can be sampled independently, we consider the problem of identifying the partition to which this vector belongs from a finitely partitioned universe of such vector of distributions. We study this as a pure exploration problem in multi armed bandit sett…
New algorithm reduces contextual bandit identification to argmax calls.
problem Best-arm identification in stochastic contextual bandits.
method Instance-optimal PAC algorithm using argmax oracle calls.
result First instance-dependent PAC sample complexity for contextual bandits.
New lower bounds for combinatorial multi-armed bandits for general reward functions.
problem Maximizing reward in sequential decisions with sets of arms.
method Proved tight regret lower bounds for all smooth reward functions under mild assumptions.
result Lower bounds are tight up to log-factors for monotone reward functions.
Optimizes arm selection with side information in Gaussian bandits.
problem Optimizing arm selection with side information in Gaussian bandits.
method Constructs an LP-based asymptotic instance-dependent lower bound on the regret and develops the first known asymptotically optimal algorithm.
result First known asymptotically optimal algorithm for Gaussian bandits with side information.
In this paper we propose the multi-objective contextual bandit problem with similarity information. This problem extends the classical contextual bandit problem with similarity information by introducing multiple and possibly conflicting objectives. Since the best arm in each objective can be different given the contex…
Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the reward distributions: the classic K-armed bandit and the linearly parameterized bandit…
Many sequential decision-making tasks require choosing at each decision step the right action out of the vast set of possibilities by extracting actionable intelligence from high-dimensional data streams. Most of the times, the high-dimensionality of actions and data makes learning of the optimal actions by traditional…
Algorithm identifies best arm in bandit game with variance consideration.
problem Identifying the best arm in a stochastic multi-armed bandit game with varying variances.
method Adaptive algorithm using grouped median elimination to explore gaps and variances.
result Guarantees to output the best arm with probability (1-δ) using optimal number of samples.
We consider stochastic multi-armed bandits where the expected reward is a unimodal function over partially ordered arms. This important class of problems has been recently investigated in (Cope 2009, Yu 2011). The set of arms is either discrete, in which case arms correspond to the vertices of a finite graph whose stru…
New algorithm identifies best arm in semiparametric bandits with near optimal efficiency.
problem Fixed-confidence Best Arm Identification in semiparametric bandits with unknown baseline shift.
method Phase-elimination algorithm based on orthogonalized regression design.
result Nearly optimal high-probability sample-complexity upper bound established.
New algorithm identifies best arm in rested bandit setting.
problem Best arm identification in rested bandit with decreasing losses.
method Introduced a novel best arm identification problem and analyzed an arm elimination algorithm.
result Regret vanishes as time horizon increases, with convergence rate depending on expected loss function.
New insights on bias in multi-armed bandits under conditional sampling.
problem Understanding bias in multi-armed bandits under conditional sampling.
method Characterized the sign of conditional bias of monotone functions of rewards.
result Sign of conditional bias can differ from marginal bias, depending on conditioning events.
We consider a variant of the classic multi-armed bandit problem where the expected reward of each arm is a function of an unknown parameter. The arms are divided into different groups, each of which has a common parameter. Therefore, when the player selects an arm at each time slot, information of other arms in the sam…
In this paper, we introduce Ballooning Multi-Armed Bandits (BL-MAB), a novel extension of the classical stochastic MAB model. In the BL-MAB model, the set of available arms grows (or balloons) over time. In contrast to the classical MAB setting where the regret is computed with respect to the best arm overall, the regr…
COF algorithm minimizes cost in multi-armed bandits with known costs and reward constraints.
problem Minimizing cost while meeting a minimum reward requirement in uncertain environments.
method COF algorithm that intelligently combines samples from all arms to gauge feasibility and minimize cost.
result COF achieves instance-dependent upper bounds on cumulative cost and quality regret.
The paper studies how arm selection in a bandit problem changes with shape constraints.
problem Stochastic Thresholding Bandit Problem under shape constraints.
method Investigation of TBP under four shape constraints: monotonic increasing, unimodal, concave, and fixed.
result Minimax rates for regret vary significantly depending on the shape constraint.
The paper tackles noisy multi-armed bandit problems with improved regret guarantees.
problem Tackling noisy evaluations in multi-armed bandit problems.
method Derives different algorithmic approaches and theoretical guarantees based on the type of observation functions.
result Improved regret guarantees for noisy linear functions of true rewards.