New algorithms identify top K arms with high confidence.
problem Identifying K arms with the largest means from N stochastic bandit arms.
method Proposed new practical algorithms with nearly optimal sample complexity.
result Outperform state-of-the-art algorithms in practical applications.
Paper optimizes sample selection for top-k arms in stochastic bandits.
problem Identifying the k arms with the largest means in stochastic bandits.
method Developed an elimination-based algorithm with sample complexity matching lower bounds.
result Algorithm strictly dominates state-of-the-art for Best-k-Arm problem.
Top-two algorithm improved for best-k-arm selection.
problem Best-k-arm identification in multi-armed bandits.
method Information-directed selection based on dual variables.
result Top-two Thompson sampling with IDS is asymptotically optimal.
The paper optimizes identifying top k arms from a fraction of ρ arms in stochastic bandits.
problem Identifying k distinct arms among the top ρ fraction of arms in stochastic bandits with a PAC tolerance. method The paper considers two cases: known and unknown threshold of top arms' expected rewards. It proves lower bounds and proposes algorithms for each case, showing sample complexity optimality for two algorithms.
result Two algorithms are sample complexity optimal (up to constant factors) and the other two are optimal up to a log factor.
This paper optimizes the number of comparisons needed to find the best k items from pairwise comparisons.
problem Finding the best k items from pairwise comparisons with limited comparisons.
method Developed algorithms for finding probably approximately correct and exact best k items under stochastic conditions.
result Upper and lower bounds on the number of comparisons for finding the best k items, with matching upper bounds for PAC best k items.
New algorithm for choosing top-K options efficiently with linear space and non-linear feedback.
problem Choosing the best K out of N options at each time instant with efficient trade-off between exploration and exploitation.
method CMAB-SM algorithm, divide-and-conquer strategy, linear space, non-linear feedback.
result Achieves a sub-linear regret bound of $O(K^{ rac{1}{2}}N^{ rac{1}{3}}T^{ rac{2}{3}})$.
No best k-layer neural network approximations exist in general for common activations.
problem The non-existence of best k-layer neural network approximations for common activation functions.
method Empirical Risk Minimization (ERM) for neural networks with k layers and various activation functions.
result For k = 2 and common activations like ReLU, hyperbolic tangent, and sigmoid, the ERM infimum is not attainable in general.
Study on a bad arm existence checking problem to minimize arm draws.
problem Judging the existence of a positive arm among K given arms.
method Proposes an algorithm with arm selection policy and stopping condition.
result Proves the effectiveness of the proposed algorithm theoretically and empirically.
New algorithms for streaming bandits with limited memory.
problem Optimizing decisions in a stream of uncertain outcomes with limited memory.
method Developed algorithms for minimizing regret and identifying the best arm under bounded memory constraints.
result Upper and lower bounds on sample complexity for best-arm identification algorithms.
Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.
problem Identify the best arm in a multi-armed bandit with time-varying states.
method Propose a sequential policy to select arms without knowing their exact TPMs.
result Upper and lower bounds on expected time to find the best arm match in a special case.
Best-arm identification in bandits with sequential elimination.
problem Identifying the arm with the highest expected reward in a budget-limited exploration process.
method Unified sequential elimination algorithms, dividing budget based on nonlinear function of remaining arms.
result Improved theoretical guarantees and performance over state-of-the-art algorithms.
Paper tackles identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.
problem Identifying an odd arm in a multi-armed bandit with restless Markov processes and trembling hand.
method Derive asymptotic lower bound on expected time to identify the odd arm, stitch together parameterised solutions to MDPs.
result First known asymptotic lower bound on expected time to identify the odd arm, with vanishing error probability.
A new bandit framework reduces K-armed to C+1-armed, achieving lower regret.
problem Designing efficient algorithms for correlated multi-armed bandits.
method Generalized UCB algorithm exploiting latent random source correlation.
result Achieves O(1) regret for certain regimes, reducing from logarithmic. A study on incentivizing strategic arms to share rewards in a multi-armed bandit problem.
problem Designing an algorithm to encourage strategic arms to share their rewards with a principal.
method An algorithm that induces a game among the arms where each arm has a dominant strategy, ensuring the principal sees expected reward μ′T−o(T). result An algorithm that ensures the principal sees expected reward μ′T−o(T), even when arms are strategic or a mix of strategic and non-strategic. Algorithm identifies top M arms from K in stochastic bandits with limited budget.
problem Identifying the top M arms from K in a stochastic bandit setting with limited exploration budget.
method Develops an iterative algorithm that allocates budget nonlinearly to deactivate arms, deciding acceptance or rejection based on a decision rule.
result The algorithm effectively identifies the top M arms with a decay rate of misidentification probability characterized by nonlinear budget allocation.
Improved theoretical guarantees for Top Two algorithms.
problem Theoretical support for best arm identification with bounded distributions.
method General analysis of Top Two methods, identifying desirable properties and replacing sampling step.
result Theoretical support for Top Two algorithms with bounded distributions.
Sequential screening and dynamic regret in multi-armed bandits with arriving arms
problem Sequential experimentation with expanding arm set
method UCB-AA with preliminary screening
result Regret bounds depend on arrival process
Paper tackles good arm identification in stochastic bandits.
problem Identifying good arms with minimal samples.
method Proposes DGAI, a differentiable algorithm to improve sample complexity.
result DGAI outperforms baseline algorithms in synthetic and real-world datasets.
A novel algorithm reduces communication costs in federated best arm identification.
problem Identifying the best arm in a federated multi-armed bandit setup with minimal communication cost.
method Proposes a novel algorithm called FedElim that communicates only in exponential time steps.
result Demonstrates that communication is almost cost-free in FedElim, with a total cost at most 3 times the maximum under its variant.
New algorithms identify best arm with less pulls, adapting to arm covariances.
problem Best arm identification under dependent and correlated arm distributions.
method Adaptive algorithms estimating arm covariances to minimize pulls.
result Substantial improvement in best arm identification over standard setting.
Study best arm identification in restless bandits with unknown TPMs.
problem Identify the best arm with fixed confidence in restless bandits with unknown TPMs.
method Proposed a policy for best arm identification and proved its expected stopping time matches the lower bound.
result The state-action visitation proportions match the optimal proportions under any asymptotically optimal policy.
New method for identifying best arm in batched multi-armed bandit problems.
problem Identifying the best arm in multi-armed bandit problems where arms are sampled in batches.
method General linear programming framework for best arm identification in batched multi-armed bandit problems.
result Demonstrated good performance in numerical studies compared to UCB-type or Thompson sampling methods.
Optimal best-arm identification with known number of optimal arms.
problem Identifying the best arm in a multi-armed bandit with multiple optimal arms under fixed confidence.
method Deriving a new information-theoretic lower bound and proposing a modified stopping rule.
result Achieving asymptotic instance-optimality with a new lower bound and new stopping rule.
Paper tackles outlier detection in multi-armed bandits, achieving high accuracy with reduced exploration costs.
problem Detecting outlier arms in multi-armed bandit settings.
method Proposes GOLD algorithm based on upper confidence bounds to identify generic outlier arms.
result Achieves 98% accuracy with 83% reduction in exploration cost compared to state-of-the-art techniques.
New algorithm identifies dominant arm with high probability.
problem Identifying the arm with the highest realized reward in multi-armed bandits.
method Dominance score criterion and joint mixing and recycling mechanism.
result Identifies the best dominant arm with nearly optimal sample complexity.
CTS reduces regret in probabilistically triggered combinatorial bandits.
problem Optimizing decisions with probabilistically triggered arms in combinatorial multi-armed bandits.
method Combinatorial Thompson Sampling (CTS) with a regret bound analysis.
result Derives an O(∑i=1mlogT/(piΔi)) regret bound for CTS. Improved best-arm identification in correlated multi-armed bandits.
problem Best-arm identification in multi-armed bandits with correlated rewards.
method Proposed C-LUCB algorithm that exploits upper bounds on conditional rewards.
result Significant reduction in sample complexity for best-arm identification.
Optimal best-arm identification in linear bandits reduces sampling budget.
problem Identifying the best arm with fixed confidence in stochastic linear bandits.
method A simple algorithm that tracks an optimal proportion of arm draws, updated as rarely as desired.
result The algorithm's sampling complexity matches known lower bounds, asymptotically almost surely and in expectation.
Algorithm reduces regret in restless multi-armed bandits by adaptively sequencing arm choices.
problem Minimizing regret in restless multi-armed bandits with unknown dynamics.
method Adaptive Sequencing Rules (ASR) algorithm that selects arms in a consecutive manner.
result Achieves logarithmic regret order with time and finite-sample bound.
New algorithm for identifying best drug arm in generalized linear bandits.
problem Identifying the best drug arm in drug design with minimal trials.
method Proposes an algorithm for best-arm identification in generalized linear bandits, providing theoretical guarantees and simulations.
result First algorithm for best-arm identification in generalized linear bandits with theoretical guarantees.
New model for multi-armed bandits with growing arms.
problem Balancing exploration and exploitation in a growing set of arms.
method Introduces Ballooning Multi-Armed Bandits (BL-MAB) and analyzes existing algorithms.
result Achieves sub-linear regret under certain conditions.
Study optimizes identifying the best arm with fixed rounds and Gaussian outcomes.
problem Designing efficient experiments to identify the best arm with fixed rounds and Gaussian outcomes.
method Developed worst-case lower bounds and the GNA-EBA strategy for optimal identification.
result GNA-EBA strategy is asymptotically worst-case optimal.
This work optimizes identifying good arms in nonparametric multi-armed bandits.
problem Efficiently identifying arms with high means in nonparametric settings.
method Combining reward-maximizing sampling with a nonparametric sequential test for anytime-valid labeling.
result Achieves minimax optimal stopping times for identifying arms above a threshold.
Develops a framework for clustering and distribution matching with bandit feedback.
problem Clustering and distribution matching problems with limited feedback.
method General framework using K-armed bandit model, Track-and-Stop method, and Frank--Wolfe algorithm. result Average number of arm pulls matches lower bound, with asymptotic convergence to fundamental limit.
New algorithm reduces super-arm selection complexity exponentially.
problem Combinatorial multi-armed bandits with cardinality constraint.
method Combination of group-testing and quantized Thompson sampling.
result Achieves same regret order as state-of-the-art algorithms with perfect oracle, but with reduced complexity.
New method for identifying nearly-optimal arms in bandit models with fixed confidence.
problem Near-optimal arm identification in infinitely armed bandit models with limited prior knowledge.
method PAC-like framework, sample complexity lower bound, algorithm with upper bound on sample complexity.
result Proposed algorithm identifies nearly-optimal arm with high probability and within log factor of lower bound.
This paper studies Thompson sampling's arm-pull dynamics and inference, revealing key differences from UCB algorithms.
problem Understanding the precise arm-pull dynamics in Thompson sampling algorithms.
method Developed new approaches to analyze the arm-pull count process and noise processes, including inverse process and reparametrization methods.
result Arm-pull count is asymptotically deterministic only for suboptimal or unique optimal arms, revealing a unifying principle of stability.
New algorithm identifies best target arm with known additive relationship between source and target MAB instances.
problem Identifying the best arm in a target MAB instance when only source arms can be pulled and there's a known additive relationship between the two.
method Proposes an LUCB-style algorithm to identify an ε-optimal target arm with high probability.
result Theoretical analysis highlights aspects of the transfer learning problem and recovers the LUCB algorithm for single domain BAI as a special case.
A new algorithm solves a regional multi-armed bandit problem with group information.
problem Optimizing decisions with unknown parameters across groups.
method UCB-g algorithm combining UCB and greedy principles.
result Proves the order-optimality of UCB-g and establishes a matching lower bound.
Study best arm identification with limited precision sampling in bandits.
problem Limited precision sampling in multi-armed bandit problems.
method Proposed a modified tracking-based algorithm to handle non-unique optimal allocations and presented non-asymptotic bounds.
result Asymptotically optimal tracking-based algorithm for best arm identification.
The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.
problem Max-quantile group bandit problem with infinitely many arms per group.
method Two-step algorithm: first request arms from each group, then apply a finite-arm max-quantile bandit algorithm.
result Characterization of instance-dependent and worst-case regret, with matching lower bounds.
New algorithm reduces worst-case sample complexity for learning best arm.
problem Identifying a best arm with confidence in multi-armed bandit settings.
method Proposed a new (ε,δ)-PAC learning algorithm for multi-armed bandits. result Algorithm achieves optimal sample complexity for (ε,δ)-learning. Greedy algorithms outperform UCB in many-armed bandit problems.
problem Optimizing multi-armed bandit problems with many arms.
method Subsampled UCB and greedy algorithms comparison.
result Greedy algorithms achieve optimal regret in many-armed bandit problems.
New strategies for identifying the best arm in bandits with decreasing variances.
problem Best arm identification in bandits with time-varying variances.
method Two policies: initial wait followed by continuous sampling, and periodic sampling with weighted average.
result Analytical guarantees and simulations show improved performance over existing methods.
We consider the Max K-Armed Bandit problem, where a learning agent is faced with several sources (arms) of items (rewards), and interested in finding the best item overall. At each time step the agent chooses an arm, and obtains a random real valued reward. The rewards of each arm are assumed to be i.i.d., with an un…
New methods optimize personalized treatment assignment in trials with many arms.
problem Poor performance of standard methods in trials with many treatment arms.
method Regularized and clustered joint assignment forest algorithm.
result Gains in predicting arm-wise outcomes and utility gains from personalization.
Sampling from distributions to find the one with the largest mean arises in a broad range of applications, and it can be mathematically modeled as a multi-armed bandit problem in which each distribution is associated with an arm. This paper studies the sample complexity of identifying the best arm (largest mean) in a m…
The paper tackles infinite-armed bandits with general reservoirs, providing necessary and sufficient conditions for ε-good arm selection.
problem Characterizing conditions for selecting ε-good arms with high probability in infinite-armed bandits.
method The approach is based on successive halving with early discarding of arms after a single pull, analyzing the reservoir distribution without assuming its form.
result The algorithm provably returns an ε-good arm with high probability, explaining the effectiveness of Hyperband's aggressive bracketing.