Algorithm identifies best arm with prior info in structured bandits.
problem Bayesian fixed-budget best-arm identification in structured bandits.
method Prior-dependent allocations based on structure and prior information.
result Improved theoretical bounds and robust performance across diverse models.
A new UCB policy improves reward-cost ratio estimation in budgeted MAB.
problem Maximizing reward-cost ratio under budget constraints.
method ω-UCB policy with asymmetric confidence intervals.
result Logarithmic regret and superior performance in various settings.
The paper develops a theory for identifying the best arm in non-parametric multi-armed bandits with a fixed budget.
problem Identifying the best arm in non-parametric multi-armed bandits with a limited number of trials.
method The paper proposes upper and lower bounds on the average log-probability of misidentification using information-theoretic quantities and a refined analysis of the successive-rejects strategy.
result The paper provides new upper and lower bounds on the average log-probability of misidentification, which generalize existing bounds.
Study resource allocation strategies in sequential decisions with unknown rewards.
problem Sequential resource allocation with unknown rewards.
method Design combinatorial multi-armed bandit algorithms for discrete or continuous budgets.
result Prove algorithms achieve logarithmic cumulative regret under semi-bandit feedback.
New insights into multi-armed bandits with budget constraints.
problem Multi-armed bandits with supply/budget constraints.
method Characterization of logarithmic regret rates, simple regret, and reduction to other bandit problems.
result Full characterization of logarithmic, instance-dependent regret rates for BwK.
HAMLET optimizes algorithm selection for machine learning tasks.
problem Limited time budgets and computational resources make traditional bandit approaches ineffective for automated algorithm selection.
method HAMLET incorporates learning curve extrapolation and time-awareness to select machine learning algorithms.
result HAMLET variants outperform other bandit-based strategies in experiments with recorded hyperparameter tuning traces.
Algorithm allocates budgets to tasks with semi-bandit feedback, achieving near-optimal regret bounds.
problem Stochastic budget allocation with censored semi-bandit feedback.
method Optimism-based algorithm operating under censored semi-bandit feedback.
result Regret scales polylogarithmically with horizon T in diminishing-returns regimes.
Algorithm optimizes a single attribute in multi-armed bandits with constraints.
problem Optimizing a single attribute under multiple constraints in multi-armed bandits.
method Successive Rejects framework, information theoretic lower bound.
result Upper bound on probability of error decays exponentially with budget, nearly optimal in certain cases.
New algorithms for efficient causal interventions with budget constraints and without constraints.
problem Efficiently learning best interventions in causal graphs with budget constraints.
method Developed algorithms for both budgeted and non-budgeted causal bandits, optimizing regret and side-information usage.
result Proposed algorithms minimize cumulative regret and perform better than standard methods.
LinearAPT optimizes decision-making under resource constraints for a linear threshold problem.
problem Optimizing sequential decisions with a linear threshold under resource limitations.
method LinearAPT, an adaptive algorithm for fixed-budget TLB problem.
result LinearAPT achieves theoretical upper bounds and robust performance on various datasets.
The stochastic multi-armed bandit model is a simple abstraction that has proven useful in many different contexts in statistics and machine learning. Whereas the achievable limit in terms of regret minimization is now well known, our aim is to contribute to a better understanding of the performance in terms of identify…
New algorithm reduces regret by allowing free exploration in multi-armed bandits.
problem Designing an adaptive policy to minimize regret with a free exploration budget.
method Introduced (α,β)-probably saving policies and a two-phase algorithm UFE-KLUCB-H. result UFE-KLUCB-H accumulates strictly less regret than non-free exploration policies.
Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.
problem Conservative bandits and reinforcement learning problems.
method Reduction technique to calculate necessary and sufficient budget from baseline policy.
result Improved lower and upper bounds for various conservative settings.
New algorithms identify Pareto optimal sets in multi-objective bandit problems.
problem Identifying Pareto optimal sets in multi-objective bandit problems.
method Empirical Gap Elimination (EGE) algorithms combining hardness estimation and elimination schemes.
result Two EGE algorithms have exponentially decaying error probabilities with budget.
Study on private and robust multi-armed bandits with contaminated heavy-tailed rewards.
problem Private and robust multi-armed bandits with contaminated heavy-tailed rewards.
method Proposed a meta-algorithm with a private and robust mean estimation sub-routine exttt{PRM}.
result Achieved nearly-optimal regret for two heavy-tailed settings.
Neural Index Policy for multi-action bandits with heterogeneous budgets.
problem Real-world settings often involve multiple interventions with heterogeneous costs and constraints, breaking classical assumptions.
method Introduces a Neural Index Policy (NIP) that learns to assign budget-aware indices to arm-action pairs using a neural network and differentiable knapsack layer.
result Empirically achieves near-optimal performance while strictly enforcing heterogeneous budgets and scaling to hundreds of arms.
We study the multi-armed bandit problem with multiple plays and a budget constraint for both the stochastic and the adversarial setting. At each round, exactly K out of N possible arms have to be played (with 1≤K≤N). In addition to observing the individual rewards for each arm played, the player also lea…
TRIPLE efficiently optimizes prompts with a budget constraint.
problem Efficiently selecting good prompts from a pool of candidates.
method TRIPLE connects prompt optimization to best arm identification in MAB, leveraging BAI-FB tools.
result TRIPLE outperforms baselines on multiple tasks with limited budget constraints.
New algorithms for best arm identification in bandits robust to misspecified parameters.
problem Inconsistent learning performance of traditional MAB algorithms when parameters are misspecified.
method Proposes two classes of asymptotically near-optimal algorithms for statistically robust MAB under fixed-budget pure exploration.
result Establishes fundamental performance limits and proposes algorithms that are asymptotically near-optimal.
We address the problem of finding the maximizer of a nonlinear smooth function, that can only be evaluated point-wise, subject to constraints on the number of permitted function evaluations. This problem is also known as fixed-budget best arm identification in the multi-armed bandit literature. We introduce a Bayesian …
We study a generalization of the multi-armed bandit problem with multiple plays where there is a cost associated with pulling each arm and the agent has a budget at each time that dictates how much she can expect to spend. We derive an asymptotic regret lower bound for any uniformly efficient algorithm in our setting. …
We consider the classical stochastic multi-armed bandit problem with a constraint that limits the total cost incurred by switching between actions to be no larger than a given switching budget. For this problem, we prove matching upper and lower bounds on the optimal (i.e., minimax) regret, and provide efficient rate-o…
New algorithm for multi-fidelity bandits reduces costs and improves regret.
problem Optimizing decisions with varying costs and accuracy in multi-fidelity bandits.
method Cost complexity bounds, algorithmic framework, elimination-based algorithm.
result New regret definition and matching upper and lower bounds for elimination-based algorithm.
New method reduces regret in budgeted learning problems.
problem Decision-making with limited reward queries.
method Confidence-Budget Matching (CBM) principle.
result CBM-based algorithms perform well in adversarial settings.
Improved adaptive algorithms for identifying the best arm in MABs with fixed budget.
problem Identifying the best arm in stochastic Multi-Armed Bandits with a fixed sampling budget.
method Established a connection between Large Deviation Principles and adaptive algorithms, improving error probability bounds and devising new algorithms.
result The \sred algorithm outperforms existing algorithms in identifying the best arm.
Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a p…
New algorithms improve privacy in bandit problems with partial information.
problem Privacy constraints in multi-armed bandit problems with partial reward information.
method Proposed a generic framework for designing ε-global DP extensions of UCB and KL-UCB algorithms. result AdaP-KLUCB algorithm achieves optimal regret bound under ε-global DP constraints. Optimal policy for multi-armed multi-action bandits with unknown parameters.
problem Optimal sequential action selection for multi-armed multi-action bandits with unknown parameters.
method Occupancy-Measured-Reward Index Policy (OMRIP) and R(MA)^2B-UCB algorithm.
result Asymptotically optimal policy with sub-linear regret and low computational complexity.
New method balances performance and cost in identifying best arm.
problem Identifying the best arm in multi-armed bandit models.
method Minimizes a risk functional that balances performance and cost.
result Proposes DBCARE algorithm that matches theoretical lower bounds.
Algorithm maximizes user rewards under per-item budget constraints.
problem Maximizing cumulative rewards in collaborative bandits with budget constraints.
method Collaborative algorithm B-LATTICE that clusters users and collaborates across groups.
result Achieves sub-linear regret bounds matching minimax bounds.
Algorithm maximizes rewards with a budget and giving up option.
problem Sequential decision-making with stochastic rewards and resource consumption.
method Upper Confidence Bound (UCB) algorithm for maximizing cumulative reward.
result Logarithmic regret bound with improved dependence on problem parameters.
We consider the best-arm identification problem in multi-armed bandits, which focuses purely on exploration. A player is given a fixed budget to explore a finite set of arms, and the rewards of each arm are drawn independently from a fixed, unknown distribution. The player aims to identify the arm with the largest expe…
We address the M-best-arm identification problem in multi-armed bandits. A player has a limited budget to explore K arms (M<K), and once pulled, each arm yields a reward drawn (independently) from a fixed, unknown distribution. The goal is to find the top M arms in the sense of expected reward. We develop an algorithm …
This paper tackles combinatorial optimization under uncertainty with limited feedback.
problem Tackling combinatorial optimization problems with uncertain or unknown parameters.
method Review of techniques for combinatorial pure exploration with limited bandit feedback.
result Introduction of methods for combinatorial optimization under uncertainty with limited observation.
New dynamic allocation methods for multi-armed bandit models.
problem Dynamic allocation problems in multi-armed bandit models.
method New types of dynamic allocation problems and proofs for Gittins index decomposition.
result New proofs for Gittins index decomposition and related results.
Optimized Sharpe Ratio for better risk-adjusted decision-making in multi-armed bandits.
problem Challenging to optimize Sharpe Ratio (SR) in multi-armed bandits (MAB) due to constant regret.
method Proposed UCB-RSSR algorithm for RSSR maximization, derived path-dependent concentration bound and regret guarantees.
result UCB-RSSR outperforms existing algorithms and finds applications in risk-aware portfolio management.
PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.
problem Improving lifelong learning in multi-armed bandits.
method PAC-Bayesian analysis for deriving lower bounds and proposing lifelong learning algorithms.
result Proposed algorithms outperform baseline methods in lifelong multi-armed bandit problems.
Given a huge set of applicants, how should a firm allocate sequential resume screenings, phone interviews, and in-person site visits? In a tiered interview process, later stages (e.g., in-person visits) are more informative, but also more expensive than earlier stages (e.g., resume screenings). Using accepted hiring mo…
New bandit model for healthcare intervention planning.
problem Maximizing patient health with limited monitoring resources.
method Developed Collapsing Bandits model and derived optimal policies.
result 3-order-of-magnitude speedup in algorithm performance.
SOO uses bandit theory to optimize functions with limited evaluations.
problem Optimizing functions with a limited number of evaluations.
method SOO is a machine learning approach that uses domain partitioning and a tradeoff between exploration and exploitation.
result SOO provides guarantees on the returned solution and is numerically efficient.
Paper uses subjective logic to estimate uncertainty in multi-armed bandit problems.
problem Estimating uncertainty in multi-armed bandit problems.
method Formalism of subjective logic applied to multi-armed bandits, proposing new algorithms.
result Subjective logic quantities enable useful assessment of uncertainty.
Algorithm improves multi-armed bandit performance by transferring reward samples.
problem Sequential multi-armed bandit problem with changing reward distributions.
method UCB algorithm with reward sample transfer.
result Significant improvement in cumulative regret over standard UCB.
Quantum algorithms for multi-armed bandits are explored with limited reward access.
problem Exploring quantum speed-ups in multi-armed bandit problems with limited reward information.
method Introduced new bandit models and showed query complexity equivalence with classical algorithms.
result No quadratic speed-up is possible for multi-armed bandits with limited reward access.
Study on Pareto optimality in multi-objective bandit problems.
problem Pareto optimality in multi-objective multi-armed bandit problems.
method Formulated adversarial multi-objective multi-armed bandit, defined Pareto regrets, presented algorithms, established upper and lower bounds.
result New algorithms are optimal in adversarial settings and nearly optimal in stochastic settings.
ALMAB-DC optimizes expensive black-box experiments using active learning and distributed computing.
problem Efficiently optimizing expensive, gradient-free objectives in computational statistics and machine learning.
method Combines active learning, multi-armed bandits, and distributed asynchronous computing.
result Achieves lower simple regret and superior performance in various tasks compared to non-ALMAB baselines.
BCI system improves word selection efficiency using sequential best-arm identification.
problem Conventional non-adaptive BCI paradigms lead to a lengthy learning process.
method Casted as sequence of best-arm identification tasks in multi-armed bandits, using pre-trained LLMs and STTS algorithm.
result Substantial empirical improvement in word selection efficiency demonstrated.
Unified formulation bridges adversarial and nonstationary bandits.
problem Handling time-varying reward distributions in multi-armed bandit problems.
method Unified oracle that switches between adversarial and nonstationary bandit oracles based on window size.
result Optimal regret achieved with matching lower bound.
New algorithms allocate sampling budget to estimate group means without exploration.
problem Allocate sampling budget to estimate means of multiple groups.
method Design exploration-free non-adaptive and adaptive algorithms.
result Prove tighter regret bounds for multi-group mean estimation.