Paper refines ExtremeHunter algorithm for detecting extreme values.
problem Max K-armed bandit problem, focusing on extreme value detection.
method Significantly refines ExtremeHunter algorithm and proposes an alternative approach.
result Extreme Bandits can be reduced to a classical bandit problem.
New algorithm improves materials discovery using max K-Armed Bandit.
problem Maximizing material breakthroughs in materials discovery.
method Proposed a search algorithm based on max K-Armed Bandit (MKB) for materials discovery.
result Stable performance in late search stages, outperforming other bandit algorithms.
We consider the Max K K K -Armed Bandit problem, where a learning agent is faced with several sources (arms) of items (rewards), and interested in finding the best item overall. At each time step the agent chooses an arm, and obtains a random real valued reward. The rewards of each arm are assumed to be i.i.d., with an un…
We consider the Max K K K -Armed Bandit problem, where a learning agent is faced with several stochastic arms, each a source of i.i.d. rewards of unknown distribution. At each time step the agent chooses an arm, and observes the reward of the obtained sample. Each sample is considered here as a separate item with the rewa…
New algorithms identify top K arms with high confidence.
problem Identifying K arms with the largest means from N stochastic bandit arms.
method Proposed new practical algorithms with nearly optimal sample complexity.
result Outperform state-of-the-art algorithms in practical applications.
Paper optimizes sample selection for top-k arms in stochastic bandits.
problem Identifying the k arms with the largest means in stochastic bandits.
method Developed an elimination-based algorithm with sample complexity matching lower bounds.
result Algorithm strictly dominates state-of-the-art for Best-k-Arm problem.
New approach to multi-armed bandit problem aims to maximize highest total reward.
problem Traditional multi-armed bandit problem objective of maximizing total reward is not suitable in certain applications.
method Adaptive explore-then-commit policy with confidence bounds and adaptive stopping criterion.
result Achieves asymptotic and worst-case regret bounds for the new objective.
Bandit algorithms handle human-like decision-making distortions.
problem Emulating human decision-making with probabilistic distortions.
method Stochastic multi-armed bandit problems with distorted probabilities, incorporating reward distortions.
result Sublinear regret for proposed algorithms in both K K K -armed and linear bandit settings. New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of O ( K 2 log 2 ( K ) ) + O ( K log ( T ) ) O(K^2\log^2(K)) + O(K\log(T)) O ( K 2 log 2 ( K )) + O ( K log ( T )) in O ( log ( T ) ) O(\log(T)) O ( log ( T )) rounds. New trade-off found in bandit problems with unknown range.
problem Stochastic bandit problems with unknown range.
method Exhibit a strategy achieving new trade-off between distribution-dependent and distribution-free regret bounds.
result Achieves the rates for regret indicated by the new trade-off.
New definitions and algorithms improve sample complexity for identifying good arms in multi-armed bandits.
problem Identifying arms with means close to the maximum or larger than a threshold.
method Formalized definitions, lower bounds, and practical algorithms.
result Achieved nearly matching upper bounds with sample complexities of Θ( rac{n}{m}) and Θ( rac{n}{m}k).
New algorithm identifies optimal subtrees in fixed-budget tree search.
problem Identifying optimal subtrees in fixed-budget Monte Carlo Tree Search.
method ε-agnostic algorithm for max-min action identification.
result Misidentification probability decays exponentially with sample size.
New algorithms for batched dueling bandits with improved regret bounds.
problem Batched dueling bandits with noisy pairwise comparisons.
method Developed algorithms for two settings: Condorcet winner and strong stochastic transitivity.
result Regret bounds match sequential bounds using only a logarithmic number of batches.
A new bandit framework reduces K-armed to C+1-armed, achieving lower regret.
problem Designing efficient algorithms for correlated multi-armed bandits.
method Generalized UCB algorithm exploiting latent random source correlation.
result Achieves O ( 1 ) \mathcal{O}(1) O ( 1 ) regret for certain regimes, reducing from logarithmic. Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.
problem Adversarial bandit problem with delayed, composite anonymous feedback.
method Propose wrapper algorithm for non-oblivious delay setting, achieving o ( T ) o(T) o ( T ) policy regret. result Achieve o ( T ) o(T) o ( T ) policy regret for many adversarial bandit problems with bounded memory loss sequences. A new approach to hedging using contextual bandit models outperforms traditional methods.
problem Effective replication of financial contracts in incomplete markets with low transaction costs.
method Viewing hedging as a contextual k k k -armed bandit problem, using reinforcement learning. result The contextual bandit model provides more accurate and sample-efficient hedging than Q Q Q -learning. New algorithm reduces high-probability regret for time-varying feedback graphs.
problem High-probability regret bounds for adversarial bandits with time-varying feedback graphs.
method Online mirror descent framework with innovative techniques for pessimistic loss estimators.
result Achieves optimal high-probability regret bound for general and weakly observable graphs.
Develops a framework for clustering and distribution matching with bandit feedback.
problem Clustering and distribution matching problems with limited feedback.
method General framework using K K K -armed bandit model, Track-and-Stop method, and Frank--Wolfe algorithm. result Average number of arm pulls matches lower bound, with asymptotic convergence to fundamental limit.
Study on policy gradient for stochastic bandits using diffusion approximation.
problem Improving policy gradient methods for stochastic bandits with optimal regret bounds.
method Continuous-time diffusion approximation of policy gradient with learning rate analysis.
result Proved optimal regret bound of O ( k log ( k ) log ( n ) / η ) O(k \log(k) \log(n) / η) O ( k log ( k ) log ( n ) / η ) for η = O ( Δ 2 / log ( n ) ) η= O(Δ^2/\log(n)) η = O ( Δ 2 / log ( n )) . Short note on soft-max and policy gradients in bandit problems using Lyapunov functions.
problem Analyzing soft-max and policy gradient methods in bandit problems.
method Lyapunov function argument for soft-max and differential equations for policy gradient algorithms.
result Regret bounds for soft-max and a different policy gradient algorithm in bandit problems.
The paper optimizes identifying top k k k arms from a fraction of ρ ρ ρ arms in stochastic bandits.
problem Identifying k k k distinct arms among the top ρ ρ ρ fraction of arms in stochastic bandits with a PAC tolerance. method The paper considers two cases: known and unknown threshold of top arms' expected rewards. It proves lower bounds and proposes algorithms for each case, showing sample complexity optimality for two algorithms.
result Two algorithms are sample complexity optimal (up to constant factors) and the other two are optimal up to a log factor.
Top-two algorithm improved for best-k-arm selection.
problem Best-k-arm identification in multi-armed bandits.
method Information-directed selection based on dual variables.
result Top-two Thompson sampling with IDS is asymptotically optimal.
New bandit problem for finding best group of arms with worst mean reward.
problem Finding the best group of arms with the worst mean reward in overlapping groups.
method Two algorithms based on successive elimination and robust optimization.
result Upper bounds on the number of samples to find max-min optimal or near-optimal group.
We study the K K K -armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…
KL-UCB+ policy outperforms KL-UCB empirically in stochastic bandits.
problem Optimizing decisions in a stochastic bandit problem.
method Demonstrates a simple proof of asymptotic optimality for KL-UCB+ policy.
result KL-UCB+ policy achieves asymptotically optimal regret bound.
Optimism stabilizes Thompson Sampling for adaptive inference in multi-armed bandits.
problem Subtle inferential properties of Thompson Sampling under adaptive data collection.
method Introduced optimism as a key mechanism to restore stability and validity of inference.
result Suitably implemented optimism stabilizes Thompson Sampling and enables asymptotically valid Wald inference.
Analyze nonparametric contextual bandits with improved UCB algorithm.
problem Optimizing decisions based on noisy context signals.
method Modified UCB algorithm with k k k NN-UCB, global intrinsic dimension dependent and ambient dimension independent regret bounds. result Sublinear regret of O ~ ( T 1 + D 2 + D ) \widetilde{O}\Big(T^{\frac{1+D}{2+D}}\Big) O ( T 2 + D 1 + D ) for top-arm identification. Improved UCB algorithm for diversity in bandits with lower bounds.
problem Improving diversity in bandit-based recommendations.
method Designing a UCB algorithm exploiting the specific structure of the problem.
result The UCB algorithm achieves bounded distribution-dependent regret under certain conditions.
This review examines bandit problems in AI using statistical methods.
problem Sequential decision-making under uncertainty in AI environments.
method Foundational models, concentration inequalities, minimax regret bounds, frequentist and Bayesian algorithms, K-armed contextual bandits, SCAB, functional data analysis.
result Exploration-exploitation trade-offs and regret analyses in various bandit problems.
Unified theory for UCB policies in total and max bandit problems.
problem Order optimality of UCB policies in max bandit problems.
method Unified definition of UCB policy using oracle quantity and failure count.
result UCB policies are order optimal in both total and max bandit problems.
Study optimizes dynamic pricing in auctions with unknown buyer valuations.
problem Maximizing revenue in auctions with unknown buyer valuations.
method Stochastic K K K -armed bandit problem approach, proving lower and upper bounds on regret. result Upper bounds on regret for different settings, including distribution-dependent and distribution-free cases.
New algorithm reduces dueling bandit regret, efficiently recommending winning arms.
problem Finding the best arms in dueling bandit problems.
method Proposed CW-RMED and ECW-RMED algorithms with asymptotic regret bounds.
result ECW-RMED efficiently recommends winning arms with optimal regret bound.
The paper tackles non-stationary MAB with periodic rewards.
problem Non-stationary mean rewards over time in a business context.
method Combines Fourier analysis with confidence-bound learning to estimate periods and minimize regret.
result Proposes a near-optimal policy with a regret bound of O ( T ∑ k = 1 K T k ) O(\sqrt{T\sum_{k=1}^K T_k}) O ( T ∑ k = 1 K T k ) . A collaborative algorithm reduces regret in federated linear contextual bandits.
problem Optimizing decision-making in federated learning with heterogeneous data.
method Fed-PE algorithm, leveraging geometric structure of rewards, multi-client G-optimal design.
result Achieves near-optimal regrets with logarithmic communication costs.
A new bandit algorithm observes arm rewards before playing, reducing regret.
problem Balancing exploration and exploitation with pre-observation costs.
method Design of OBP-UCB for single-player and C-MP-OBP for multi-player settings.
result Proved regret bounds for both single-player and multi-player settings.
A new algorithm for selecting top-k arms in extreme contextual bandits with improved efficiency.
problem Selecting top-k arms from a large set with contextual information and limited rewards.
method Proposes an algorithm for both non-extreme and extreme settings, using Inverse Gap Weighting and arm hierarchy models.
result Achieves improved regret guarantees for extreme settings with significant computational and statistical efficiency.
We consider the stochastic bandit problem in the sublinear space setting, where one cannot record the win-loss record for all K K K arms. We give an algorithm using O ( 1 ) O(1) O ( 1 ) words of space with regret \[ \sum_{i=1}^{K}\frac{1}{Δ_i}\log \frac{Δ_i}Δ\log T \] where Δ i Δ_i Δ i is the gap between the best arm and arm i i i and Δ Δ Δ is …
New algorithms improve multi-task bandit performance by transferring reward samples.
problem Sequential multi-task bandit problems with adjacent similar tasks.
method Two UCB-based algorithms that transfer reward samples between tasks.
result Transfer of reward samples reduces overall regret compared to no transfer.
A new algorithm reduces communication costs for collaborative decision-making across clients.
problem Collaborative decision-making with sparse rewards and heterogeneous contexts.
method Federated Lasso algorithm for sparse linear contextual bandits.
result Achieves near-optimal regret with logarithmic communication costs.
ODC protocol improves learning in asynchronous multi-agent bandits.
problem Cooperative multi-agent bandits with unknown, irregular pull times.
method ODC protocol for on-demand communication between agents.
result ODC protocol reduces communication complexity based on empirical pull times.
A new algorithm reduces regret in bandit problems with adversarial corruptions.
problem Optimizing decision-making in bandit problems with variable uncertainties and adversarial interference.
method Proposes HCW-GLB-OMD, an OMD-based estimator with Hessian-based confidence weights for robustness.
result Achieves instance-wise minimax optimality with a κ κ κ -factor in the corruption term. KL-UCB-switch optimizes bandit strategies for both distribution-dependent and distribution-free performance.
problem Optimizing regret bounds for stochastic bandits.
method Combining MOSS and KL-UCB strategies.
result Achieves both optimal distribution-dependent and distribution-free regret bounds.
Improved bounds for adversarial bandits with variance analysis.
problem Optimizing decision-making in adversarial environments with varying gaps.
method First-order bounds, variance analysis, gap-dependent bounds for follow the regularized leader.
result Improved bounds for adversarial bandits with a factor of log(n)/log(log(n)) improvement.
New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
Algorithm identifies Pareto front in multi-objective bandits efficiently.
problem Sequentially learning the Pareto front in multi-objective bandits.
method Efficient algorithm achieving optimal sample complexity.
result Correct answer with high probability in minimal rounds.
Improved algorithm for bandits with delayed feedback, combining adversarial and stochastic performance.
problem Adversarial and stochastic multiarmed bandits with delayed feedback.
method Modified Zimmert and Seldin's algorithm with near-optimal regret guarantees.
result Near-optimal regret guarantees in both adversarial and stochastic settings.
Unified approach for non-stationary linear bandits with dynamic regret.
problem Non-stationary linear bandits with round-specific feasible actions and drifting reward models.
method Unified misspecification-reduction viewpoint, restarting algorithms with misspecification-dependent regret guarantees.
result Optimal \(T^{2/3}P_T^{1/3}\) dynamic-regret dependence for both linear bandits and contextual linear bandits.
The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.
problem Max-quantile group bandit problem with infinitely many arms per group.
method Two-step algorithm: first request arms from each group, then apply a finite-arm max-quantile bandit algorithm.
result Characterization of instance-dependent and worst-case regret, with matching lower bounds.