Thompson Sampling tackles USS, a sequential selection problem without feedback.
problem Unsupervised Sequential Selection (USS) problem with fixed costs and ordered arms.
method Thompson Sampling algorithm for USS problem.
result Thompson Sampling achieves near optimal regret and better performance than existing algorithms.
New algorithm reduces super-arm selection complexity exponentially.
problem Combinatorial multi-armed bandits with cardinality constraint.
method Combination of group-testing and quantized Thompson sampling.
result Achieves same regret order as state-of-the-art algorithms with perfect oracle, but with reduced complexity.
Algorithm identifies best arm with biased proxy and selective ground truth audits.
problem Fixed-confidence best-arm identification with biased proxy and selective ground truth.
method Propensity-weighted estimator and adaptive auditing algorithm.
result Plug-in Neyman rule achieves near-oracle audit efficiency.
Proposes a max-utility arm selection strategy for reducing cumulative regret in sequential query recommendations.
problem Reduces cumulative regret in sequential query recommendations for closed loop interactive learning settings.
method Proposes a max-utility arm selection strategy based on the maximum utility of arms.
result Improves cumulative regret substantially compared to baseline algorithms and random selection.
Algorithm learns optimal arm selection in unsupervised sequential selection with contextual information.
problem Learning optimal arm selection in unsupervised sequential selection with contextual information.
method Proposes an algorithm for the contextual USS problem under the CWD property, demonstrating sub-linear regret.
result Demonstrates sub-linear regret for the proposed algorithm.
Paper tackles risk-aware portfolio selection using multi-armed bandit.
problem Sequential portfolio selection under uncertainty.
method Incorporates risk-awareness into multi-armed bandit, constructs portfolio through asset filtering and risk minimization.
result Achieves balance between risk and return.
Improved theoretical guarantees for Top Two algorithms.
problem Theoretical support for best arm identification with bounded distributions.
method General analysis of Top Two methods, identifying desirable properties and replacing sampling step.
result Theoretical support for Top Two algorithms with bounded distributions.
Top-two algorithm improved for best-k-arm selection.
problem Best-k-arm identification in multi-armed bandits.
method Information-directed selection based on dual variables.
result Top-two Thompson sampling with IDS is asymptotically optimal.
Paper optimizes sample selection for top-k arms in stochastic bandits.
problem Identifying the k arms with the largest means in stochastic bandits.
method Developed an elimination-based algorithm with sample complexity matching lower bounds.
result Algorithm strictly dominates state-of-the-art for Best-k-Arm problem.
New algorithm identifies best arm in rested bandit setting.
problem Best arm identification in rested bandit with decreasing losses.
method Introduced a novel best arm identification problem and analyzed an arm elimination algorithm.
result Regret vanishes as time horizon increases, with convergence rate depending on expected loss function.
Algorithm selects k arms from context-dependent options using Plackett-Luce model.
problem Selecting k arms from context-dependent options with Plackett-Luce feedback.
method Proposes CPPL algorithm inspired by UCB, evaluated on synthetic and real data.
result Demonstrates effectiveness of CPPL algorithm in online algorithm selection.
Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.
problem Identify the best arm in a multi-armed bandit with time-varying states.
method Propose a sequential policy to select arms without knowing their exact TPMs.
result Upper and lower bounds on expected time to find the best arm match in a special case.
A new algorithm for selecting top-k arms in extreme contextual bandits with improved efficiency.
problem Selecting top-k arms from a large set with contextual information and limited rewards.
method Proposes an algorithm for both non-extreme and extreme settings, using Inverse Gap Weighting and arm hierarchy models.
result Achieves improved regret guarantees for extreme settings with significant computational and statistical efficiency.
New method selects best exploration strategies in uncertain environments.
problem Selecting optimal strategies in unknown, multi-strategy environments.
method Formulates Multi-Armed Bandits problem with diversity of effects as reward signal.
result Method outperforms fixed mixtures of strategies in diverse, challenging conditions.
Algorithm reduces regret in restless multi-armed bandits by adaptively sequencing arm choices.
problem Minimizing regret in restless multi-armed bandits with unknown dynamics.
method Adaptive Sequencing Rules (ASR) algorithm that selects arms in a consecutive manner.
result Achieves logarithmic regret order with time and finite-sample bound.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
Proposes DEXP3.M for unknown delay in multi-arm bandit with multiple play.
problem Unknown delays in adversarial multi-armed bandit with multiple play.
method DEXP3.M algorithm addressing the challenge of associating feedback losses to arms.
result Regret bound is only slightly worse than single play setting.
New method selects both algorithm and hyperparameters for data analysis.
problem Selecting the right algorithm and its hyperparameters for data analysis.
method Reduced to multi-armed bandit problem, where algorithms are arms and hyperparameters search is arm play. Suggested reward function for optimization.
result Significantly better performance compared to existing Auto-WEKA method on 10 real datasets.
Two novel methods identify influential features in CMABs for better reward distribution.
problem Suboptimal features degrade rewards, interpretability, and efficiency in CMABs.
method Heterogeneous Incremental Effect (HIE) and Heterogeneous Distribution Divergence (HDD) methods.
result Consistent ability to identify influential HTE features, enhancing CMAB performance.
A novel algorithm reduces communication costs in federated best arm identification.
problem Identifying the best arm in a federated multi-armed bandit setup with minimal communication cost.
method Proposes a novel algorithm called FedElim that communicates only in exponential time steps.
result Demonstrates that communication is almost cost-free in FedElim, with a total cost at most 3 times the maximum under its variant.
Neural algorithms optimize arm selection with human preference feedback for complex reward functions.
problem Optimizing arm selection with noisy human preference feedback for complex, non-linear reward functions.
method Neural network to estimate reward function using preference feedback, upper confidence bound and Thompson sampling algorithms.
result Sub-linear regret guarantees for efficient arm selection in contextual dueling bandits.
DART optimizes subset selection in non-linear bandit problems.
problem Optimizing subset selection in non-linear bandit problems with correlated rewards.
method DART algorithm for combinatorial bandits without individual arm feedback or linearity assumption.
result DART achieves a regret bound of i l d e O ( K K N T ) ilde{\mathcal{O}}(K\sqrt{KNT}) i l d e O ( K K N T ) . New algorithm finds best arm in linear bandits with adaptive selection.
problem Finding the arm with the highest expected reward in linear bandits with unknown parameters.
method Fully adaptive algorithm that changes arm selection based on past observations.
result Sample complexity matches lower bound in extreme case.
A new algorithm reduces suboptimal arm selection in correlated bandits.
problem Structured bandits with correlated rewards.
method Confidence-based phased algorithm.
result Regret is uniformly bounded in certain structures.
TRIPLE efficiently optimizes prompts with a budget constraint.
problem Efficiently selecting good prompts from a pool of candidates.
method TRIPLE connects prompt optimization to best arm identification in MAB, leveraging BAI-FB tools.
result TRIPLE outperforms baselines on multiple tasks with limited budget constraints.
Study best arm identification with limited precision sampling in bandits.
problem Limited precision sampling in multi-armed bandit problems.
method Proposed a modified tracking-based algorithm to handle non-unique optimal allocations and presented non-asymptotic bounds.
result Asymptotically optimal tracking-based algorithm for best arm identification.
BAICS identifies best arm with fairness constraints on subpopulations.
problem Identify the best arm while ensuring fairness across subpopulations.
method Formulated and solved BAICS problem, analyzed complexity, designed algorithm.
result Algorithm's sample complexity matches theoretical lower bound.
A new algorithm THV-UCB reduces regret in multi-objective bandit problems.
problem Maintaining a small set of actions that jointly approximate the Pareto frontier in multi-objective slate selection.
method THV-UCB, an optimistic algorithm that selects arms based on optimistic estimates of their marginal hypervolume contributions.
result The algorithm achieves a gap-free regret bound of i l d e O ( d n k T ) ilde{O}(d\sqrt{nkT}) i l d e O ( d nk T ) and a gap-dependent bound of i l d e O ( n k 2.5 / Δ min ) ilde{O}(nk^{2.5}/Δ_{\min}) i l d e O ( n k 2.5 / Δ m i n ) . HAMLET optimizes algorithm selection for machine learning tasks.
problem Limited time budgets and computational resources make traditional bandit approaches ineffective for automated algorithm selection.
method HAMLET incorporates learning curve extrapolation and time-awareness to select machine learning algorithms.
result HAMLET variants outperform other bandit-based strategies in experiments with recorded hyperparameter tuning traces.
Optimal strategy found for constrained multi-armed bandit problems.
problem Constrained multi-armed bandit problems.
method Extended ε_t-greedy strategy with asymptotic optimality.
result Asymptotic optimality achieved with a simple strategy.
Efficiently identifies best algorithms for game tasks.
problem Selecting optimal algorithms for game tasks efficiently.
method Best arm identification for multi-armed bandits with confidence intervals.
result Significantly improved performance in simple regret and error probability.
Paper tackles anomaly detection in restless Markov arms with unknown TPMs.
problem Detecting an anomalous arm in a multi-armed bandit with unknown transition probability matrices.
method Developed a policy based on the principle of certainty equivalence, achieving the lower bound arbitrarily closely under specific assumptions.
result Achieved the lower bound on expected time required to find the odd arm index, demonstrating the policy's effectiveness.
Unified algorithm for efficient pure exploration using dual variables.
problem Efficiently achieving a specific goal through adaptive experimentation.
method Introducing dual variables to derive optimal allocation conditions, leading to Information-Directed Selection.
result Top-two Thompson sampling attains asymptotic optimality for Gaussian best-arm identification.
Study uses randomized allocation for delayed rewards in multi-armed bandits.
problem Delayed rewards in contextual multi-armed bandits.
method Randomized allocation with nonparametric estimation.
result Strongly consistent strategy for delayed rewards.
Study on a bad arm existence checking problem to minimize arm draws.
problem Judging the existence of a positive arm among K given arms.
method Proposes an algorithm with arm selection policy and stopping condition.
result Proves the effectiveness of the proposed algorithm theoretically and empirically.
The paper examines how insurers can select claims for fraud investigation, proposing a randomized approach.
problem Inconsistent learning from biased claim selection.
method Formalizes selection in binary regression, proposes a randomized alternative, and defines consistency.
result The randomized selection strategy is consistent, while the traditional strategy is not.
New algorithm SELECT minimizes satisficing regret in bandits.
problem Minimizing regret in bandit optimization with satisficing arms.
method SELECT algorithm for satisficing regret minimization.
result SELECT achieves constant expected satisficing regret.
Paper introduces risk-sensitive bandits with optimal arm mixtures.
problem Designing algorithms for risk-sensitive multi-armed bandits.
method Formalizes risk-sensitive bandits framework, identifies optimal arm mixtures, designs regret-efficient algorithms.
result Regret-efficient algorithms track optimal arm mixtures or solitary arms.
New algorithms ensure fair selection in combinatorial semi-bandit with unrestricted delays.
problem Fair selection in stochastic combinatorial semi-bandit with delayed feedback.
method Introduced merit-based fairness constraints and new bandit algorithms for reward and fairness.
result Achieved sublinear expected reward and fairness regrets with dependence on delay distribution quantiles.
New algorithm for combinatorial bandit problems reduces regret.
problem Optimal selection of sets of arms in bandit problems.
method SGB algorithm with optimized exploration of unselected arms.
result Achieves ( 1 − 1 / e ) (1-1/e) ( 1 − 1/ e ) -regret bound of O ( n 1 3 k 2 3 T 2 3 log ( T ) 2 3 ) \mathcal{O}(n^{\frac{1}{3}} k^{\frac{2}{3}} T^{\frac{2}{3}} \log(T)^{\frac{2}{3}}) O ( n 3 1 k 3 2 T 3 2 log ( T ) 3 2 ) . The paper develops algorithms to minimize misallocation and identify the arm with the highest variance.
problem Minimizing misallocation and identifying the arm with the highest variance from a set of arms.
method Developed novel online algorithms UCB-VV for misallocation minimization and SHVV for fixed budget best arm identification.
result The algorithms achieve optimal performance in terms of misallocation and error probability.
We extend an offline evaluation method for continuous-armed bandits.
problem Evaluating policies in continuous-armed bandits with limited data.
method Extend existing offline evaluation method for continuous-armed bandits.
result Our method provides a consistent ranking of policies.
New findings show that multi-armed bandit methods can't guarantee correct population selection with finite samples.
problem Selecting the best population or probability distribution from many with unknown means.
method Adapting sequential methods from stochastic multi-armed bandit literature.
result Infinite number of samples are required for correct population selection with finite probability.
Con-TS optimizes wireless link throughput with latency constraints.
problem Optimizing rate selection for wireless links with latency constraints.
method Proposes Con-TS, a constrained Thompson sampling algorithm for stochastic MAB problems.
result Con-TS achieves upper bounds on expected constraint violations and throughput loss.
New framework shifts bandit algorithms from expected reward to preference metrics, optimizing mixtures of arms.
problem Traditional bandit algorithms focus on expected rewards, ignoring variability and risk.
method Introduces preference metrics (PMs) and designs algorithms to optimize mixtures of arms.
result Optimal policy selects mixtures of arms based on specific weights, not a single best arm.
Hedge improves decision-making in resource selection problems.
problem Resource selection and distribution problems in networks and transportation.
method Hedge as an adaptive scheme with upper bounded total loss.
result Worst performance of Hedge studied under bounded loss.
A new algorithm for cryo-EM data collection that balances reward and latency.
problem Optimizing data collection in cryo-EM experiments with action delays.
method Latency-aware contextual bandit framework and COAF algorithm.
result The COAF algorithm efficiently maximizes reward over time in cryo-EM experiments.
The paper tackles identifying multiple best arms in stochastic bandits, improving sample efficiency.
problem Identifying multiple best arms in stochastic multi-armed bandits.
method Developed a fully sequential PAC algorithm, GLUCB, and presented lower and upper bounds.
result Improved sample efficiency for identifying multiple best arms.