A new method for dueling bandits improves performance.
problem Improving decision-making in dueling bandits.
method Sup-KLUCB method for K-armed dueling bandits, converting Copeland dueling bandits into standard MAB problems.
result Sup-KLUCB outperforms state-of-the-art methods in Copeland dueling bandits.
Simple algorithms for Dueling Bandits with improved regret bounds.
problem Optimizing decisions in a comparison-based environment with unknown preference gaps.
method Presented simple algorithms with specific regret bounds.
result Regret bounds of order O(T^rho ) with 1/2 <= rho <= 3/4, independent of preference gaps.
New algorithm for dueling bandits with qualitative feedback outperforms existing methods.
problem Qualitative feedback in dueling bandits.
method Direct algorithms using qualitative feedback probabilities.
result Proposed algorithms significantly outperform existing DB algorithms.
Study optimizes dueling bandit learning with a stochastic mirror descent algorithm.
problem Optimizing dueling bandit learning with noisy comparison feedback.
method Proposes a stochastic mirror descent algorithm for continuous dueling bandit.
result Achieves an O ( T log T ) O(\sqrt{T\log T}) O ( T log T ) -regret bound under strong convexity and smoothness assumptions. New algorithm for contextual dueling bandits achieves nearly optimal regret.
problem Contextual dueling bandits with feedback on preferred options.
method Proposes FGTS.CDB, a Thompson sampling algorithm for linear contextual dueling bandits.
result Achieves nearly minimax-optimal regret of i l d e O ( d T ) ilde{\mathcal{O}}(d\sqrt T) i l d e O ( d T ) . New algorithm optimizes dueling bandits for both stochastic and adversarial preferences.
problem Optimizing decision-making in environments where only relative preferences are observed.
method Proposed a reduction from dueling bandits to multi-armed bandits, achieving optimal regret bounds.
result First best-of-both-world result for dueling bandits, optimal regret bound for Condorcet-winner benchmark.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
New algorithms improve dueling bandit performance in multiplayer settings.
problem Challenges in collaborative exploration of non-informative arm pairs in multiplayer dueling bandits.
method Demonstrated Follow Your Leader approach and message-passing fully distributed protocol.
result Multiplayer algorithms outperform single-player benchmarks.
In this paper, we propose a Double Thompson Sampling (D-TS) algorithm for dueling bandit problems. As indicated by its name, D-TS selects both the first and the second candidates according to Thompson Sampling. Specifically, D-TS maintains a posterior distribution for the preference matrix, and chooses the pair of arms…
Algorithm identifies Copeland winners in dueling bandits with ternary feedback.
problem Identifying Copeland winners in dueling bandits with indifferences.
method Proposed POCOWISTA algorithm with a sample complexity close to lower bound.
result Algorithm shows excellent performance, even for conventional dueling bandits.
Solves a new bandit problem with duels and pulls for crowdsourcing.
problem Finding the best arms with mean rewards above a threshold.
method Alternates between ranking and binary search to solve TBP-DC.
result Proves optimality of the Rank-Search algorithm.
Paper tackles dueling bandits with delayed feedback, revealing preference bias.
problem Real-world dueling bandit applications often face delays in feedback.
method Introduces biased dueling bandit problem with stochastic delayed feedback, presents two algorithms.
result Two algorithms achieve optimal regret bounds for dueling bandit problems with delay.
New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of O ( K 2 log 2 ( K ) ) + O ( K log ( T ) ) O(K^2\log^2(K)) + O(K\log(T)) O ( K 2 log 2 ( K )) + O ( K log ( T )) in O ( log ( T ) ) O(\log(T)) O ( log ( T )) rounds. This paper addresses dueling bandits with contextual information, improving regret bounds by accounting for variance.
problem Minimizing cumulative regret in dueling bandits with contextual information.
method Proposes a new SupLinUCB-type algorithm for contextual dueling bandits with variance-aware regret bound.
result Achieves a variance-aware regret bound of i l d e O ( d ∑ t = 1 T σ t 2 + d ) ilde O\big(d\sqrt{\sum_{t=1}^Tσ_t^2} + d\big) i l d e O ( d ∑ t = 1 T σ t 2 + d ) . New algorithm for identifying Condorcet team in noisy comparisons.
problem Online learning with noisy comparisons of teams.
method Formalized dueling teams problem, developed algorithms for stochastic and deterministic settings.
result Identifies Condorcet winning team with reduced number of duels.
Algorithm minimizes regret in non-stationary dueling bandits with unknown parameters.
problem Minimizing regret in dueling bandits with time-varying preferences.
method Proposes Beat the Winner Reset algorithm and meta-algorithms DETECT and Monitored Dueling Bandits.
result Proves bounds on expected weak and strong regret for non-stationary dueling bandits.
The paper minimizes Borda regret in dueling bandits models.
problem Minimizing Borda regret in dueling bandits models.
method Proposes explore-then-commit and EXP3-type algorithms for stochastic and adversarial settings respectively.
result Achieves nearly matching regret upper bounds of O ( d 2 / 3 T 2 / 3 ) O(d^{2/3} T^{2/3}) O ( d 2/3 T 2/3 ) for both settings. The paper tackles learning from imperfect human feedback, especially in dueling bandit problems.
problem Learning from human feedback that can be irrational or imperfect.
method Developed a Robustified Stochastic Mirror Descent for Imperfect Dueling (RoSMID) algorithm.
result Achieved nearly optimal regret for dueling bandit problems under imperfect human feedback.
A new algorithm for conversational recommendation systems using dueling bandits in GLMs.
problem Limited user feedback in existing conversational bandit methods.
method Integrates dueling bandits with relative feedback in generalized linear models.
result Theoretical and empirical validation of ConDuel's efficacy.
New algorithms for batched dueling bandits with improved regret bounds.
problem Batched dueling bandits with noisy pairwise comparisons.
method Developed algorithms for two settings: Condorcet winner and strong stochastic transitivity.
result Regret bounds match sequential bounds using only a logarithmic number of batches.
New algorithm for minimizing regret in adversarial dueling bandits.
problem Minimizing regret in dueling bandits with adversarial feedback.
method Introduced an algorithm with T T T -round regret of i l d e O ( K 1 / 3 T 2 / 3 ) ilde{O}(K^{1/3}T^{2/3}) i l d e O ( K 1/3 T 2/3 ) . result Algorithm achieves Ω ( K 1 / 3 T 2 / 3 ) Ω(K^{1/3}T^{2/3}) Ω ( K 1/3 T 2/3 ) regret lower bound. This paper tackles combinatorial pure exploration for dueling bandits, aiming to find the best candidate-position match.
problem Finding the best candidate-position match in a dueling bandit setting.
method The paper adapts combinatorial pure exploration for multi-armed bandits to dueling bandits, considering both Borda winner and Condorcet winner cases. It designs PAC and exact algorithms for Borda winner and a fully polynomial time approximation scheme (FPTAS) for Condorcet winner.
result The paper introduces the first algorithm with polynomial running time per round for identifying the Condorcet winner in CPE-DB.
Optimizes decision-making in dueling bandits with contextual features.
problem Identifying the best arm in dueling bandits with contextual features.
method Develops algorithms for minimizing regret in stochastic contextual dueling bandits.
result Proves optimal regret bounds for contextual dueling bandits.
Study on tracking preference shifts in dueling bandits problems.
problem Tracking significant preference shifts in dueling bandits problems.
method Analysis of dueling bandits with distribution shifts, focusing on significant shifts (Suk and Kpotufe, 2022).
result Design of adaptive algorithms with O ( K i l d e L T ) O(\sqrt{K ilde{L}T}) O ( K i l d e L T ) dynamic regret for certain preference distribution classes. A dueling bandit problem with resource constraints is solved using EXP3.
problem Maximizing rewards in dueling bandits with resource constraints.
method EXP3 algorithm considering resource consumptions.
result Achieves $ ilde{\mathcal{O}}\left({\frac{OPT^{(b)}}{B}}K^{1/3}T^{2/3}
ight)$ regret.
We study the K K K -armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…
New method identifies Condorcet winner in dueling bandits with improved sample complexity.
problem Identifying Condorcet winner in noisy pairwise comparisons.
method Exploits full gap matrix Δ to improve sample complexity.
result Improves sample complexity guarantees by leveraging informative comparisons.
Algorithm optimizes non-convex functions using dueling comparisons.
problem Optimizing non-convex functions with limited function evaluations.
method COMP-GP-UCB algorithm, leveraging dueling-choice bandits.
result Theoretical guarantee of O ( Φ T ) O(\fracΦ{\sqrt{T}}) O ( T Φ ) on simple regret. We study the K-armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. The hardness of recommending Copeland winners, the arms that beat the greatest number of other arms, is characterized by deriving an asymptotic regr…
Unified framework for best arm identification and dueling bandits regret minimization.
problem Best arm identification and dueling bandits regret minimization.
method Tree-Guided Identify-Then-Exploit (TG-ITE) framework.
result Unified approach achieving optimal sample complexity and regret guarantees.
Improved algorithm for adaptive dueling bandits with near-optimal regret bound.
problem Non-stationary dueling bandits with unknown number of preference changes.
method Elimination-based rescheduling algorithm for adaptive dynamic regret.
result Near-optimal i l d e O ( S e x t t t C W T ) ilde{O}(\sqrt{S^{ exttt{CW}} T}) i l d e O ( S e x ttt C W T ) dynamic regret bound. New ranking algorithms are continually being developed and refined, necessitating the development of efficient methods for evaluating these rankers. Online ranker evaluation focuses on the challenge of efficiently determining, from implicit user feedback, which ranker out of a finite set of rankers is the best. Online …
New algorithms reduce dueling bandits' regret with neural networks and efficient exploration.
problem Optimizing dueling bandits with neural networks for better performance.
method Combines shallow exploration strategies with neural networks for utility approximation, using iterative self-improvement and spectral analysis to reduce network width.
result Achieves sublinear regret of O ~ ( d ∑ t = 1 T σ t 2 + d T ) \widetilde{\mathcal{O}}(d\sqrt{\sum_{t=1}^{T} σ_t^2} + \sqrt{dT}) O ( d ∑ t = 1 T σ t 2 + d T ) . Algorithm minimizes regret in dueling bandits with contextualized utilities.
problem Minimizing regret in dueling bandits with context-dependent utilities.
method Proposes CoLSTIM algorithm based on perturbed utility estimates.
result Achieves regret of order i l d e O ( d T ) ilde O(\sqrt{dT}) i l d e O ( d T ) . Factored bandits model learns with limited feedback using decomposable actions.
problem Limited feedback learning with decomposable actions.
method Introduces factored bandits model, provides anytime algorithm, and matching upper and lower bounds.
result Improves regret bounds for utility-based dueling bandits.
Neural algorithms optimize arm selection with human preference feedback for complex reward functions.
problem Optimizing arm selection with noisy human preference feedback for complex, non-linear reward functions.
method Neural network to estimate reward function using preference feedback, upper confidence bound and Thompson sampling algorithms.
result Sub-linear regret guarantees for efficient arm selection in contextual dueling bandits.
Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently identifying good policies in applications with high feedback costs.
method Introduces offline contextual dueling bandit setting and an upper-confidence-bound style algorithm.
result Proves a regret bound and shows superior performance over uniformly sampled contexts.
Survey of preference-based online learning methods.
problem Learning from relative preferences instead of numerical rewards.
method Analyzes various approaches to dueling bandits.
result Survey of state-of-the-art methods and their assumptions.
Formulates a Dueling Bandits problem for eliciting Kemeny rankings.
problem Eliciting preferences to find a Kemeny ranking.
method Formulates the problem as a Dueling Bandits problem, considering sampling with and without replacement.
result Approximation bounds and algorithms for finding PAC solutions with sample complexity.
PDO optimizes LLM prompts without labels, improving performance.
problem Optimizing prompts for LLMs without access to labeled data.
method Pairwise preference feedback, dueling bandits, Thompson Sampling, mutation.
result PDO identifies stronger prompts than label-free methods.
New algorithm reduces dynamic regret in non-stationary dueling bandits using a weighted Borda score.
problem Designing algorithms with low dynamic regret in non-stationary dueling bandits.
method Introducing a novel weighted Borda score framework to analyze the Condorcet problem and establish improved bounds.
result First optimal and adaptive dynamic regret upper bound of i l d e O ( i l d e L 1 / 3 K 1 / 3 T 2 / 3 ) ilde{O}( ilde{L}^{1/3} K^{1/3} T^{2/3} ) i l d e O ( i l d e L 1/3 K 1/3 T 2/3 ) . GL-LowPopArt improves minimax-optimal estimation for trace regression.
problem Minimizing estimation error in generalized low-rank trace regression.
method Two-stage approach: nuclear norm regularization followed by matrix Catoni estimation.
result Achieves instance-wise optimal error bounds up to condition number.
The dueling bandit problem is a variation of the classical multi-armed bandit in which the allowable actions are noisy comparisons between pairs of arms. This paper focuses on a new approach for finding the "best" arm according to the Borda criterion using noisy comparisons. We prove that in the absence of structural a…
Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently acquiring human feedback for preference alignment in large language models.
method Formalizes active exploration as a dueling bandit problem and proposes an active exploration algorithm with a polynomial worst-case regret bound.
result Proposed method outperforms baselines with limited human preferences on various language models and datasets.
Paper analyzes finite-time guarantees for preference-based RL.
problem Understanding finite-time guarantees for preference-based RL.
method Combines dueling bandits and policy search to navigate state space.
result Identifies best policy up to accuracy ε with high probability.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
New algorithm reduces exploration in structured stochastic bandits.
problem Wide class of stochastic bandit problems with known structural properties.
method Developed OSSB algorithm that matches minimal exploration rates of sub-optimal arms.
result OSSB's regret matches asymptotic instance-specific regret lower bound.
PAC Battling-Bandit tackles online learning with subset choice and Plackett-Luce feedback.
problem Identify near-best items in a PL model with subset choice and stochastic feedback.
method Introduces PAC Battling-Bandit problem, studies various feedback models, proposes algorithms with optimal sample complexity.
result Sample complexity is $O\left( \frac{n}{ε^2} \ln \frac{1}δ
ight)$ for WI feedback, Ω ( n m ε 2 ln 1 δ ) Ω\bigg( \frac{n}{mε^2} \ln \frac{1}δ\bigg) Ω ( m ε 2 n ln δ 1 ) for TR feedback.