Study on Pareto optimality in multi-objective bandit problems.
problem Pareto optimality in multi-objective multi-armed bandit problems.
method Formulated adversarial multi-objective multi-armed bandit, defined Pareto regrets, presented algorithms, established upper and lower bounds.
result New algorithms are optimal in adversarial settings and nearly optimal in stochastic settings.
OSOM solves multi-armed and linear contextual bandits efficiently.
problem Simultaneously optimal algorithm for multi-armed and linear contextual bandits.
method Design of a single computationally efficient algorithm that adapts to both regimes.
result Simultaneously optimal regret rates in both simple multi-armed and linear contextual bandits.
Study finds optimal regret bound for multi-armed bandit problem with expert advice.
problem Optimizing decision-making in a multi-armed bandit problem with expert advice.
method Proved a tight lower bound matching the upper bound of Kale (2014) for minimax expected regret.
result The minimax optimal expected regret is Θ(√(T K log (N/K))) for the problem.
Paper uses subjective logic to estimate uncertainty in multi-armed bandit problems.
problem Estimating uncertainty in multi-armed bandit problems.
method Formalism of subjective logic applied to multi-armed bandits, proposing new algorithms.
result Subjective logic quantities enable useful assessment of uncertainty.
Optimal regret achieved in stochastic, discrete multi-armed bandits using information-theoretic exploration.
problem Optimal exploration vs. exploitation in stochastic, discrete multi-armed bandits.
method Proposes an information-theoretic strategy based on the value of information criterion, using simulated-annealing-like updates of a parameter.
result Achieves logarithmic optimal regret with respect to the number of episodes.
Two non-communicating players minimize regret in a multi-armed bandit game.
problem Optimal regret in non-communicating multi-armed bandit players.
method Proposed a strategy with no collisions, achieving near-optimal regret.
result Near-optimal regret of O ( T log ( T ) ) O(\sqrt{T \log(T)}) O ( T log ( T ) ) with very high probability. BaSE policy optimizes multi-armed bandits with batched data.
problem Optimizing multi-armed bandits with batched data.
method BaSE (batched successive elimination) policy for batched multi-armed bandits.
result Achieves rate-optimal regrets with adaptive batch sizes.
Unified analysis of perturbation-based strategies in stochastic and adversarial bandit problems.
problem Optimality of perturbation-based strategies in multi-armed bandit problems.
method Unified regret analysis for stochastic and adversarial settings, using perturbations of sub-Weibull and bounded support.
result Unified bounds for perturbations in both stochastic and adversarial settings, with optimal perturbations of Frechet-type.
Paper tackles risk-aware portfolio selection using multi-armed bandit.
problem Sequential portfolio selection under uncertainty.
method Incorporates risk-awareness into multi-armed bandit, constructs portfolio through asset filtering and risk minimization.
result Achieves balance between risk and return.
The study analyzes batched methods for early stopping in stochastic multi-armed bandits.
problem Early stopping in stochastic multi-armed bandits with fixed confidence.
method Instance-dependent lower bounds and a general batched algorithm with upper bounds.
result Upper and lower bounds on the number of batches and sample complexity.
Algorithm improves multi-armed bandit performance by transferring reward samples.
problem Sequential multi-armed bandit problem with changing reward distributions.
method UCB algorithm with reward sample transfer.
result Significant improvement in cumulative regret over standard UCB.
Paper tackles good arm identification in stochastic bandits.
problem Identifying good arms with minimal samples.
method Proposes DGAI, a differentiable algorithm to improve sample complexity.
result DGAI outperforms baseline algorithms in synthetic and real-world datasets.
PHE adds pseudo-rewards to history to minimize regret in stochastic bandits.
problem Minimizing cumulative regret in stochastic multi-armed bandits.
method PHE algorithm that adds O ( t ) O(t) O ( t ) i.i.d. pseudo-rewards to history and pulls the best arm based on the perturbed history. result Near-optimal regret bounds derived for PHE.
We consider the problem of minimizing the regret in stochastic multi-armed bandit, when the measure of goodness of an arm is not the mean return, but some general function of the mean and the variance.We characterize the conditions under which learning is possible and present examples for which no natural algorithm can…
New model for display advertising with stochastic and adversarial components.
problem Display advertising with stochastic and adversarial click-through-rates.
method Adversarial scaling model; two algorithms tested: action elimination and mirror descent.
result Two algorithms are robust to adversarial scaling.
New algorithm uses control variates to improve multi-armed bandit performance.
problem Stochastic multi-armed bandits with auxiliary reward information.
method Developed UCB-CV algorithm using control variates for mean estimation.
result UCB-CV algorithm provides tighter confidence bounds and smaller variance.
Study collaborative learning among multi-agents in multi-armed bandits.
problem Minimizing group cumulative regret in a heterogeneous multi-agent setting.
method Developed decentralized algorithms for collaboration between N N N agents learning M M M stochastic multi-armed bandits. result Proved near-optimal behavior of proposed algorithms for group regret.
We present a formal model of human decision-making in explore-exploit tasks using the context of multi-armed bandit problems, where the decision-maker must choose among multiple options with uncertain rewards. We address the standard multi-armed bandit problem, the multi-armed bandit problem with transition costs, and …
New algorithm tackles fairness in multi-armed bandit problem.
problem Fairness in stochastic multi-armed bandit problem.
method Characterized a class of Fair-SMAB algorithms with two parameters.
result Achieves O(log(T)) r-Regret with UCB1 learning algorithm.
The paper studies MAB problems with LDP to balance privacy and service quality.
problem Balancing privacy and service quality in multi-armed bandit systems.
method Investigates regret minimization for MAB with LDP guarantee, proving lower bounds and proposing matching upper bounds algorithms.
result Regret upper bounds match lower bounds up to constant factors for MAB algorithms with LDP guarantee.
Paper studies attacks on bandit algorithms and shows how attackers can manipulate data to hijack behavior.
problem Potential attacks on bandit algorithms can cause catastrophic loss in real-world applications.
method Proposes a framework of offline and online attacks on bandit algorithms using convex optimization and adaptive strategies.
result Attackers can force bandit algorithms to pull target arms with high probability by manipulating data.
Regularized contextual bandits use bins to solve multi-armed bandit problems.
problem Contextual bandit problems with a known baseline policy.
method Nonparametric model, splitting context space into bins, solving bandit instances independently.
result Intermediate convergence rates interpolating between slow and fast rates.
The paper analyzes bias in sample means of multi-armed bandits.
problem Analyzing bias in sample means of multi-armed bandits.
method Decoupling three sources of bias: sampling, stopping, choosing; using optimism to capture monotonic behaviors.
result Optimistic sampling induces negative bias, while optimistic stopping and choosing induce positive bias.
RBMLE learns from reward bias to optimize bandit decisions.
problem Optimizing decisions in stochastic multi-armed bandits.
method Reward-Biased Maximum Likelihood Estimation (RBMLE) for SMABs.
result RBMLE achieves optimal regret bounds for Gaussian and sub-Gaussian bandits.
Graph-based feedback improves bandit algorithms' performance.
problem Stochastic multi-armed bandit problem with graph feedback.
method Analysis of Thompson Sampling and UCB algorithms in graph-based feedback setting.
result Regret bounds that combine graph structure and arm means gaps.
New insights into balancing reward and fairness in stochastic MAB.
problem Balancing reward and fairness in stochastic multi-armed bandits.
method Formulated a penalization framework and proposed a hard-threshold UCB-like algorithm.
result Asymptotic fairness, nearly optimal regret, better reward-fairness tradeoff.
Adaptive and sequential experiment design is a well-studied area in numerous domains. We survey and synthesize the work of the online statistical learning paradigm referred to as multi-armed bandits integrating the existing research as a resource for a certain class of online experiments. We first explore the tradition…
New algorithm reduces regret in corrupted bandits.
problem Stochastic multi-armed bandits with adversarial corruption.
method A new algorithm that is agnostic to corruption levels.
result Regret is nearly optimal and can handle significant corruption.
Optimal strategy found for constrained multi-armed bandit problems.
problem Constrained multi-armed bandit problems.
method Extended ε_t-greedy strategy with asymptotic optimality.
result Asymptotic optimality achieved with a simple strategy.
The paper connects discrete choice models to multi-armed bandit algorithms with sublinear regret bounds.
problem Optimizing user choices in a multi-armed bandit setting.
method Establishes connections between discrete choice models and multi-armed bandit algorithms, providing sublinear regret bounds and novel algorithms.
result Sublinear regret bounds for a family of algorithms, including the Exp3 algorithm.
New algorithm reduces regret in multi-player bandits with collision information.
problem Optimizing decisions in multi-player bandits with collision penalties.
method Developed an algorithm with optimal T \sqrt{T} T regret under collision announcements, and sublinear regret without collision info. result First T \sqrt{T} T -type regret guarantee for non-stochastic multi-player multi-armed bandits with collision information. New algorithm tackles fairness in multi-armed bandit problem.
problem Fairness constraints in multi-armed bandit.
method Characterized Fair-SMAB algorithms with fairness tolerance and learning algorithm parameters.
result Achieves O ( ln T ) O(\ln T) O ( ln T ) r r r -Regret with UCB1 learning algorithm. New study on regret lower bounds for multi-agent multi-armed bandit problems.
problem Understanding the limits of performance in multi-agent multi-armed bandit problems.
method Comprehensive study on different settings, establishing tight lower bounds.
result First comprehensive study on regret lower bounds across various settings.
Improved FTRL algorithm for multi-armed bandits with various regularizers and multiple optimal arms.
problem Designing adaptive multi-armed bandit algorithms that perform optimally in both stochastic and adversarial settings.
method Follow-the-Regularized-Leader (FTRL) algorithm with a broad family of regularizers and a new learning rate schedule.
result Uniqueness of optimal arm assumption is unnecessary for FTRL with a broad family of regularizers.
New bandit model for e-commerce with ordered categories.
problem Optimizing customer experience in e-commerce with unknown preferences.
method Introducing three types of ordering between categories and proving lower bounds on cumulative regret.
result Proved that algorithms can fully leverage the structure of the model with theoretical guarantees.
New algorithm reduces memory-regret trade-off for bandits.
problem Optimizing memory usage in stochastic multi-armed bandits.
method Designing an algorithm with specific regret bound and proving lower bound.
result Optimal regret characterization in terms of memory and rounds.
A budget-constrained multi-armed bandit problem with multiple plays is analyzed for both stochastic and adversarial settings.
problem Optimizing decisions in a multi-armed bandit problem with a budget constraint for multiple plays.
method Upper Confidence Bound (UCB) algorithm for stochastic case and an extension of Exp3 algorithm for adversarial case.
result Achieved regret bounds for both stochastic and adversarial settings.
We study the stochastic multi-armed bandit problem when one knows the value μ ( ⋆ ) μ^{(\star)} μ ( ⋆ ) of an optimal arm, as a well as a positive lower bound on the smallest positive gap Δ Δ Δ . We propose a new randomized policy that attains a regret {\em uniformly bounded over time} in this setting. We also prove several lower bound…
New algorithm tackles unknown utility network resource allocation.
problem Maximizing network utility with unknown agent utilities.
method Modeling as a bandit problem, proposing algorithms for resource allocation.
result Proposed algorithms are optimal when all agents have the same utility.
Study multiplayer bandits without collision info, achieving regret bounds.
problem Multiplayer bandits without collision info.
method Two feedback models considered; algorithms for both models.
result First theoretical guarantees for second model with square-root regret.
Study on information evolution in interactive decision making using multi-armed bandits.
problem Understanding information dynamics in interactive decision making.
method Stochastic multi-armed bandit problem, focusing on optimal arm with a fixed margin.
result Distinct growth phases in mutual information, showing decoupling between success probability and information gain.
New algorithm optimizes multi-armed bandit performance in stochastic and adversarial settings.
problem Optimizing multi-armed bandit performance in both stochastic and adversarial environments.
method Follow-the-regularized-leader method with adaptive learning rates.
result First BOBW algorithm with gap-variance-dependent regret bounds in adversarial settings.
Multi-armed bandit problems are the most basic examples of sequential decision problems with an exploration-exploitation trade-off. This is the balance between staying with the option that gave highest payoffs in the past and exploring new options that might give higher payoffs in the future. Although the study of band…
A new multi-armed bandit framework with credal sets for uncertain outcomes.
problem Optimizing decisions under uncertainty with unknown outcomes.
method Introduces a novel multi-armed bandit framework with credal sets and defines regret as lower prevision.
result Upper bounds on regret for certain hypothesis classes and lower bounds for special cases.
A new algorithm reduces regret in multi-player bandits without collision info.
problem Decentralized multi-player multi-armed bandits with no collision info.
method EC-SIC algorithm using optimal error correction coding for reward statistics.
result Regret approaches that of centralized with collision info.
A new framework for risk-aware multi-armed bandits tackles volatile environments.
problem Volatility in healthcare and finance makes naive reward maximization unreliable.
method Risk-aware strategies with adaptive risk measures and change-point detection.
result Finite-time theoretical guarantees and asymptotic regret bound of order i l d e O ( K T T ) ilde O(\sqrt{K_T T}) i l d e O ( K T T ) . INF-clip optimizes heavy-tailed MAB problems with improved performance.
problem Optimizing multi-armed bandit problems with heavy-tailed rewards.
method INF-clip algorithm for adversarial and stochastic heavy-tailed MAB settings.
result INF-clip is optimal for linear and non-linear heavy-tailed stochastic MAB problems.
New algorithm ensures replicable results in multi-armed bandits with minimal extra regret.
problem Ensuring consistent results in multi-armed bandit studies.
method Incorporates randomness into decision-making to ensure replicability while maintaining minimal extra regret.
result For large time horizons, proposed algorithm suffers only K 2 / ρ 2 K^2/ρ^2 K 2 / ρ 2 times smaller amount of exploration than existing algorithms.