First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
The paper provides robustness guarantees for mode estimation in bandits.
problem Understanding robustness in mode estimation under adversarial data contamination.
method Simple randomization and theoretical analysis of multi-armed bandits.
result Regret guarantees for various modal bandit problems.
A new algorithm reduces regret in cooperative multi-agent bandits with heavy-tailed data.
problem Cooperative multi-agent bandits with heavy-tailed data.
method MP-UCB algorithm incorporating robust estimation with message-passing protocol.
result Optimal regret bounds for MP-UCB in various settings.
This paper improves offline contextual bandits using distributional robustness.
problem Improving offline contextual bandits with robustness.
method Extends Distributionally Robust Optimization (DRO) for offline contextual bandits, introducing a convex reformulation of Counterfactual Risk Minimization.
result Automatic calibration of asymptotic confidence intervals for policy optimization.
Paper addresses privacy and robustness in stochastic linear bandits.
problem Stochastic linear bandits with differential privacy and adversarial robustness.
method Logarithmic batch queries, arm elimination algorithm, two privacy models.
result First algorithms providing differential privacy and adversarial robustness.
New bandit algorithms focus on extreme values, outperforming existing methods.
problem Optimizing decisions based on extreme values rather than expected values.
method Robust statistics-based algorithms with vanishing extremal regret.
result The proposed algorithms achieve superior performance compared to existing methods.
Bandit algorithms struggle with consistent performance and robustness.
problem Achieving consistent and robust performance in stochastic multi-armed bandit settings.
method Analyzing regret minimization trade-offs and proposing distribution-oblivious algorithms.
result Logarithmic regret is inconsistent and super-logarithmic regret is necessary for consistent learning.
Paper addresses robust federated linear bandits against Byzantine attacks.
problem Byzantine attacks on a small fraction of agents in federated learning.
method Proposes a geometric median-based robust aggregation oracle.
result Achieves sublinear regret bound of i l d e O ( T 3 / 4 ) ilde{\mathcal{O}}({T^{3/4}}) i l d e O ( T 3/4 ) robust to fewer than half Byzantine agents. Paper tackles robust batched bandits for heavy-tailed rewards.
problem Clinical trials and other applications with heavy-tailed rewards.
method Proposes robust batched bandit algorithms for heavy-tailed rewards in finite-arm and Lipschitz-continuous settings.
result Heavier-tailed rewards require fewer batches for near-optimal regret in the instance-independent regime and Lipschitz setting.
The paper tackles misspecification in contextual bandits by incorporating arm-specific variables.
problem Misspecification in contextual bandits due to unexplained inter-arm heterogeneity.
method Develops robust contextual bandit algorithms (RoLinUCB and RoLinTS) that incorporate arm-specific variables to address misspecification.
result The developed algorithms bound the n n n -round Bayes regret and show superior performance in various misspecification scenarios. Study on private and robust multi-armed bandits with contaminated heavy-tailed rewards.
problem Private and robust multi-armed bandits with contaminated heavy-tailed rewards.
method Proposed a meta-algorithm with a private and robust mean estimation sub-routine exttt{PRM}.
result Achieved nearly-optimal regret for two heavy-tailed settings.
Study robust best-arm identification in linear bandits with lower bounds and algorithms.
problem Identify a near-optimal robust arm in linear bandits with adversarial actions.
method Propose instance-dependent lower bounds and both static and adaptive bandit algorithms.
result Sample complexity matches the lower bound and algorithms effectively identify robust arms.
Improved ε ε ε -greedy handles strategic bidding in PPC auctions.
problem Strategic bidding in PPC auctions with personalization and corruptions.
method Extended ε ε ε -greedy to handle strategic arms in contextual multi-arm bandit. result ε ε ε -greedy is robust to adversarial corruptions and degrades linearly with corruption. Robust algorithm optimizes corrupted Gaussian process bandits.
problem Sequential optimization of corrupted, expensive reward functions.
method Robust GP Phased Elimination (RGP-PE) algorithm.
result Algorithm balances robustness to corruptions with exploration and exploitation.
New bandit problem for finding best group of arms with worst mean reward.
problem Finding the best group of arms with the worst mean reward in overlapping groups.
method Two algorithms based on successive elimination and robust optimization.
result Upper bounds on the number of samples to find max-min optimal or near-optimal group.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
Motivated by applications of bandit algorithms in education, we consider a stochastic multi-armed bandit problem with ε \varepsilon ε -contaminated rewards. We allow an adversary to give arbitrary unbounded contaminated rewards with full knowledge of the past and future. We impose the constraint that for each time t t t the…
A robust bandit algorithm uses Dirichlet sampling to minimize regret under various distributional assumptions.
problem Robustness of bandit algorithms to model misspecification.
method Dirichlet Sampling (DS) algorithm based on pairwise comparisons and re-sampling of arm observations.
result Different DS variants achieve optimal regret guarantees for bounded distributions and logarithmic regret for semi-bounded distributions.
RoME optimizes mobile health interventions by modeling user and time-specific effects.
problem Challenges in optimizing mobile health interventions due to participant heterogeneity, nonstationarity, and nonlinear relationships.
method RoME uses a Robust Mixed-Effects contextual bandit algorithm with random effects, network cohesion penalties, and debiased machine learning.
result RoME achieves robust regret bounds even with complex baseline rewards, demonstrating superior performance in simulations and studies.
New algorithm tackles adversarial corruption in Lipschitz bandits with sub-linear regret.
problem Adversarial corruption in Lipschitz bandits.
method Developed robust Lipschitz bandit algorithms for weak and strong adversaries.
result Achieved sub-linear regret under both weak and strong adversaries.
Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.
problem Effective learning in linear bandits with corrupted rewards across different corruption models.
method Unified framework for analyzing strong and weak corruption, connection to gap-dependent misspecification, and specialized algorithm.
result Optimal bounds for gap-dependent misspecification in linear bandits.
New algorithms balance collaboration and adversarial behavior in linear bandits.
problem Minimizing regret in a collaborative linear bandit problem with adversarial agents.
method Robust collaborative phased elimination algorithm with tight analyses.
result Achieves near-optimal regret bounds of $O\left(α+ 1/\sqrt{M}
ight) \sqrt{dT}$ for good agents.
New algorithm reduces regret in GLM bandits with tighter bounds.
problem Reducing regret in generalized linear contextual bandits.
method Double Doubly Robust (DDR) estimator for independence.
result First d \sqrt{d} d regret bound for GLM bandits. New algorithms combat adversarial attacks in stochastic linear bandits.
problem Adversarial attacks on stochastic linear bandit rewards.
method Two variants of Robust Phased Elimination algorithms, one knowing C C C and one not. result Near-optimal regret in non-corrupted case and additive terms dependent on C C C . New algorithms robust to adversarial data achieve optimal performance.
problem Adversarial robustness in high-dimensional online learning problems.
method Alternating minimization scheme combining least-squares and convex reweighting.
result Achieves optimal robustness guarantees without distributional assumptions.
New model for display advertising with stochastic and adversarial components.
problem Display advertising with stochastic and adversarial click-through-rates.
method Adversarial scaling model; two algorithms tested: action elimination and mirror descent.
result Two algorithms are robust to adversarial scaling.
The paper develops a robust algorithm for contextual bandits with heavy-tailed rewards.
problem Contextual bandits with heavy-tailed rewards.
method Develops an algorithm based on Catoni's estimator for robust statistics, applying it to contextual bandits with general function approximation.
result Establishes regret bounds that depend on cumulative reward variance and logarithmically on the reward range and number of rounds.
New algorithms for best arm identification in bandits robust to misspecified parameters.
problem Inconsistent learning performance of traditional MAB algorithms when parameters are misspecified.
method Proposes two classes of asymptotically near-optimal algorithms for statistically robust MAB under fixed-budget pure exploration.
result Establishes fundamental performance limits and proposes algorithms that are asymptotically near-optimal.
Contextual multi-armed bandit algorithms are widely used in sequential decision tasks such as news article recommendation systems, web page ad placement algorithms, and mobile health. Most of the existing algorithms have regret proportional to a polynomial function of the context dimension, d d d . In many applications ho…
Paper establishes lower bounds for Gaussian process bandit optimization under various perturbation models.
problem Lower bounds for Gaussian process bandit optimization in noisy and robust settings.
method Novel proof techniques for standard and robust settings, including deterministic strategies.
result Demonstrates inevitable joint dependence of cumulative regret on corruption level and time horizon in robust settings.
Adapts to misspecification in contextual bandits using offline regression.
problem Unexpected regret due to misspecified reward models.
method Adapts to misspecification by reverting to a safe policy when necessary.
result Regret guarantees degrade gracefully with misspecification level.
Improves A/B testing power using a two-armed bandit framework.
problem Comparing outcomes under a new policy to a control.
method Doubly robust estimation, two-armed bandit framework, permutation-based method.
result Superior performance in A/B testing compared to existing methods.
A new algorithm RESYNC for defenders against malicious attackers in multi-player bandits.
problem Malicious players colliding with cooperative players to prevent rewards.
method Decentralized and robust algorithm RESYNC for defenders.
result RESYNC algorithm is order-optimal, performing gracefully as the number of collisions increases.
A new algorithm improves regret bounds for contextual bandits.
problem Complexity of missing data in multi-armed bandits.
method Doubly Robust (DR) Thompson Sampling with contexts.
result Improved regret bound with i l d e O ( d T ) ilde{O}(d\sqrt{T}) i l d e O ( d T ) . The paper tackles learning from imperfect human feedback, especially in dueling bandit problems.
problem Learning from human feedback that can be irrational or imperfect.
method Developed a Robustified Stochastic Mirror Descent for Imperfect Dueling (RoSMID) algorithm.
result Achieved nearly optimal regret for dueling bandit problems under imperfect human feedback.
We tackle linear bandits with partially observable features, achieving sublinear regret.
problem Linear regret due to unobserved features in partially observable linear bandits.
method Feature augmentation with orthogonal basis vectors and a doubly robust estimator.
result Sublinear regret bound of i l d e O ( ( d + d h ) T ) ilde{O}(\sqrt{(d + d_h)T}) i l d e O ( ( d + d h ) T ) . The paper shows optimal robustness against adversarial corruption in sequential decision-making problems.
problem Optimal robustness to adversarial corruption in online decision-making problems.
method Investigates prediction with expert advice and multi-armed bandit problems, focusing on algorithms with decreasing learning rates and second-order regret bounds.
result Optimal robustness can be expressed by a square-root dependency on the amount of corruption, achieving O ( log N Δ + C log N Δ ) O(\frac{\log N}{\Delta} + \sqrt{\frac{C \log N}{\Delta}}) O ( Δ l o g N + Δ C l o g N ) -regret. This paper studies the deviations of the regret in a stochastic multi-armed bandit problem. When the total number of plays n is known beforehand by the agent, Audibert et al. (2009) exhibit a policy such that with probability at least 1-1/n, the regret of the policy is of order log(n). They have also shown that such a …
The paper refines and extends batched kernelized bandits, improving regret bounds and introducing a robust setting.
problem Optimizing black-box functions with noisy batches in Reproducing Kernel Hilbert Space.
method Refined and extended existing regret bounds, including adaptive batch sizes and robust optimization.
result Improved regret bounds for batched kernelized bandits, showing optimal number of batches and adaptive batch sizes.
New algorithm reduces worst-case regret for heavy-tailed bandits.
problem Stochastic Multi-Armed Bandit problem with heavy-tailed rewards.
method Modified minimax policy MOSS with saturated empirical mean.
result Worst-case regret matching lower bound for heavy-tailed distributions.
New method for semiparametric bandits reduces regret to optimal levels.
problem Complex reward structures in semiparametric bandits.
method Experimental-design approach with sharp regret bound and PAC bound.
result Minimax regret of i l d e O ( d T ) ilde{O}(\sqrt{dT}) i l d e O ( d T ) and logarithmic regret under positive suboptimality gap. Paper tackles stochastic k k k -submodular bandits with full feedback, achieving sublinear regret.
problem Online optimization of k k k -submodular functions with full-bandit feedback. method Proposes online algorithms for various k k k -submodular stochastic combinatorial multi-armed bandit problems. result Achieves sublinear α α α -regret bounds for multiple k k k -submodular stochastic combinatorial multi-armed bandit problems. Unified framework for robust A/B testing under model misspecification.
problem Improving sample efficiency in A/B testing with model misspecification.
method Unified framework for contextual bandit and dynamic settings, proving worst-case mean squared error bounds.
result Empirically validated approach using synthetic and real-world datasets.
A TS algorithm improves performance in multi-task bandits with transfer.
problem Improving performance across multiple related bandit tasks.
method A TS-type algorithm for online multi-task learning with a novel concentration inequality.
result The TS-type algorithm achieves nearly-optimal performance guarantees.
The paper tackles robust policy learning in multitask contextual bandits with adversarial users.
problem Learning optimal policies in multitask contextual bandits with a small fraction of adversarial users.
method Developed efficient robust mean estimators for both uni-variate and high-dimensional random variables.
result Lower bound of i l d e Ω ( min ( S , A ) ⋅ α 2 / ε 2 ) ildeΩ(\min(S,A) \cdot α^2 / ε^2) i l d e Ω ( min ( S , A ) ⋅ α 2 / ε 2 ) per-user interactions to learn an ε ε ε -optimal policy for good users. Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
New testing method for robust actor-critic bandit algorithms.
problem Balancing data collection for app performance and user adherence.
method Modified actor-critic algorithm and novel testing procedure.
result Testing procedure is robust to critic misspecification.
New definition resolves ambiguity in non-stationary bandit classification.
problem Ambiguity in classifying non-stationary bandits using existing definitions.
method Introducing a formal definition that resolves ambiguity and provides a unified approach.
result Unified approach applicable to both Bayesian and frequentist formulations, resolves classification issues.