New study on regret lower bounds for multi-agent multi-armed bandit problems.
problem Understanding the limits of performance in multi-agent multi-armed bandit problems.
method Comprehensive study on different settings, establishing tight lower bounds.
result First comprehensive study on regret lower bounds across various settings.
The paper improves regret lower bounds for communicating MDPs.
problem Regret lower bounds for communicating MDPs.
method Lower bound proof and optimization problem formulation.
result Regret lower bound becomes significantly more complex in communicating MDPs.
Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.
problem Achieving optimal regret bounds for optimistic Hedge in two-player zero-sum games.
method Refined regret analysis and optimization problem formulation.
result Optimistic Hedge achieves O ( log m log n ) O(\sqrt{\log m \log n}) O ( log m log n ) regret bounds, matching upper and lower bounds. Study finds optimal regret bound for multi-armed bandit problem with expert advice.
problem Optimizing decision-making in a multi-armed bandit problem with expert advice.
method Proved a tight lower bound matching the upper bound of Kale (2014) for minimax expected regret.
result The minimax optimal expected regret is Θ(√(T K log (N/K))) for the problem.
New algorithm for multi-fidelity bandits reduces costs and improves regret.
problem Optimizing decisions with varying costs and accuracy in multi-fidelity bandits.
method Cost complexity bounds, algorithmic framework, elimination-based algorithm.
result New regret definition and matching upper and lower bounds for elimination-based algorithm.
In this short note we consider a dynamic assortment planning problem under the capacitated multinomial logit (MNL) bandit model. We prove a tight lower bound on the accumulated regret that matches existing regret upper bounds for all parameters (time horizon T T T , number of items N N N and maximum assortment capacity K K K )…
We prove non-asymptotic lower bounds on the expectation of the maximum of d d d independent Gaussian variables and the expectation of the maximum of d d d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
We study the stochastic multi-armed bandit problem when one knows the value μ ( ⋆ ) μ^{(\star)} μ ( ⋆ ) of an optimal arm, as a well as a positive lower bound on the smallest positive gap Δ Δ Δ . We propose a new randomized policy that attains a regret {\em uniformly bounded over time} in this setting. We also prove several lower bound…
Unified framework for expert selection with bandit and lower-bound feedback.
problem Selecting the best expert in scenarios with bandit feedback and lower-bound information.
method Introduces a new feedback model combining bandit and lower-bound information, proving optimal regret bounds for modified Exp3 algorithms.
result Optimal regret bounds for modified Exp3 algorithms, generalizing both bandit and full-information settings.
We study the K K K -armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…
Improved regret bounds for bandits with expert advice.
problem Optimizing decision-making in environments with expert advice.
method Proved lower and upper bounds for regret in restricted and standard feedback models.
result Proved a new upper bound of order K T ln ( N / K ) \sqrt{K T \ln(N/K)} K T ln ( N / K ) for the worst-case regret, matching a previously known lower bound. Paper establishes new lower bounds for MDPs with changing transition kernels.
problem Minimizing sample complexity and regret in non-stationary MDPs.
method Developed novel lower bounds and constructed hard MDPs.
result Proved Ω ( ( H 3 S A / ε 2 ) log ( 1 / δ ) ) Ω((H^3SA/ε^2)\log(1/δ)) Ω (( H 3 S A / ε 2 ) log ( 1/ δ )) sample complexity lower bound. New algorithms avoid a dominant lower-order term in heavy-tailed loss settings.
problem Prediction with heavy-tailed losses without prior knowledge.
method Adaptive algorithms that avoid the maximum of losses as a lower-order term in regret.
result Improved regret bounds of O ( θ T log ( K ) ) \mathcal{O}(\sqrt{θT\log(K)}) O ( θ T log ( K ) ) and O ( θ log ( K T ) / Δ min ) \mathcal{O}(θ\log(KT)/Δ_{\min}) O ( θ log ( K T ) / Δ m i n ) . Improved regret bounds for DP-KLUCB and DP-IMED in Bernoulli bandits.
problem Minimizing regret in stochastic bandits under ε-global Differential Privacy.
method Developed DP versions of KLUCB and IMED, proving tighter lower bounds and matching upper bounds.
result DP-KLUCB and DP-IMED achieve asymptotically optimal regret under ε-global DP.
Study on regret minimization in deterministic MDPs.
problem Minimizing regret in deterministic reinforcement learning.
method Logarithmic regret lower bounds, leveraging graph theory and cycles.
result Explicitly quantifies the fundamental limit of performance achievable by any learning algorithm.
New lower bounds for combinatorial multi-armed bandits for general reward functions.
problem Maximizing reward in sequential decisions with sets of arms.
method Proved tight regret lower bounds for all smooth reward functions under mild assumptions.
result Lower bounds are tight up to log-factors for monotone reward functions.
New algorithm reduces regret in infinite MDPs with optimal variance-dependent bounds.
problem Infinite horizon MDPs lack optimal algorithms with low regret.
method Developed a UCB-style algorithm for average-reward and γ-regret.
result Achieved optimal variance-dependent regret bounds for both objectives.
The paper improves regret bounds for admission control in queueing systems.
problem Improving regret bounds for admission control in queueing systems.
method Proposes an algorithm inspired by UCRL2 and uses problem structure to bound regret.
result Proves an upper bound on the expected total regret of O ( S log T + m T log T ) O(S\log T + \sqrt{mT \log T}) O ( S log T + m T log T ) . The study sets limits on how well systems can be controlled adaptively.
problem Learning to control unknown linear Gaussian systems with quadratic costs.
method Combining ideas from experiment design, estimation theory, and perturbation bounds of information matrices.
result Regret lower bounds of the order of T \sqrt{T} T in the time horizon T T T accurately capture control-theoretic parameters. Improved UCB algorithm for diversity in bandits with lower bounds.
problem Improving diversity in bandit-based recommendations.
method Designing a UCB algorithm exploiting the specific structure of the problem.
result The UCB algorithm achieves bounded distribution-dependent regret under certain conditions.
Improved regret bounds for linear bandits with heavy-tailed rewards.
problem Stochastic linear bandits with heavy-tailed rewards.
method Elimination-based algorithm guided by experimental design.
result Regret bound of \(\tilde{\mathcal{O}}(d^\frac{1+3ε}{2(1+ε)} T^\frac{1}{1+ε})\) for \(ε\in (0,1)\).
This is a brief technical note to clarify the state of lower bounds on regret for reinforcement learning. In particular, this paper: - Reproduces a lower bound on regret for reinforcement learning, similar to the result of Theorem 5 in the journal UCRL2 paper (Jaksch et al 2010). - Clarifies that the proposed proof of …
Paper establishes lower bounds for Gaussian process bandit optimization under various perturbation models.
problem Lower bounds for Gaussian process bandit optimization in noisy and robust settings.
method Novel proof techniques for standard and robust settings, including deterministic strategies.
result Demonstrates inevitable joint dependence of cumulative regret on corruption level and time horizon in robust settings.
New algorithm minimizes Bayesian regret in offline linear bandits.
problem Minimizing Bayesian regret in offline linear bandits.
method Proposes a new algorithm that directly minimizes upper bounds on Bayesian regret using conic optimization.
result Upper bounds are tight and guarantee superior performance compared to LCB.
Improved regret bounds for contextual bandits considering variance sequences.
problem Establishing lower bounds for contextual bandits with variance-dependent regret.
method Developed variance-dependent lower bounds for linear contextual bandits under two settings: fixed and adaptive variance sequences.
result Lower bounds match upper bounds of SAVE algorithm up to logarithmic factors.
GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.
problem The suboptimality of GP-UCB under polynomial effective optimism.
method Analysis of effective optimism level and new regret lower bound.
result GP-UCB is not minimax optimal under polynomial growth of effective optimism.
We study the linear contextual bandit problem with finite action sets. When the problem dimension is d d d , the time horizon is T T T , and there are n ≤ 2 d / 2 n \leq 2^{d/2} n ≤ 2 d /2 candidate actions per time period, we (1) show that the minimax expected regret is Ω ( d T ( log T ) ( log n ) ) Ω(\sqrt{dT (\log T) (\log n)}) Ω ( d T ( log T ) ( log n ) ) for every algorithm, and (2) introduce a V…
Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to mi…
The paper studies MAB problems with LDP to balance privacy and service quality.
problem Balancing privacy and service quality in multi-armed bandit systems.
method Investigates regret minimization for MAB with LDP guarantee, proving lower bounds and proposing matching upper bounds algorithms.
result Regret upper bounds match lower bounds up to constant factors for MAB algorithms with LDP guarantee.
The paper achieves nearly optimal regret bounds for contextual multinomial logit bandits.
problem The contextual multinomial logit (MNL) bandit problem with varying rewards.
method Established lower bounds and proposed OFU-MNL+ algorithm with matching upper bounds.
result Achieved minimax optimal regret bounds for both uniform and non-uniform reward settings.
Optimal simple regret bound for Gaussian Process bandits.
problem Sequential optimization of expensive-to-evaluate functions.
method Proved a bound on simple regret for pure exploration algorithms.
result Order optimal bound on simple regret for Gaussian Process bandits.
Study on Pareto optimality in multi-objective bandit problems.
problem Pareto optimality in multi-objective multi-armed bandit problems.
method Formulated adversarial multi-objective multi-armed bandit, defined Pareto regrets, presented algorithms, established upper and lower bounds.
result New algorithms are optimal in adversarial settings and nearly optimal in stochastic settings.
A new linear contextual bandit algorithm with improved regret bound.
problem Efficiently solving linear contextual bandit problems with reduced regret.
method Proposes a novel estimator embedded with exploration and a self-normalized bound.
result Regret bound matches lower bound of Ω ( d T ) Ω(\sqrt{dT}) Ω ( d T ) up to logarithmic factors. We consider stochastic multi-armed bandits where the expected reward is a unimodal function over partially ordered arms. This important class of problems has been recently investigated in (Cope 2009, Yu 2011). The set of arms is either discrete, in which case arms correspond to the vertices of a finite graph whose stru…
New algorithm for shareable arms with load-dependent rewards in stochastic bandits.
problem Learning optimal play strategy with shareable finite-capacity arms in stochastic bandits.
method Developed a capacity estimator and online learning algorithm for MP-MAB with shareable arms.
result Regret upper bound matches the lower bound, validating the algorithm's performance.
The paper improves bounds on regret in Gaussian process bandits.
problem Sequential optimization of expensive, possibly non-convex functions with noisy feedback.
method Analyzes maximal information gain and decay rates of GP kernel eigenvalues to improve regret bounds.
result General bounds on maximal information gain and improved regret bounds for various settings, including Matérn kernels.
This paper tackles near-optimal adversarial RL with switching costs, providing algorithms and matching lower bounds.
problem Adversarial RL with switching costs, where loss distribution can be non-stationary or adversarial.
method Developed novel switching-reduced algorithms with matching lower bounds for known and unknown transition functions.
result Achieved near-optimal performance in adversarial RL with switching costs, matching theoretical lower bounds.
A new algorithm for CTMAB minimizes regret with sampling costs.
problem Continuous-time multi-arm bandit problem with sampling costs.
method Established lower bounds and proposed algorithms to minimize regret.
result Achieved lower bounds on regret up to logarithmic factors.
Adversarial online multi-task RL with task separation.
problem Minimize regret in an adversarial online multi-task setting with unknown MDPs.
method Prove minimax and instance-specific lower bounds, develop a clustering algorithm with optimal sample complexity and regret.
result Tight sample complexity and regret bounds for adversarial online multi-task RL.
New algorithm bounds regret in mediator feedback bandit problems.
problem Mediator feedback bandit game with policy sets.
method Adopting EXP4 algorithm, new regret bounds based on policy set capacity.
result Nearly-matching lower bounds for policy set families.
This paper establishes strong lower bounds for learning in revealing POMDPs.
problem Understanding the fundamental limits of reinforcement learning in revealing partially observable Markov Decision Processes (POMDPs).
method Develops strong PAC and regret lower bounds for learning in revealing POMDPs using multi-step revealing POMDPs as a case study.
result Strong polynomial lower bounds for learning in revealing POMDPs, achieving significantly smaller gaps against current upper bounds.
New lower bound shows bandit convex optimization is harder than previously thought.
problem Establishing a lower bound on the minimax expected regret for bandit convex optimization.
method Constructing a hard class of convex functions and analyzing the posterior spread of Fisher information matrices.
result A Ω ~ ( d 5 / 4 T ) \widetildeΩ(d^{5/4}\sqrt{T}) Ω ( d 5/4 T ) lower bound on the minimax expected regret. New algorithm reduces regret in collaborative multi-agent bandit problems.
problem Optimizing decisions in a network of agents with communication delays.
method Follow-the-Regularized-Leader (FTRL) algorithm with suitable regularizers and communication protocols.
result Upper bound on individual regret matches lower bound up to a constant factor.
Near-optimal per-action regret bounds for sleeping bandits are derived.
problem Optimizing performance in sleeping bandits where arms and losses are chosen by an adversary.
method Directly minimizing per-action regret using generalized versions of EXP3, EXP3-IX, and FTRL with Tsallis entropy.
result Near-optimal bounds of order O ( T A ln K ) O(\sqrt{TA\ln{K}}) O ( T A ln K ) and O ( T A K ) O(\sqrt{T\sqrt{AK}}) O ( T A K ) are obtained. This paper analyzes regret bounds for Gaussian process Thompson sampling.
problem Analyzing the performance of Gaussian process Thompson sampling (GP-TS) in Bayesian optimization.
method The paper derives several regret bounds for GP-TS, including a lower bound, upper bounds on the second moment of cumulative regret, expected lenient regret, and improved cumulative regret.
result The paper provides improved regret upper bounds for GP-TS, showing that it suffers from a polynomial dependence on 1 / δ 1/δ 1/ δ with probability δ δ δ . Study optimal stopping for diffusion processes using data-driven methods.
problem Optimal stopping for diffusion processes under unknown conditions.
method Data-driven approach, deriving upper and lower bounds on simple and cumulative regret.
result Verified minimax optimality and improved convergence rates.
New algorithm reduces submodular maximization regret to nearly optimal bounds.
problem Maximizing submodular functions under stochastic bandit feedback with cardinality constraint.
method Sub-UCB algorithm, robust greedy approximation, minimax lower bounds.
result Achieves nearly optimal regret bounds for submodular maximization.
Improved Bandit PCA with optimal regret bound.
problem Minimizing regret in online PCA with bandit feedback.
method Combines online mirror descent and multiscale exploration.
result Minimax optimal regret bound of r d T r\sqrt{dT} r d T .