New algorithm CROP achieves asymptotic optimality with bounded regret.
problem Optimistic algorithms fail to achieve asymptotic instance-dependent regret optimality.
method CRush Optimism with Pessimism (CROP) algorithm that eliminates optimistic hypotheses.
result CROP achieves constant-factor asymptotic optimality and bounded regret.
We prove non-asymptotic lower bounds on the expectation of the maximum of d d d independent Gaussian variables and the expectation of the maximum of d d d independent symmetric random walks. Both lower bounds recover the optimal leading constant in the limit. A simple application of the lower bound for random walks is an (…
We study the K-armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. The hardness of recommending Copeland winners, the arms that beat the greatest number of other arms, is characterized by deriving an asymptotic regr…
TVBO optimizes time-varying functions with asymptotically vanishing regret.
problem Understanding the asymptotic performance of TVBO for time-varying black-box functions.
method Provided upper and lower bounds for cumulative regret of TVBO algorithms.
result TVBO algorithms can achieve asymptotically vanishing regret under certain conditions.
The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.
problem Improving regret bounds in multi-armed bandit problems.
method Establishes non-asymptotic regret bounds for upper confidence bound indices.
result New regret bounds match Lai-Robbins lower bound.
This paper establishes that optimistic algorithms attain gap-dependent and non-asymptotic logarithmic regret for episodic MDPs. In contrast to prior work, our bounds do not suffer a dependence on diameter-like quantities or ergodicity, and smoothly interpolate between the gap dependent logarithmic-regret, and the $\wid…
Optimal strategy identified for minimizing regret in fixed-budget best arm selection.
problem Minimizing expected simple regret in fixed-budget best arm selection.
method Two-Stage (TS)-Hirano-Imbens-Ridder (HIR) strategy using HIR estimator.
result TS-HIR strategy is asymptotically minimax optimal.
UCB-V algorithm improves on UCB for MAB problems with variance estimates.
problem Optimizing arm selection in MAB problems with variance information.
method Asymptotic and high probability analysis of UCB-V algorithm.
result UCB-V can exhibit instability in arm-pulling rates but achieves refined regret bounds.
Paper stabilizes bandit learning with regularization, improving inference under adaptive sampling.
problem Challenges in statistical inference with adaptive sampling.
method Refined stability condition for online algorithms, using regularized stochastic-mirror-descent-style methods.
result Derives precise regret bounds and asymptotic normality, showing necessity of regularization for valid inference.
E 4 ^4 4 algorithm optimizes batched linear bandits with minimal regret and batches.
problem Optimizing batched linear bandits for minimal regret.
method Explore-Estimate-Eliminate-Exploit framework with optimal exploration rate.
result Achieves minimax and asymptotic optimality in regret and batch complexity.
Study optimal stopping for diffusion processes using data-driven methods.
problem Optimal stopping for diffusion processes under unknown conditions.
method Data-driven approach, deriving upper and lower bounds on simple and cumulative regret.
result Verified minimax optimality and improved convergence rates.
New algorithm achieves asymptotically optimal regret without horizon dependence.
problem Horizon-free regret minimization for reinforcement learning.
method Proposes a new algorithm and proves a regret upper bound.
result Regret upper bound of \(\tilde O(\sqrt{SAK} + S^8A^3)\) with failure probability \(\delta\).
New method for semiparametric bandits reduces regret to optimal levels.
problem Complex reward structures in semiparametric bandits.
method Experimental-design approach with sharp regret bound and PAC bound.
result Minimax regret of i l d e O ( d T ) ilde{O}(\sqrt{dT}) i l d e O ( d T ) and logarithmic regret under positive suboptimality gap. Optimizes arm selection with side information in Gaussian bandits.
problem Optimizing arm selection with side information in Gaussian bandits.
method Constructs an LP-based asymptotic instance-dependent lower bound on the regret and develops the first known asymptotically optimal algorithm.
result First known asymptotically optimal algorithm for Gaussian bandits with side information.
New Thompson sampling algorithm reduces regret for exponential family bandits.
problem Minimizing regret in multi-armed bandit problems with exponential family rewards.
method Proposes ExpTS and ExpTS + ^+ + algorithms using novel sampling distributions. result Minimizes both finite-time and asymptotic regret for exponential family rewards.
Optimal algorithm for identifying best-arm with minimal regret.
problem Identifying the best arm in two treatments with limited budget.
method Neyman allocation based on outcome standard deviations.
result Neyman allocation is minimax optimal for simple regret.
We consider stochastic multi-armed bandits where the expected reward is a unimodal function over partially ordered arms. This important class of problems has been recently investigated in (Cope 2009, Yu 2011). The set of arms is either discrete, in which case arms correspond to the vertices of a finite graph whose stru…
Improved regret bounds for DP-KLUCB and DP-IMED in Bernoulli bandits.
problem Minimizing regret in stochastic bandits under ε-global Differential Privacy.
method Developed DP versions of KLUCB and IMED, proving tighter lower bounds and matching upper bounds.
result DP-KLUCB and DP-IMED achieve asymptotically optimal regret under ε-global DP.
We study the K K K -armed dueling bandit problem, a variation of the standard stochastic bandit problem where the feedback is limited to relative comparisons of a pair of arms. We introduce a tight asymptotic regret lower bound that is based on the information divergence. An algorithm that is inspired by the Deterministic…
New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of O ( K 2 log 2 ( K ) ) + O ( K log ( T ) ) O(K^2\log^2(K)) + O(K\log(T)) O ( K 2 log 2 ( K )) + O ( K log ( T )) in O ( log ( T ) ) O(\log(T)) O ( log ( T )) rounds. The paper tackles best arm identification with minimal regret in experiments.
problem Identifying the best arm with minimal regret in experiments.
method Information-theoretic techniques and Double KL-UCB algorithm.
result Achieves asymptotic optimality in identifying the best arm with minimal regret.
Improved analysis shows Maillard sampling achieves optimal regret bounds.
problem Optimal regret bounds for K-armed bandit problem.
method Improved analysis of Maillard sampling (MS) to achieve asymptotical optimality and minimax regret bound.
result MS achieves both asymptotical optimality and minimax regret bound of √(KT log T).
Study gap-dependent regret bounds for risk-sensitive RL.
problem Risk-sensitive reinforcement learning with entropic risk measure.
method Propose cascaded gaps to adapt to problem structures, derive regret bounds.
result Exponential improvement over existing bounds in appropriate settings.
Lower bounds on online logistic regression regret rates derived.
problem Achieving optimal regret rates in online logistic regression.
method Deriving lower bounds using information theory and grid methods.
result Achievable regret rates are dominated by d / 2 log T d/2 \log T d /2 log T under L 1 L_1 L 1 , L 2 L_2 L 2 , and L ∞ L_\infty L ∞ constraints. A classic setting of the stochastic K-armed bandit problem is considered in this note. In this problem it has been known that KL-UCB policy achieves the asymptotically optimal regret bound and KL-UCB+ policy empirically performs better than the KL-UCB policy although the regret bound for the original form of the KL-UCB…
Improved risk-sensitive RL with exponential Bellman equation and better regret bounds.
problem Exponential gap between upper and lower bounds in risk-sensitive RL.
method Identified and addressed deficiencies in existing algorithms and analysis; developed novel analysis and exploration mechanism.
result Improved regret upper bounds over existing ones.
New algorithm DUSA minimizes regret in structured bandits with structural information.
problem Optimal decision-making under uncertainty with structural reward information.
method DUSA algorithm exploiting convex structural information.
result Regret matches information-theoretic lower bound up to a constant factor.
We develop a new theoretical framework, the \emph{envelope complexity}, to analyze the minimax regret with logarithmic loss functions and derive a Bayesian predictor that adaptively achieves the minimax regret over high-dimensional ℓ 1 \ell_1 ℓ 1 -balls within a factor of two. The prior is newly derived for achieving the mini…
New algorithm tackles multi-armed bandits with abstention, achieving optimal regret bounds.
problem Optimizing decisions with abstention in multi-armed bandits.
method Developed algorithms meeting information-theoretic lower bounds on regret.
result Achieved asymptotically and minimax optimal regret bounds.
New algorithm tackles multimodal bandits with optimal regret bounds.
problem Stochastic multi-armed bandit problem with multimodal rewards.
method Proposed computationally tractable algorithm for multimodal rewards.
result Asymptotically optimal algorithms implemented for the bandit problem.
Improved online Q-learning for MDPs with concentration bounds.
problem Online Q-learning in infinite-horizon discounted MDPs with sublinear regret for large gaps.
method Smoothed ε n ε_n ε n -Greedy exploration scheme combining ε n ε_n ε n -greedy and Boltzmann exploration, analyzed using concentration bounds for contractive Markovian stochastic approximation. result Near- i l d e O ( N 9 / 10 ) ilde{O}(N^{9/10}) i l d e O ( N 9/10 ) regret bound for Smoothed ε n ε_n ε n -Greedy exploration scheme. CRIMED optimizes regret in bandits with unbounded stochastic corruption.
problem Minimizing regret in bandits with arbitrary unbounded corruptions.
method Introduces CRIMED, an asymptotically-optimal algorithm for Gaussian distributions with known variance.
result Achieves exact lower bound on regret for Gaussian distributions with high corruption probability.
KL-MS improves regret bounds for multi-armed bandits with bounded rewards.
problem Designing efficient exploration algorithms for multi-armed bandits with bounded rewards.
method Kullback-Leibler Maillard Sampling (KL-MS) for multi-armed bandits with bounded rewards.
result KL-MS achieves a worst-case regret bound of O ( μ ∗ ( 1 − μ ∗ ) K T ln K + K ln T ) O(\sqrt{μ^*(1-μ^*) K T \ln K} + K \ln T) O ( μ ∗ ( 1 − μ ∗ ) K T ln K + K ln T ) . VO Q Q Q L optimizes RL with sparse rewards using weighted bounds.
problem Sparse rewards and non-linear function approximation in RL.
method VO Q Q Q L combines Q Q Q -learning with weighted bounds for optimal regret. result Achieves asymptotically optimal regret for linear function approximation.
Logarithmic regret achieved in Q-learning with positive gap.
problem Achieving logarithmic cumulative regret in Q-learning with positive sub-optimality gap.
method Optimistic Q-learning with logarithmic regret bound.
result Logarithmic cumulative regret bound proven for optimistic Q-learning.
TRAiL is a linear bandit algorithm that ensures optimal regret and guarantees inference quality.
problem Optimal regret and inference quality in linear bandits with convex action sets.
method TRAiL estimates the parameter through regularized least squares and perturbs the action set along the tangent plane.
result TRAiL achieves an Ω ( T ) Ω(\sqrt{T}) Ω ( T ) upper bound on cumulative regret with high probability. We study online prediction of bounded stationary ergodic processes. To do so, we consider the setting of prediction of individual sequences and build a deterministic regression tree that performs asymptotically as well as the best L-Lipschitz constant predictors. Then, we show why the obtained regret bound entails the …
The paper sets bounds on how much regret is unavoidable in adaptive LQR with unknown B-matrix.
problem Understanding the limits of adaptive LQR with unknown B-matrix.
method Local asymptotic minimax regret lower bounds using van Trees' inequality and Bellman error representation.
result Logarithmic regret is impossible if the parametrization induces an uninformative optimal policy.
The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.
problem Reward concentration in Markov Decision Processes (MDPs).
method Unified approach to reward concentration in MDPs, including asymptotic and non-asymptotic bounds.
result Rate-equivalent definitions of regret for learning policies.
New algorithm reduces neural net error in contextual bandits.
problem Neural contextual bandits with general activation functions.
method Proposed an efficient algorithm with sublinear regret bound.
result Demonstrated provably sublinear regret bound in finite regime.
This paper unifies risk-averse Thompson sampling for continuous risk functionals.
problem Designing and analyzing risk-averse Thompson sampling algorithms for continuous risk functionals.
method Developed analytical toolkits to prove asymptotically optimal regret bounds for various risk measures.
result Proved asymptotic optimality of ρ ρ ρ -MTS for Bernoulli distributions and a class of risk measures. We propose an online convex optimization algorithm (RescaledExp) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an exponential separation between the regret of existing algorithms that require a known bound on the los…
We consider the setting of online linear regression for arbitrary deterministic sequences, with the square loss. We are interested in the aim set by Bartlett et al. (2015): obtain regret bounds that hold uniformly over all competitor vectors. When the feature sequence is known at the beginning of the game, they provide…
Proposes a general method to derive regret bounds for multi-armed bandit algorithms.
problem Deriving regret bounds for randomized multi-armed bandit algorithms.
method Checking sufficient conditions on sampling probabilities and distributions.
result Proves logarithmic regret bounds for various bandit algorithms and new models.
Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to mi…
New method improves on existing algorithms for rank-one bandits.
problem Minimizing regret in stochastic rank-one bandits.
method Unimodal Thompson Sampling (UTS) with new analysis.
result UTS provides an asymptotically optimal regret bound.
One-bit feedback suffices for a bandit problem's optimal strategy.
problem Optimal strategy for multi-armed bandit problem with limited feedback.
method Coding and decoding schemes for one-bit feedback to mimic full-reward feedback.
result Regret ratio approaches 1 with one-bit feedback.
Improved BO algorithms reduce prediction error under Gaussian noise.
problem Reducing prediction error in Bayesian optimization with Gaussian noise.
method Established new prediction error bounds for Gaussian process under frequentist setting.
result Proved improved convergence rates of cumulative regret for GP-UCB and GP-TS.