E 4 ^4 4 algorithm optimizes batched linear bandits with minimal regret and batches.
problem Optimizing batched linear bandits for minimal regret.
method Explore-Estimate-Eliminate-Exploit framework with optimal exploration rate.
result Achieves minimax and asymptotic optimality in regret and batch complexity.
Study nonparametric contextual bandits with batched updates, achieving optimal regret.
problem Optimal regret in nonparametric contextual bandits with batch constraints.
method Dynamic binning of covariate space, optimal regret achieved.
result Achieves optimal regret (up to logarithmic factors) for nonparametric contextual bandits.
The paper refines and extends batched kernelized bandits, improving regret bounds and introducing a robust setting.
problem Optimizing black-box functions with noisy batches in Reproducing Kernel Hilbert Space.
method Refined and extended existing regret bounds, including adaptive batch sizes and robust optimization.
result Improved regret bounds for batched kernelized bandits, showing optimal number of batches and adaptive batch sizes.
GN algorithm solves batched bandit for nondegenerate functions near-optimally.
problem Batched bandit learning for nondegenerate functions.
method Introduces Geometric Narrowing (GN) algorithm with a O ~ ( A + d T ) \widetilde{\mathcal{O}} ( A_{+}^d \sqrt{T} ) O ( A + d T ) regret bound and O ( log log T ) \mathcal{O} (\log \log T) O ( log log T ) batches. result GN achieves near optimal regret with minimal number of batches.
In this paper, we study the multi-armed bandit problem in the batched setting where the employed policy must split data into a small number of batches. While the minimax regret for the two-armed stochastic bandits has been completely characterized in \cite{perchet2016batched}, the effect of the number of arms on the re…
The paper derives a new theorem for predicting batches of data.
problem Finding lower bounds on minimal batch regret.
method Derives a conditional version of the regret-capacity theorem.
result Reveals a connection between conditional Rényi divergence and conditional Sibson's mutual information.
New algorithms for batched dueling bandits with improved regret bounds.
problem Batched dueling bandits with noisy pairwise comparisons.
method Developed algorithms for two settings: Condorcet winner and strong stochastic transitivity.
result Regret bounds match sequential bounds using only a logarithmic number of batches.
We analyze how batch learning impacts bandit problems.
problem Impact of batch learning in stochastic bandits.
method Policy-agnostic regret analysis, upper and lower bounds demonstration.
result The impact of batch learning can be measured in terms of online behavior.
New study reveals a polynomial penalty for adapting to unknown margin parameters in batched nonparametric bandits.
problem Adapting to an unknown margin parameter in batched nonparametric bandits.
method Introduces the regret inflation criterion and develops RoBIN algorithm to achieve optimal regret inflation.
result The optimal regret inflation grows polynomially with the horizon T, characterized by a convex optimization problem.
BLAE solves batched linear bandits with optimal regret and practical performance.
problem Batched linear bandit problem with limited adaptivity.
method Integrates arm elimination with regularized G-optimal design, achieving minimax optimal regret.
result Achieves minimax optimal regret in both large- K K K and small- K K K regimes with O ( log log T ) O(\log\log T) O ( log log T ) batches. TS-RSR improves batch Bayesian Optimization by minimizing redundancy and focusing on high uncertainty points.
problem Efficient batch selection in Bayesian Optimization to reduce redundancy and improve performance.
method TS-RSR uses Thompson Sampling to minimize a regret to uncertainty ratio for batch selection.
result TS-RSR outperforms state-of-the-art batch BO algorithms on various test functions.
A new batch ensemble method reduces regret in stochastic bandits.
problem Efficiently balancing exploration and exploitation in online RL.
method Proposes a batch ensemble scheme to achieve near-optimal regret.
result Proves near-optimal regret with a single parameter independent of variance.
Paper tackles robust batched bandits for heavy-tailed rewards.
problem Clinical trials and other applications with heavy-tailed rewards.
method Proposes robust batched bandit algorithms for heavy-tailed rewards in finite-arm and Lipschitz-continuous settings.
result Heavier-tailed rewards require fewer batches for near-optimal regret in the instance-independent regime and Lipschitz setting.
Algorithm achieves comparable performance to fully dynamic data with only a few batches.
problem High-dimensional multi-armed contextual bandits with batched feedback.
method Provable sample-efficient algorithm using batch allocation method.
result Achieves regret bounds comparable to fully sequential setting with only L = O(log T) batches.
Efficient RL algorithms for linear function approximation with limited adaptivity constraints.
problem Limited adaptivity in reinforcement learning with linear function approximation.
method Proposed two efficient online RL algorithms for episodic linear Markov decision processes under batch learning and rare policy switch models.
result Achieved efficient regret bounds for both batch learning and rare policy switch models, with substantial reduction in adaptivity.
A batched Gaussian Process bandit optimization method achieves near-optimal regret bounds.
problem Black-box optimization with limited function evaluations.
method Batched Gaussian Process bandit optimization algorithm.
result Achieves near-optimal cumulative regret bound of O ∗ ( T γ T ) O^\ast(\sqrt{Tγ_T}) O ∗ ( T γ T ) using O ( log log T ) O(\log\log T) O ( log log T ) batches. Study batch learning in linear bandits with context, achieving near-optimal performance.
problem Sequential batch learning in linear contextual bandits with finite actions.
method Established regret bounds and provided algorithms for two settings: arbitrary contexts and i.i.d. contexts.
result Regret upper bound nearly matches lower bound, showing polynomial and logarithmic batch requirements.
BaNk-UCB tackles batched nonparametric bandits with k-NN regression and UCB.
problem Sequential decision-making with limited online feedback in domains like medicine and marketing.
method Combines k-NN regression with UCB principle for fully nonparametric, adaptive, and simple implementation.
result Near-optimal regret guarantees under Lipschitz smoothness and margin assumptions, with minimax-optimal rates.
Paper improves CMAB regret bounds by reducing batch-size dependency.
problem Reducing batch-size dependency in combinatorial semi-bandits.
method Developed BCUCB-T and SESCB algorithms with new TPVM conditions.
result Significantly improved regret bounds for various applications.
Batched Neural Bandits reduces policy updates in sequential decision-making.
problem Sequential decision-making with batched policy changes.
method BatchNeuralUCB algorithm combining neural networks and optimism.
result Achieves similar regret as fully sequential version with fewer policy updates.
Study dynamic batch learning in high-dimensional sparse linear bandits.
problem Dynamic batch learning in high-dimensional sparse linear contextual bandits under batch constraints.
method Characterized fundamental learning limits via regret lower bound and provided matching upper bound.
result Prescribed an optimal scheme for dynamic batch learning in high-dimensional sparse linear contextual bandits.
New method evaluates LLMs fairness in universal prediction.
problem Evaluating fairness of large language models in universal prediction.
method Introducing batch regret as a modification of average regret for LLMs.
result Asymptotical value of batch regret for add-constant predictors on memoryless and first-order Markov sources.
Bayesian optimization method for permutations accelerates combinatorial search.
problem Optimizing expensive-to-evaluate objectives on permutation problems.
method LAW2ORDER, a batch Bayesian optimization method based on the acquisition weighted kernel.
result LAW2ORDER achieves sublinear batch cumulative regret, demonstrating accelerated search.
New constraints on space and adaptivity in bandits force more batches and memory use.
problem Simultaneous space and adaptivity constraints in stochastic bandits.
method Proved lower bounds and constructed an algorithm with near-minimax regret.
result Near-minimax regret requires more batches and memory than previously thought.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
problem Stochastic linear bandits with 1-bit communication constraints.
method Phased-elimination algorithms based on G-optimal designs and 1-bit mean estimation.
result Achieves near-optimal regret bounds for broad scaling regimes.
In this work, we investigate black-box optimization from the perspective of frequentist kernel methods. We propose a novel batch optimization algorithm, which jointly maximizes the acquisition function and select points from a whole batch in a holistic way. Theoretically, we derive regret bounds for both the noise-free…
Optimal algorithm for minimizing regret in heavy-tailed bandits.
problem Minimizing regret in stochastic multi-armed bandits with heavy-tailed distributions.
method Proposes an optimal algorithm under the assumption of uniformly bounded moments of order (1+ε).
result Matches the lower bound exactly in the first-order term and provides a finite-time bound on its regret.
We consider the combinatorial multi-armed bandit (CMAB) problem, where the reward function is nonlinear. In this setting, the agent chooses a batch of arms on each round and receives feedback from each arm of the batch. The reward that the agent aims to maximize is a function of the selected arms and their expectations…
We present two algorithms for Bayesian optimization in the batch feedback setting, based on Gaussian process upper confidence bound and Thompson sampling approaches, along with frequentist regret guarantees and numerical results.
New algorithm learns optimal policy with multi-step lookahead information.
problem Learning optimal policy in reinforcement learning with multi-step lookahead information is NP-hard.
method Adaptive batching policies that process lookahead in state-dependent chunks.
result Order-optimal regret bounds up to a constant factor of lookahead horizon.
DPP-BBO diversifies batched Bayesian optimization using DPPs.
problem Efficiently proposing diverse and informative batches in batched Bayesian optimization.
method Introducing DPP-Batch Bayesian Optimization (DPP-BBO) with DPP-Thompson Sampling (DPP-TS).
result Novel Bayesian simple regret bounds for DPP-TS show improved performance over classical methods.
Actor-Critic method achieves optimal regret for unichain MDPs.
problem Scalable regret analysis for infinite-horizon average-reward MDPs.
method NAC-B, a Natural Actor-Critic with batching.
result Order-optimal regret of i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) in infinite-horizon average-reward MDPs. A new algorithm learns policies from batch data in hierarchical RL.
problem Learning policies from fixed batches of data without full exploration.
method Modeling RL as a two-player game with a leader-follower structure, proposing StackelbergLearner.
result StackelbergLearner achieves competitive performance in batch RL and real-world datasets.
New algorithms tackle adversarial combinatorial bandits with switching costs.
problem Adversarial combinatorial bandits with switching costs.
method Design algorithms operating in batches to restrict switches, proving lower bounds and achieving upper bounds on regret.
result Achieved upper bounds on regret for both bandit and semi-bandit feedback settings.
New algorithm achieves near-optimal performance in dueling bandit problem.
problem Optimizing decision-making in dueling bandit problems with limited adaptive rounds.
method Developed a batched algorithm that matches the asymptotic regret bounds of sequential algorithms under the Condorcet condition.
result Asymptotic regret of O ( K 2 log 2 ( K ) ) + O ( K log ( T ) ) O(K^2\log^2(K)) + O(K\log(T)) O ( K 2 log 2 ( K )) + O ( K log ( T )) in O ( log ( T ) ) O(\log(T)) O ( log ( T )) rounds. The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.
problem Maximizing long-term average reward in Markov Decision Processes with batch learning.
method Doubly robust estimator for average reward, optimization algorithm for optimal policy, finite-sample regret guarantee.
result The proposed method achieves semiparametric efficiency and provides a finite-sample regret guarantee.
The paper explores trade-offs between regret and variance in online learning algorithms.
problem Investigating the trade-offs between regret and variance in online learning.
method Analysis of the Exponentially Weighted Average (EWA) algorithm and its variants.
result A variant of EWA either achieves negative regret or guarantees a logarithmic bound on both variance and regret.
Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.
problem Closing the gap between decentralized MP-MAB and natural centralized lower bound.
method BEACON: Batched Exploration with Adaptive COmmunicatioN, incorporating ADC and batched exploration.
result Proves logarithmic regret for a generalized MP-MAB problem.
New algorithms reduce matching regret by limiting frequent updates.
problem Minimizing regret in stochastic matching with rare optimization updates.
method Batched algorithms that limit matching updates to Θ(log log T) rounds.
result Achieve a regret bound of \(\widetilde{\mathcal{O}}(\sqrt{T})\) with reduced computational cost.
Study on adaptivity constraints in linear contextual bandits with optimal design.
problem Impact of adaptivity constraints on linear contextual bandits.
method Two models of limited adaptivity: batch learning and rare policy switches. Proposed distributional optimal design.
result Achieves minimax-optimal regret with optimal number of policy switches and batches.
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
problem Balancing exploration and exploitation in online decision making.
method Introducing a batch Thompson Sampling framework for stochastic multi-arm bandit and linear contextual bandit problems.
result Achieves asymptotic regret bound with O ( log T ) O(\log T) O ( log T ) batch queries, significantly reducing interactions. Two algorithms achieve optimal regret with limited adaptivity in multinomial logistic bandits.
problem Achieving optimal regret with limited adaptivity in multinomial logistic bandits.
method Presented two algorithms, B-MNL-CB and RS-MNL, for batched and rarely-switching paradigms.
result Achieved i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret with limited adaptivity. Efficient algorithms for contextual slate bandits with limited adaptivity.
problem Contextual slate bandit problem with limited adaptivity.
method Proposed B-SlateGLinCB and RS-SlateGLinCB algorithms for batched and rarely-switching settings.
result Achieved regret bounds of O(Nd^(3/2)√T) and O(Nd√T) under diversity assumption.
The paper studies early stopping methods in linear contextual bandits.
problem Minimizing in-experiment regret and conducting robust post-experiment inferences in contextual bandits.
method The study proposes early stopping rules based on the Opportunity Cost and Threshold Method, using variances of estimators to quantify upper regret bounds.
result The proposed method provides a systematic approach to minimize in-experiment regret and conduct robust post-experiment inferences.
New PAC-Bayesian bounds for online learning with data streams.
problem Challenges of traditional PAC-Bayesian bounds in dynamic data collection.
method Developed new PAC-Bayesian bounds in online learning framework, using updated regret definition and batch-to-online conversion.
result PAC-Bayesian bounds hold for online learning with dependent data and non-convex losses.
A new method for risk-averse decision-making in Markov processes with improved regret bounds.
problem Risk-averse decision-making in Markov processes.
method Introduces mini-batch measures and multipattern risk-averse problems in a feature-based Q Q Q -learning method. result Proves a high-probability regret bound of O ( H 2 N H K ) \mathcal{O}\big(H^2 N^H \sqrt{ K}\big) O ( H 2 N H K ) for the Q Q Q -learning method. DETC algorithm achieves asymptotic optimality in multi-armed bandit problems.
problem Multi-armed bandit problem with subgaussian rewards.
method DETC algorithm with two exploration and exploitation phases.
result DETC achieves asymptotically optimal regret bound.
New algorithm guarantees performance on noisy data.
problem Learning with noisy data and heavy-tailed distributions.
method Anytime online-to-batch conversion for smooth objectives.
result Stochastic gradient-based algorithm with sub-Gaussian error bounds.