New algorithms tackle RKHS bandits with reduced complexity and improved performance.
problem Adversarial and stochastic RKHS bandit problems with high computational complexity.
method Combining approximation theory with misspecified linear bandit methods.
result First general algorithm for adversarial RKHS bandit problem.
Cramming method evaluates learned policies from contextual bandits efficiently.
problem Evaluating final learned policies from contextual bandit algorithms.
method On-policy evaluation using a single pass of data, ensuring consistency and asymptotic normality.
result Cramming method reduces evaluation standard error by approximately 40% compared to off-policy methods.
A meta-UCB method combines stochastic bandit algorithms.
problem Combining multiple stochastic bandit algorithms efficiently.
method Meta-UCB procedure solving an N-armed bandit problem.
result Final regret depends only on the best base algorithm's regret.
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
Contextual bandits require careful exploration vs. exploitation to avoid biased outcome model estimation.
problem Contextual bandits face challenges in estimating outcome models due to rich heterogeneity and complex models.
method Developed parametric and non-parametric contextual bandits integrating balancing methods from causal inference and econometrics.
result First regret bound analyses for contextual bandits with balancing show competitive performance.
New method tackles bandit problem with varying rewards over time.
problem Varying rewards in bandit problems over time.
method Gaussian processes for estimation and planning.
result Improved computational efficiency through optimistic planning.
Develops balanced linear contextual bandit algorithms to reduce estimation bias.
problem Estimation bias in contextual bandits with rich heterogeneity or complex models.
method Integrates balancing methods from causal inference to reduce estimation bias.
result First regret bound analyses for linear contextual bandits with balancing.
A new method for dueling bandits improves performance.
problem Improving decision-making in dueling bandits.
method Sup-KLUCB method for K-armed dueling bandits, converting Copeland dueling bandits into standard MAB problems.
result Sup-KLUCB outperforms state-of-the-art methods in Copeland dueling bandits.
RONM method reduces regret in stochastic convex bandits with decreasing noise.
problem Stochastic convex bandit problem with decreasing noise.
method Regularized Online Newton Method (RONM) based on Online Newton Method (ONM).
result RONM achieves polylogarithmic regret in time horizon n.
New dynamic allocation methods for multi-armed bandit models.
problem Dynamic allocation problems in multi-armed bandit models.
method New types of dynamic allocation problems and proofs for Gittins index decomposition.
result New proofs for Gittins index decomposition and related results.
A study evaluates contextual bandit algorithms using supervised learning datasets.
problem Understanding practical behavior of contextual bandit algorithms.
method Empirical evaluation of contextual bandit algorithms using optimization oracles from supervised learning.
result A recent method using optimism under uncertainty performs best overall.
Study identifies methods for handling sparse data in web settings with contextual bandits.
problem Handling sparse data in web settings with contextual bandits.
method Identified and categorized methods for addressing sparse data issues.
result Updated understanding of sparse data problems using contextual bandits in web settings.
Polynomial-time method solves complex combinatorial semi-bandits.
problem Optimal strategies for combinatorial semi-bandits with uncorrelated Gaussian rewards.
method Proposes a polynomial-time method to solve the Graves-Lai optimization problem for various combinatorial structures.
result First known approach to implement asymptotically optimal algorithms in polynomial time for combinatorial semi-bandits.
New algorithms improve linear bandit performance with low computation.
problem Optimizing reward in linear stochastic bandits.
method Reward-biased maximum likelihood method modified for linear and generalized linear bandits.
result New policies achieve order-optimality and competitive empirical performance.
Paper analyzes nonconvex bandit problems with improved adaptive methods.
problem Continuous armed bandit problems for nonconvex cost functions.
method Simple and adaptive bin splitting methods.
result Adaptive method achieves locally minimax optimal expected cumulative regret.
New method for contextual bandit with missing rewards.
problem Contextual bandit with missing rewards in online settings.
method Combining contextual bandit approach with unsupervised learning (clustering) to estimate missing rewards.
result Promising empirical results on real-life datasets.
New method improves on existing algorithms for rank-one bandits.
problem Minimizing regret in stochastic rank-one bandits.
method Unimodal Thompson Sampling (UTS) with new analysis.
result UTS provides an asymptotically optimal regret bound.
Bayesian method infers contextual bandit policies robustly.
problem Inference of contextual bandit policies in small sample sizes.
method Empirical likelihood for Bayesian inference.
result Accurate uncertainty measurements and policy comparison.
EVILL uses randomised perturbations to improve exploration in bandit problems.
problem Improving exploration in structured stochastic bandit problems.
method Solves for the minimiser of a linearly perturbed regularised negative log-likelihood function.
result EVILL matches the performance of Thompson-sampling-style methods in theory and practice.
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. …
New method for semiparametric bandits reduces regret to optimal levels.
problem Complex reward structures in semiparametric bandits.
method Experimental-design approach with sharp regret bound and PAC bound.
result Minimax regret of i l d e O ( d T ) ilde{O}(\sqrt{dT}) i l d e O ( d T ) and logarithmic regret under positive suboptimality gap. New method prevents forgetting in neural-linear bandits for high-dimensional data.
problem Catastrophic forgetting in neural-linear bandits with high-dimensional features.
method Limited memory neural-linear bandit with likelihood matching.
result Resilient to catastrophic forgetting and superior performance.
Improved UCB method for stochastic bandits using distance tuning.
problem Improving performance in stochastic bandit problems.
method Tuning confidence bounds based on bandit distance.
result Empirically shows increased performance compared to existing methods.
Develops methods for reliable inference on batched bandit data.
problem Need for reliable inference methods based on adaptively-collected data from bandit algorithms.
method Introduces Batched OLS (BOLS) estimator for reliable inference on bandit data.
result BOLS is asymptotically normal and robust to non-stationarity in the baseline reward.
New algorithm eliminates arms to minimize regret in complex bandit problems.
problem Minimizing regret in combinatorial bandit problems with explicit exploration.
method Introduces a novel arm elimination scheme that partitions arms into three categories and incorporates explicit exploration.
result Achieves near-optimal regret in combinatorial multi-armed and linear contextual bandit problems.
New algorithms reduce regret for convex bandits with small comparator norms.
problem Optimizing in bandit convex optimization with varying comparator norms.
method Developed algorithms using techniques from full-information setting and new gradient estimators.
result Regret bounds are small when comparator norm is small.
New framework uses tree ensembles for contextual bandits.
problem Optimizing decisions in dynamic environments with contextual information.
method Adapts Upper Confidence Bound and Thompson Sampling to tree ensemble methods.
result Tree ensemble methods outperform traditional methods in regret minimization and runtime.
This review examines bandit problems in AI using statistical methods.
problem Sequential decision-making under uncertainty in AI environments.
method Foundational models, concentration inequalities, minimax regret bounds, frequentist and Bayesian algorithms, K-armed contextual bandits, SCAB, functional data analysis.
result Exploration-exploitation trade-offs and regret analyses in various bandit problems.
New algorithm reduces regret in stochastic bandit convex optimization.
problem Optimizing decisions in uncertain environments with convex losses.
method Introduces a second-order method for zeroth-order stochastic convex bandits.
result Regret bound of ( 1 + r / d ) [ d 1.5 n + d 3 ] p o l y l o g ( n , d , r ) (1 + r/d)[d^{1.5} \sqrt{n} + d^3] polylog(n, d, r) ( 1 + r / d ) [ d 1.5 n + d 3 ] p o l y l o g ( n , d , r ) . Simple method improves exploration in various decision problems.
problem Improving exploration in sequential decision problems.
method Parameterized Exploration (PE) method that considers time horizon and state of knowledge.
result PE outperforms un-tuned methods in various bandit and decision problem settings.
A new algorithm for conversational recommendation systems using dueling bandits in GLMs.
problem Limited user feedback in existing conversational bandit methods.
method Integrates dueling bandits with relative feedback in generalized linear models.
result Theoretical and empirical validation of ConDuel's efficacy.
A new algorithm for top-k combinatorial bandits with full-bandit feedback.
problem Optimizing decisions among subsets of arms with only aggregate rewards observed.
method CSAR algorithm using Hadamard matrices for estimating individual arm rewards.
result Proved a tight lower bound on sample complexity for k = O ( 1 ) k=O(1) k = O ( 1 ) . Paper presents adaptive Lipschitz bandit framework for efficient optimization.
problem Optimizing rewards and minimizing regret in stochastic Lipschitz bandit problems.
method Adaptive learning of partitions in context- and arm-space using hierarchical Bayesian models.
result Achieves state-of-the-art performance in real-world tasks like neural network hyperparameter tuning.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
Proposes BSI for valid statistical inference on bandit algorithms.
problem Valid statistical inference on bandit algorithms' performance.
method Fits a simulator of the bandit environment from observed data and uses it to estimate mean reward under any policy.
result Proves asymptotically valid confidence intervals and maintains nominal coverage.
Efficiently handles contextual bandits with diffusion models.
problem Challenges in online decision-making with contextual bandits.
method Leverage pre-trained diffusion models as priors to capture action dependencies.
result Developed an algorithm for efficient posterior approximation.
A new method boosts exploration in bandit algorithms, reducing regret.
problem Improving exploration in bandit algorithms with bounded or unbounded rewards.
method Residual Bootstrap Exploration (ReBoot) method that injects data-driven randomness.
result Proves logarithmic regret in Gaussian multi-armed bandits with appropriate variance inflation.
New method tackles confounded bandit problems with dual instrumental variables.
problem Confounded contextual bandit problems where noise affects both contexts and rewards.
method Dual instrumental variable regression applied to reproducing kernel Hilbert spaces.
result Near-optimal convergence rate and computationally efficient algorithms proved.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
Paper solves Bayesian bandit problem with continuous-time limit and approximate policy.
problem Finding optimal policy in Bayesian bandit problems with large horizons.
method Reformulates Bayesian bandit problem as continuous Hamilton-Jacobi-Bellman (HJB) equation and proposes approximate Bayes-optimal policy.
result Approximate Bayes-optimal policy for large horizons with constant computational cost.
A new framework tunes hyperparameters in real-time for contextual bandits.
problem Optimizing hyperparameters for contextual bandits in real-time.
method CDT (Continuous Dynamic Tuning) framework using Zooming TS algorithm.
result Achieves sublinear regret and performs better than existing methods.
New algorithm for partially observable contexts in finance.
problem Decision making based on partially observable, correlated market information.
method EMKF-Bandit algorithm integrating system identification, filtering, and bandit algorithms.
result Sub-linear regret under conditions on filtering.
A new algorithm reduces regret in cooperative multi-agent bandits with heavy-tailed data.
problem Cooperative multi-agent bandits with heavy-tailed data.
method MP-UCB algorithm incorporating robust estimation with message-passing protocol.
result Optimal regret bounds for MP-UCB in various settings.
New algorithms reduce slate bandit regret for large slates, outperforming existing methods.
problem Non-separable reward functions in slate bandits with many slates.
method Design of algorithms with sub-linear regret.
result Sub-linear regret with respect to the time horizon for large number of slates.
The study analyzes batched methods for early stopping in stochastic multi-armed bandits.
problem Early stopping in stochastic multi-armed bandits with fixed confidence.
method Instance-dependent lower bounds and a general batched algorithm with upper bounds.
result Upper and lower bounds on the number of batches and sample complexity.
A new bandit algorithm for web page item display.
problem Optimizing item display at specific positions on a web page.
method Position-based multiple-play bandits with Thompson sampling.
result Delivers better recommendations with less prior information.
Paper solves nonparametric contextual bandit with unbounded contexts.
problem Sequential decision making with unbounded context distributions.
method Two nearest neighbor methods combined with UCB exploration.
result Achieves minimax optimal regret under weak margin condition and light-tailed distributions.