New method tackles high-dimensional contextual bandits with flexible kernel models.
problem Maximizing rewards in decision-making scenarios with many features.
method Introduces stochastic assumptions and no-regret learning for Gaussian kernels.
result Achieves no-regret learning even with feature dimensions growing with samples.
New algorithm reduces neural net error in contextual bandits.
problem Neural contextual bandits with general activation functions.
method Proposed an efficient algorithm with sublinear regret bound.
result Demonstrated provably sublinear regret bound in finite regime.
A new algorithm for differential privacy in kernelized contextual bandits reduces error rate.
problem Joint differential privacy in kernelized contextual bandits.
method Proposes a novel algorithm with a specific error rate and privacy parameter dependence.
result Achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}
ight)$ after T T T queries. A new algorithm tackles adversarial linear contextual bandits using kernelized loss functions.
problem Online learning in adversarial linear contextual bandits with flexible loss functions.
method Proposes a computationally efficient algorithm using an optimistically biased estimator for reproducing kernel Hilbert space loss functions.
result Achieves near-optimal regret guarantees under polynomial and exponential eigendecay assumptions.
Kernelized bandit algorithm tackles adaptive contextual bandits with single-index models.
problem Adaptive contextual bandits with single-index models and unknown link functions.
method Kernelized ε-greedy algorithm combining Stein-based index estimation and kernel ridge regression for reward functions.
result Unified framework for simultaneous learning and inference in single-index contextual bandits.
New algorithms use neural networks to minimize regret in contextual bandits.
problem Minimizing regret in sequential decisions with side information.
method Proposes NN-UCB and NTK-UCB algorithms using neural networks.
result Proves sublinear regret bounds for contextual bandits.
Study sparsity benefits in infinite feature contextual bandits.
problem Minimizing regret in infinite feature contextual bandits.
method Novel reduction to multi-armed bandits, Feel-Good Thompson Sampling algorithm.
result Regret bounds match lower bounds up to logarithmic factors, logarithmic dependence on effective features.
Kernel method improves cooperative decision-making among agents.
problem Cooperative multi-agent decision making with contextual information.
method Proposed extsc{Coop-KernelUCB} algorithm for near-optimal per-agent regret.
result Near-optimal bounds on per-agent regret with efficient computation and communication.
Explearn learns to explain predictions using Gaussian Processes.
problem Learning to explain predictions effectively.
method Gaussian Processes-based contextual bandits.
result Guaranteed convergence with high probability.
New algorithm for differential private kernel bandits with improved regret bounds.
problem Contextual bandits with differential privacy constraints.
method Proposes a private kernel-ridge regression estimator using random projections.
result Achieves state-of-the-art cumulative regret bounds in both joint and local models of differential privacy.
We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward…
Unified framework for multi-user bandits using Laplacian kernels.
problem Multi-user contextual bandits with graph-related users and non-linear rewards.
method Joint penalty combining graph smoothness and individual roughness in a unified RKHS.
result Unified multi-user RKHS and effective dimension for regret bounds.
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
New method tackles confounded bandit problems with dual instrumental variables.
problem Confounded contextual bandit problems where noise affects both contexts and rewards.
method Dual instrumental variable regression applied to reproducing kernel Hilbert spaces.
result Near-optimal convergence rate and computationally efficient algorithms proved.
Sharp policy value estimation for contextual bandits with unobserved confounders.
problem Estimating policy value under unobserved confounders with sensitivity analysis.
method Kernel method to approximate conditional moment constraints, leveraging f-divergence.
result Sharp lower bound of policy value, avoiding coarse relaxation of uncertainty set.
New algorithm tackles non-linear utility in MNL bandits with i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret.
problem Sequential assortment selection with intricate user-item interactions.
method Upper Confidence Bound principle for non-linear parametric utility functions, including neural networks.
result Achieves i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret bound for neural network-based utilities. Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently identifying good policies in applications with high feedback costs.
method Introduces offline contextual dueling bandit setting and an upper-confidence-bound style algorithm.
result Proves a regret bound and shows superior performance over uniformly sampled contexts.
Paper solves stochastic contextual linear bandits using linear bandit algorithms.
problem Stochastic contextual linear bandits with unknown context distribution.
method Establishes a reduction framework to convert to linear bandit problems.
result Achieves nearly optimal regret bound of O ( d T log T ) O(d\sqrt{T\log T}) O ( d T log T ) . Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.
problem Improving the performance of Thompson Sampling in contextual bandits with sub-Gaussian rewards.
method Proved comprehensive bounds on Thompson Sampling expected cumulative regret based on mutual information and lifted information ratio for sub-Gaussian rewards.
result Explicit regret bounds for various contextual bandit scenarios.
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
The study explores whether model selection guarantees apply to contextual bandits.
problem Applying model selection guarantees to contextual bandits.
method Investigates whether similar guarantees for model selection in statistical learning can be extended to contextual bandit learning.
result Initial findings suggest that model selection guarantees may not directly apply to contextual bandits.
Faster algorithm reduces contextual bandit regret with fewer offline regression calls.
problem Optimizing reward in contextual bandits with unknown functions.
method Designing a simple algorithm with O ( log T ) {O}(\log T) O ( log T ) offline regression calls. result Achieves statistically optimal regret with minimal offline calls.
Efficiently handles contextual bandits with diffusion models.
problem Challenges in online decision-making with contextual bandits.
method Leverage pre-trained diffusion models as priors to capture action dependencies.
result Developed an algorithm for efficient posterior approximation.
New algorithms robust to adversarial data achieve optimal performance.
problem Adversarial robustness in high-dimensional online learning problems.
method Alternating minimization scheme combining least-squares and convex reweighting.
result Achieves optimal robustness guarantees without distributional assumptions.
IDS improves reinforcement learning with contextual information.
problem Optimizing IDS for contextual reinforcement learning.
method Investigated contextual bandit problems and proposed a computationally-efficient IDS.
result Contextual IDS outperforms conditional IDS by considering future contexts.
First robust bandit algorithm for contextual bandits with sub-linear regret.
problem Vulnerability of linear contextual bandit algorithms to adversarial attacks.
method Proposes a robust bandit algorithm for stochastic linear contextual bandits under fully adaptive and omniscient attacks.
result Sub-linear regret under various attacks without requiring attack information.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
New method for contextual bandit with missing rewards.
problem Contextual bandit with missing rewards in online settings.
method Combining contextual bandit approach with unsupervised learning (clustering) to estimate missing rewards.
result Promising empirical results on real-life datasets.
New algorithm for contextual dueling bandits achieves nearly optimal regret.
problem Contextual dueling bandits with feedback on preferred options.
method Proposes FGTS.CDB, a Thompson sampling algorithm for linear contextual dueling bandits.
result Achieves nearly minimax-optimal regret of i l d e O ( d T ) ilde{\mathcal{O}}(d\sqrt T) i l d e O ( d T ) . Cellular network configuration plays a critical role in network performance. In current practice, network configuration depends heavily on field experience of engineers and often remains static for a long period of time. This practice is far from optimal. To address this limitation, online-learning-based approaches hav…
The paper tackles minimax optimality in continuum contextual bandits with Hölder continuity.
problem Minimizing regret in a continuum of contexts with Hölder continuity.
method Proves a static-to-contextual regret conversion theorem and analyzes various dependency cases.
result Achieves minimax optimal contextual regret for convex and strongly convex bandits.
New algorithms for neural bandits learn from context and arm features.
problem Learning from context and arm features in combinatorial bandit problems.
method Proposed algorithms CN-UCB and CN-TS using deep neural networks.
result Achieved regret guarantees for the first time in combinatorial neural bandits.
New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
Contextual bandit algorithms provide principled online learning solutions to balance the exploitation-exploration trade-off in various applications such as recommender systems. However, the learning speed of the traditional contextual bandit algorithms is often slow due to the need for extensive exploration. This poses…
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information. Algorithms that are designed solely for one of the regimes are known to be sub-optimal for the alternate regime…
GLCB uses Gated Linear Networks for online contextual bandits.
problem Online learning in contextual bandits with uncertainty estimation.
method Gated Linear Networks (GLNs) for prediction and uncertainty estimation.
result GLCB outperforms state-of-the-art methods in online contextual bandits.
LMC-TS uses MCMC for efficient posterior sampling in contextual bandits.
problem Efficiency of Thompson sampling for high-dimensional contextual bandits.
method Langevin Monte Carlo for direct posterior sampling.
result LMC-TS achieves sublinear regret bound for linear contextual bandits.
New action poisoning attacks improve LinUCB's performance by changing action signals.
problem Improving understanding of adversarial attacks on contextual bandit algorithms.
method Proposed action poisoning attacks in white-box and black-box settings.
result Action poisoning attacks can force LinUCB to pull a target arm frequently with low cost.
We consider an online decision making setting known as contextual bandit problem, and propose an approach for improving contextual bandit performance by using an adaptive feature extraction (representation learning) based on online clustering. Our approach starts with an off-line pre-training on unlabeled history of co…
Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We study a consideration for the expl…
This paper achieves optimal regret bounds for locally private linear contextual bandit.
problem Designing locally private linear contextual bandit algorithms with optimal regret bounds.
method New algorithmic and analytical ideas, including mean absolute deviation analysis and layered principal component regression.
result Achieves an i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret upper bound for locally private linear contextual bandit. New algorithm for contextual bandits with linear constraints using regression.
problem Contextual bandits with packing and covering constraints.
method Modular Lagrangian approach via regression.
result First vanishing-regret guarantees for CBwLC (or CBwK).
New approach combines semi-supervised learning and bandits for better predictions.
problem Online semi-supervised learning with bandit feedback for applications like clinical trials and ad recommendations.
method Adjusted Graph Convolutional Network (GCN) for contextual bandits, with semi-supervised missing rewards imputation.
result Developed multi-GCN embedded contextual bandit algorithms verified on real-world datasets.
New algorithm for context bandits with continuous actions.
problem Efficient decision-making with unknown action structures.
method Reduction-style algorithm combining supervised learning.
result Proven to work in general and validated with experiments.
A new algorithm avoids worst-case outcomes in risky contexts.
problem Risk-averse behavior in contextual bandits is challenging.
method Developed a first risk-averse contextual bandit algorithm with online regret guarantees.
result First algorithm with an online regret guarantee for risk-averse contextual bandits.