arXiv research

CADR estimator improves inference for contextual bandit data.

Online learning algorithm identifies contextual changes for streaming data.

IDS improves reinforcement learning with contextual information.

We study offline data poisoning attacks in contextual bandits, a class of reinforcement learning problems with important applications in online recommendation and adaptive medical treatment, among others. We provide a general attack framework based on convex optimization and show that by slightly manipulating rewards i…

Study identifies methods for handling sparse data in web settings with contextual bandits.

The paper tackles learning varying DAG structures based on contextual features.

Novel approach to contextual bandits using self-supervised learning.

Contextual bandits with linear payoffs, which are also known as linear bandits, provide a powerful alternative for solving practical problems of sequential decisions, e.g., online advertisements. In the era of big data, contextual data usually tend to be high-dimensional, which leads to new challenges for traditional l…

A method for unsupervised contextual anomaly detection is proposed using a cross-linked pair of Variational Auto-Encoders for assigning a normality score to an observation. The method enables a distinct separation of contextual from behavioral attributes and is robust to the presence of anomalous or novel contextual at…

Gaussian Process model improves blood glucose prediction using contextual data.

Active learning method improves AI performance by balancing exploration and exploitation.

A new algorithm for cryo-EM data collection that balances reward and latency.

CACTI improves tabular data imputation by leveraging missingness patterns and contextual information.

COPP provides reliable intervals for outcomes under a new policy in contextual bandits.

We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information. Algorithms that are designed solely for one of the regimes are known to be sub-optimal for the alternate regime…

GLCB uses Gated Linear Networks for online contextual bandits.

New algorithms for model selection in linear contextual bandits without feature diversity conditions.

A new algorithm avoids worst-case outcomes in risky contexts.

We improve optimization for data with varying variance.

CAMS selects best pre-trained model for unlabeled data points.

Bayesian method infers contextual bandit policies robustly.

Paper improves Thompson Sampling for linear contextual bandits.

New algorithm estimates treatment effects for more efficient contextual bandits.

Proposes a new algorithm for graph-based semi-parametric contextual bandits.

New algorithm for contextual dueling bandits achieves nearly optimal regret.

The article presents methods to select models from behavioral learning data, with applications to contextual bandits.

LMC-TS uses MCMC for efficient posterior sampling in contextual bandits.

Paper tackles transfer learning for contextual multi-armed bandits under covariate shift.

Contextual bandit algorithms provide principled online learning solutions to balance the exploitation-exploration trade-off in various applications such as recommender systems. However, the learning speed of the traditional contextual bandit algorithms is often slow due to the need for extensive exploration. This poses…

Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different task contexts. Contextual policy search offers data-efficient learning and generalization by explicitly conditioning the policy on a parametric context space. In this paper…

Study contextual online pricing with biased offline data, achieving optimal regret bounds.

Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We develop algorithms for contextual …

A new algorithm CAP learns optimal policies from observational data with confounding bias and missing observations.

This paper improves offline contextual bandits using distributional robustness.

Bayesian classifier improves robustness with optimistic score ratio.

Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We study a consideration for the expl…

Meta learning improves with contextualizers that adapt to examples.

Contextual multi-armed bandit problems arise frequently in important industrial applications. Existing solutions model the context either linearly, which enables uncertainty driven (principled) exploration, or non-linearly, by using epsilon-greedy exploration policies. Here we present a deep learning framework for cont…

Privacy-preserving multi-party contextual bandits learn without sharing data.

Improved off-policy selection and learning in contextual bandits with better guarantees.

Study linear contextual bandits with confounded offline data, improving regret bounds.

Contextualized ML learns context-dependent effects using deep learning.

We describe MELEE, a meta-learning algorithm for learning a good exploration policy in the interactive contextual bandit setting. Here, an algorithm must take actions based on contexts, and learn based only on a reward signal from the action taken, thereby generating an exploration/exploitation trade-off. MELEE address…

Contextual bandits are a form of multi-armed bandit in which the agent has access to predictive side information (known as the context) for each arm at each time step, and have been used to model personalized news recommendation, ad placement, and other applications. In this work, we propose a multi-task learning frame…

The paper improves off-policy evaluation in contextual bandits using conformal prediction.

Aims to improve personalized treatment decisions through Bayesian experimental design.

We study the off-policy evaluation problem---estimating the value of a target policy using data collected by another policy---under the contextual bandit model. We consider the general (agnostic) setting without access to a consistent model of rewards and establish a minimax lower bound on the mean squared error (MSE).…

The paper improves DR estimators for off-policy evaluation in contextual bandits.