A new policy for contextual bandits adapts to reward vector shifts.
problem Learning under reward vector shifts with ordered rewards.
method Adaptive-discretization and optimistic elimination policy.
result Established upper bounds on preference-based regret.
We study offline data poisoning attacks in contextual bandits, a class of reinforcement learning problems with important applications in online recommendation and adaptive medical treatment, among others. We provide a general attack framework based on convex optimization and show that by slightly manipulating rewards i…
A new algorithm learns from raw feature vectors using deep neural networks and UCB for exploration.
problem Learning from raw feature vectors with unknown reward functions.
method Deep representation learning followed by UCB exploration in the last layer.
result Achieves i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) finite-time regret. Model predicts political ideology using context vectors to mitigate bias and scarcity.
problem Scarcity and selection bias in political ideology prediction.
method Proposes a statistical model decomposing embeddings into context and position vectors, training an end-to-end model for deployment.
result Model can predict ideological labels even with minimal biased data, outperforming state-of-the-art methods.
Algorithm reduces regret in misspecified linear contextual bandits.
problem Misspecified linear contextual bandits with bounded misspecification.
method Data selection scheme for online regression, leveraging uncertainty.
result Regret bound of O ~ ( d 2 / Δ ) \tilde O(d^2/Δ) O ~ ( d 2 /Δ ) when ζ ≤ O ~ ( Δ / d ) ζ \leq \tilde O(Δ/\sqrt{d}) ζ ≤ O ~ ( Δ/ d ) . New algorithms reduce costly feature collection in bandits.
problem Costly feature collection in contextual bandits.
method Proposes algorithms avoiding unnecessary feature collection.
result Strong regret guarantees maintained with reduced feature collection.
Predict stock movement with news headlines using BERT embeddings.
problem Predicting stock price movement after financial news.
method Fine-Tuned Contextualized-Embedding Recurrent Neural Network (FT-CE-RNN) using BERT.
result Obtains state-of-the-art results on stock movement prediction task.
Study on adaptivity constraints in linear contextual bandits with optimal design.
problem Impact of adaptivity constraints on linear contextual bandits.
method Two models of limited adaptivity: batch learning and rare policy switches. Proposed distributional optimal design.
result Achieves minimax-optimal regret with optimal number of policy switches and batches.
We consider an online decision making setting known as contextual bandit problem, and propose an approach for improving contextual bandit performance by using an adaptive feature extraction (representation learning) based on online clustering. Our approach starts with an off-line pre-training on unlabeled history of co…
Bayesian classifier improves robustness with optimistic score ratio.
problem Limited information on class-conditional distribution.
method Optimistic score ratio for robust binary classification.
result Bayesian classifier using optimistic score ratio is robust and computationally tractable.
The bundle approach and n-contextuality reveal quantum model contextuality.
problem Understanding contextuality in quantum models using topology.
method Using the bundle approach, we describe contextuality as the non-existence of global sections in the measure bundle. We introduce n-contextuality to explore model dependence on scenario topology.
result Quantum theory and GHZ models exhibit all levels of n-contextuality, showing contextuality is related to holonomy group non-triviality.
A simple algorithm reduces federated contextual linear bandits' regret efficiently.
problem Solving federated contextual linear bandits with asynchronous agents.
method Proposed a simple algorithm exttt{FedLinUCB} based on optimism principle.
result Proved exttt{FedLinUCB} has bounded regret i l d e O ( d ∑ m = 1 M T m ) ilde{O}(d\sqrt{\sum_{m=1}^M T_m}) i l d e O ( d ∑ m = 1 M T m ) and communication complexity i l d e O ( d M 2 ) ilde{O}(dM^2) i l d e O ( d M 2 ) . This paper explores the complexity of learning representations in contextual linear bandits.
problem Understanding the complexity of representation learning in contextual linear bandits.
method Systematic approach to representation learning in contextual linear bandits, focusing on instance-dependent perspective.
result Representation learning is fundamentally more complex than linear bandits, with some cases being arbitrarily harder.
We revisit the proof by Qin et al. (2014) of bounded regret of the C 2 ^2 2 UCB contextual combinatorial bandit. We demonstrate an error in the proof of volumetric expansion of the moment matrix, used in upper bounding a function of context vector norms. We prove a relaxed inequality that yields the originally-stated regre…
Optimizes decision-making in dueling bandits with contextual features.
problem Identifying the best arm in dueling bandits with contextual features.
method Develops algorithms for minimizing regret in stochastic contextual dueling bandits.
result Proves optimal regret bounds for contextual dueling bandits.
SLiCE learns contextual node embeddings for link prediction in heterogeneous networks.
problem Link prediction requires specific contextual information not captured by static node embeddings.
method Self-supervised pre-training with localized attention mechanisms.
result SLiCE significantly outperforms existing methods on link prediction tasks.
A new algorithm for personalized recommendations adapts to changing user interests.
problem Adapting to time-varying user interests in recommendation systems.
method Contextual bandit approach with models for disjoint and hybrid payoffs.
result Sublinear regret in time length T for abrupt reward changes.
Two algorithms address limited adaptivity in generalized linear contextual bandits.
problem Limited adaptivity in generalized linear contextual bandits.
method Two algorithms, B-GLinCB and RS-GLinCB, designed for two settings of limited adaptivity.
result Achieved i l d e O ( T ) ilde{O}(\sqrt{T}) i l d e O ( T ) regret in both settings. Algorithm minimizes regret in dueling bandits with contextualized utilities.
problem Minimizing regret in dueling bandits with context-dependent utilities.
method Proposes CoLSTIM algorithm based on perturbed utility estimates.
result Achieves regret of order i l d e O ( d T ) ilde O(\sqrt{dT}) i l d e O ( d T ) . A collaborative algorithm reduces regret in federated linear contextual bandits.
problem Optimizing decision-making in federated learning with heterogeneous data.
method Fed-PE algorithm, leveraging geometric structure of rewards, multi-client G-optimal design.
result Achieves near-optimal regrets with logarithmic communication costs.
The paper tackles minimax optimality in continuum contextual bandits with Hölder continuity.
problem Minimizing regret in a continuum of contexts with Hölder continuity.
method Proves a static-to-contextual regret conversion theorem and analyzes various dependency cases.
result Achieves minimax optimal contextual regret for convex and strongly convex bandits.
Transformer learns context and regularization for ICL in inverse problems.
problem Learning context and effective regularization for transformer-based in-context learning (ICL) in inverse problems.
method Introduced a linear transformer to learn inverse mapping from contextual examples to weight vectors, addressing rank-deficient problems.
result Transformer implicitly learns a prior distribution and effective regularization strategy, outperforming traditional methods.
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
Optimal algorithm for maximizing rewards in contextual bandits with resource constraints.
problem Maximizing rewards in contextual bandits with resource constraints.
method Proposed a universal and optimal algorithmic framework for CBwK by reducing it to online regression.
result Established the optimality of the proposed algorithm for various function classes.
Improved algorithms solve multi-period multi-class packing problems with bandit feedback.
problem Optimizing item packing under budget constraints with class-dependent rewards and bandit feedback.
method Developed a new estimator and a closed-form bandit policy for linear contextual multi-class multi-period packing problems.
result The proposed policy achieves sublinear regret in non-degenerate contexts, significantly outperforming benchmarks.
Greedy policies perform poorly in imperfectly observed contextual bandits.
problem Performance of Greedy policies in bandits with partially observed contexts.
method Analysis of Greedy reinforcement learning policies under imperfectly observed contextual bandits.
result Worst-case regret grows poly-logarithmically with the time horizon and the failure probability.
Contextual bandits are widely used in Internet services from news recommendation to advertising, and to Web search. Generalized linear models (logistical regression in particular) have demonstrated stronger performance than linear models in many applications where rewards are binary. However, most theoretical analyses …
Paper introduces G-LowTESTR for efficient tensor bandits.
problem Efficient decision-making in multi-dimensional data with non-linear reward functions.
method Generalized low-rank tensor contextual bandits model and G-LowTESTR algorithm.
result G-LowTESTR achieves superior regret bound compared to vectorization and matricization methods.
New algorithm for fair ranking in contextual bandits with concave rewards.
problem Fair ranking in recommendation systems.
method Geometric interpretation of CBCR as optimization, Frank-Wolfe analyses.
result First algorithm with provably vanishing regret for CBCR.
Proposes EE-Net for neural exploration in contextual bandits.
problem Exploitation-Exploration tradeoff in contextual bandits.
method Uses two neural networks: Exploitation and Exploration, to learn reward function and adaptively explore.
result Achieves O ( T log T ) \mathcal{O}(\sqrt{T\log T}) O ( T log T ) regret and outperforms existing methods. Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.
problem The performance gap between value-based and policy-based algorithms in offline contextual bandits with overparameterized models.
method Analysis of action-stability in objectives and formal proofs of regret bounds.
result The performance gap is due to action-stability of objectives, with value-based objectives being stable and policy-based objectives unstable.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
Modified Meta-TS for linear contextual bandits reduces regret.
problem Optimizing decision-making in dynamic environments with context vectors.
method Meta-TSLB algorithm for linear contextual bandits, analyzing Bayes regret.
result Derives an O ( ( m + log ( m ) ) n log ( n ) ) O((m+\log(m))\sqrt{n\log(n)}) O (( m + log ( m )) n log ( n ) ) bound on Bayes regret. Algorithm maximizes revenue from user choices with contextual information.
problem Maximizing revenue from user choices with contextual preference information.
method Proposes an algorithm that learns from user feedback and achieves a revenue regret of order \( \widetilde{O}(d \sqrt{K T} / L_0 ) \).
result Achieves a revenue regret of order \( \widetilde{O}(d \sqrt{K T} / L_0 ) \) and a lower bound of order \( \Omega(d \sqrt{T}/ L_0) \).
Paper tackles binary feedbacks in contextual search learning.
problem Learning underlying mean value function in context with binary feedbacks.
method Tri-section search combined with margin-based active learning.
result Algorithm achieves O ( 1 / ε 2 ) O(1/\varepsilon^2) O ( 1/ ε 2 ) queries for ε ε ε -estimation accuracy. Prominently used in support vector machines and logistic regressions, kernel functions (kernels) can implicitly map data points into high dimensional spaces and make it easier to learn complex decision boundaries. In this work, by replacing the inner product function in the softmax layer, we explore the use of kernels …
We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…
Improved regret bounds for contextual combinatorial semi-bandits with linear payoffs.
problem Maximizing rewards in decision-making problems with feature vectors and constraints.
method Proposed C^2UCB algorithm and modified reward estimates for general constraints.
result Optimal regret bounds of C^2UCB algorithm and modified algorithm for various constraints.
New algorithms reduce sample complexity for multiclass contextual bandits.
problem Designing efficient algorithms for multiclass contextual bandits with sparse rewards.
method Two complementary approaches: decision-estimation coefficient analysis and low-variance exploration.
result Achieved optimal sample complexity bounds for multiclass contextual bandits.
Context-aware recommender systems (CARSs) apply sensing and analysis of user context in order to provide personalized services. Adding context to a recommendation model is challenging, since the addition of context may increases both the dimensionality and sparsity of the model. Recent research has shown that modeling …
MOL-TS uses Thompson Sampling for multi-objective linear bandits with Pareto guarantees.
problem Optimizing multiple conflicting objectives in linear contextual bandits.
method Proposes MOL-TS, a Thompson Sampling algorithm with Pareto regret guarantees.
result Achieves a worst-case Pareto regret bound of O ~ ( d 3 / 2 T ) \widetilde{O}(d^{3/2}\sqrt{T}) O ( d 3/2 T ) . New algorithm resists corruption in linear contextual bandits.
problem Adversarial corruption in linear contextual bandits.
method Variance-aware algorithm with multi-level partition and adaptive confidence sets.
result Regret bound of i l d e O ( C 2 d ∑ t = 1 T σ t 2 + C 2 R d T ) ilde{O}(C^2d\sqrt{\sum_{t = 1}^T σ_t^2} + C^2R\sqrt{dT}) i l d e O ( C 2 d ∑ t = 1 T σ t 2 + C 2 R d T ) . New algorithms robust to adversarial data achieve optimal performance.
problem Adversarial robustness in high-dimensional online learning problems.
method Alternating minimization scheme combining least-squares and convex reweighting.
result Achieves optimal robustness guarantees without distributional assumptions.
Decentralized learning for matching markets with time-varying preferences.
problem Matching between competing agents and supply arms with time-varying preferences.
method Linear contextual bandit framework, learning algorithms to identify latent environment and stable matchings.
result Achieve instance-dependent logarithmic regret, applicable for large markets.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
A new model optimizes portfolios by learning stock return distributions conditioned on factors.
problem Optimizing portfolios with high-dimensional asset-specific factors.
method Conditional Diffusion Transformer architecture linking each asset's return to its factor vector.
result The model outperforms benchmarks in mean-variance and mean-CVaR optimization.
We consider the linear contextual bandit problem with resource consumption, in addition to reward generation. In each round, the outcome of pulling an arm is a reward as well as a vector of resource consumptions. The expected values of these outcomes depend linearly on the context of that arm. The budget/capacity const…