New algorithm for partially observable contexts in finance.
problem Decision making based on partially observable, correlated market information.
method EMKF-Bandit algorithm integrating system identification, filtering, and bandit algorithms.
result Sub-linear regret under conditions on filtering.
New Thompson Sampling for partially observed context bandits reduces regret logarithmically with time.
problem Improving Thompson Sampling for partially observed context bandits.
method Proposed a Thompson Sampling algorithm for partially observable contextual multi-armed bandits with theoretical performance guarantees.
result Regret scales logarithmically with time and the number of arms, and linearly with the dimension.
Greedy policies perform poorly in imperfectly observed contextual bandits.
problem Performance of Greedy policies in bandits with partially observed contexts.
method Analysis of Greedy reinforcement learning policies under imperfectly observed contextual bandits.
result Worst-case regret grows poly-logarithmically with the time horizon and the failure probability.
We tackle linear bandits with partially observable features, achieving sublinear regret.
problem Linear regret due to unobserved features in partially observable linear bandits.
method Feature augmentation with orthogonal basis vectors and a doubly robust estimator.
result Sublinear regret bound of i l d e O ( ( d + d h ) T ) ilde{O}(\sqrt{(d + d_h)T}) i l d e O ( ( d + d h ) T ) . The paper tackles restless bandits with limited observation, proposing a method to analyze and approximate their optimal strategies.
problem Restless bandits with limited observation.
method General probabilistic model, PCL analysis, and approximation process.
result The proposed method can transform the problem into a finite-state problem, enabling the use of existing algorithms.
New algorithm learns optimal decisions from imperfectly observed contexts.
problem Learning optimal decisions in bandits with unobserved contexts.
method Posterior sampling algorithm for imperfectly observed contexts.
result Efficient learning from noisy imperfect observations.
ARC algorithm optimizes dynamic pricing with correlated observations.
problem Optimizing dynamic pricing with correlated and generally distributed observations.
method Extends ARC algorithm to batched bandits with generalised linear model.
result ARC algorithm outperforms alternative approaches in dynamic pricing.
Investigates sequential problems on graph structures and large action spaces.
problem Sequential decision-making on graph structures and large action spaces.
method Spectral bandits, side observations, influence maximization, kernel bandits, polymatroid bandits, function optimization, infinitely many-arms bandits.
result Contributions to graph and structured bandits.
A new algorithm CAP learns optimal policies from observational data with confounding bias and missing observations.
problem Offline contextual bandit with confounding bias and missing observations.
method CAP policy learning, forming reward function as solution of integral equation system, building confidence set, and greedily taking action with pessimism.
result Developed an upper bound to the suboptimality of CAP for the offline contextual bandit problem.
Adaptive algorithm reduces regret in causal bandits.
problem Minimize regret in causal bandits with unknown d-separators.
method Adaptive algorithm exploiting d-separators without prior knowledge.
result Significantly smaller regret than previous methods.
We use online convex optimization (OCO) for setpoint tracking with uncertain, flexible loads. We consider full feedback from the loads, bandit feedback, and two intermediate types of feedback: partial bandit where a subset of the loads are individually observed and the rest are observed in aggregate, and Bernoulli feed…
Study invariant Lipschitz bandits, improving regret bounds.
problem Optimizing decisions under symmetry in online settings.
method Integrates side observations using group orbits into UniformMesh algorithm.
result Improved regret bound for invariant Lipschitz bandit class.
We obtain the conditions for the emergence of the swarm intelligence effect in an interactive game of restless multi-armed bandit (rMAB). A player competes with multiple agents. Each bandit has a payoff that changes with a probability p c p_{c} p c per round. The agents and player choose one of three options: (1) Exploit (a …
Proposes a method to learn from historical data for personalized decision-making.
problem Sample hunger in sequential decision-making algorithms for personalized medicine.
method Identifiable latent bandit framework using nonlinear independent component analysis.
result Optimal decision-making with shorter exploration time than classical bandits.
Motivated by clinical trials, we study bandits with observable non-compliance. At each step, the learner chooses an arm, after, instead of observing only the reward, it also observes the action that took place. We show that such noncompliance can be helpful or hurtful to the learner in general. Unfortunately, naively i…
New bandit algorithm for non-i.i.d. noise, improving standard rates.
problem Linear stochastic bandit with non-i.i.d. observation noise.
method Developed new confidence sequences and an algorithm based on optimism in uncertainty.
result Regret bounds for the new algorithm, showing recovery of standard rates up to a factor of the mixing time.
Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of k k k out of n n n arms may be chosen and the sum of the rewards is gained. We address the full-bandit feedback, in which the agent observes only the sum of rewards, in contrast to the semi-bandit feedback, in which the agent obse…
Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function cl…
New algorithm for contextual dueling bandits achieves nearly optimal regret.
problem Contextual dueling bandits with feedback on preferred options.
method Proposes FGTS.CDB, a Thompson sampling algorithm for linear contextual dueling bandits.
result Achieves nearly minimax-optimal regret of i l d e O ( d T ) ilde{\mathcal{O}}(d\sqrt T) i l d e O ( d T ) . Algorithm improves decision-making with partially observed contexts using pretrained models.
problem Improving decision-making with partially observed contexts in online linear contextual bandits.
method PULSE-UCB algorithm that uses pretrained models trained on auxiliary data to impute missing features.
result Achieves near-optimal performance in i.i.d. context case with Hölder-smooth missing features.
We consider the partial observability model for multi-armed bandits, introduced by Mannor and Shamir. Our main result is a characterization of regret in the directed observability model in terms of the dominating and independence numbers of the observability graph. We also show that in the undirected case, the learner …
Thompson Sampling tackles noisy context in stochastic bandits.
problem Designing an action policy for noisy, corrupted contexts in stochastic bandits.
method Introducing a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise, adopting an information-theoretic analysis.
result Demonstrates the Bayesian regret of the proposed algorithm concerning the oracle's action policy.
CORe uses randomization to explore bandit problems without external noise.
problem Exploration in stochastic bandit problems.
method Randomizes past observations to exploit variance in rewards.
result Achieves i l d e O ( d n log K ) ilde O(d\sqrt{n\log K}) i l d e O ( d n log K ) regret bound in stochastic linear bandits. Study online learning with off-policy feedback in adversarial bandit problems.
problem Learning with limited direct feedback in sequential decision making.
method Proposed algorithms that adapt pessimistic reward estimators to handle unknown behavior policy.
result Guaranteed regret bounds scaling with policy mismatch, improving performance against well-covered comparators.
New algorithm improves bandit with graph feedback by decomposing regret.
problem Improving performance in bandit problems with graph feedback.
method Partition-based algorithm framework using regret decomposition.
result Improved and optimal regret bounds on various graph families.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
New method uses correlated auxiliary feedback to reduce regret in parameterized bandits.
problem Reducing regret in parameterized bandits with correlated auxiliary feedback.
method Develops a reward estimator using auxiliary feedback with tight confidence bounds.
result Shows significant reduction in regret compared to standard methods.
This paper tackles open problem of tight bounds for KBs with Bernoulli rewards.
problem Open problem of tight bounds for Kernelized Bandits with Bernoulli rewards.
method Focus on Bernoulli model, not subgaussian noise, and optimize function in RKHS.
result Open problem remains unsolved in this context.
Observer learns optimal policy from learner's actions without rewards.
problem Learning optimal policy from non-rewarded actions of a non-stationary learner.
method Two-Phase Suffix Imitation framework.
result Observer achieves convergence rate of O ~ ( 1 / N ) \tilde O(1/\sqrt{N}) O ~ ( 1/ N ) . Two algorithms minimize regret in adversarial bandit problems with side-observation losses.
problem Minimizing regret in adversarial multi-armed bandit problems with side-observation losses.
method Proposes two algorithms for different ranges of side-observation probability.
result Regret bounds for different values of side-observation probability.
A new algorithm for resource-aware multi-armed bandits minimizes regret.
problem Optimizing resource usage in a multi-armed bandit problem with censored observations.
method UCB-inspired online learning algorithm with theoretical regret analysis.
result The proposed algorithm outperforms standard multi-armed bandit algorithms in simulations.
New algorithms for efficient learning with partial information, reducing regret.
problem Online learning with partial observability and semi-bandit feedback.
method Implicit exploration strategy for near-optimal regret guarantees.
result First algorithms with near-optimal regret guarantees without knowing the observation system.
Bayesian optimization improved for biased data.
problem Adversarial bias in observations, especially hidden confounders.
method Reduction to dueling bandits, information-directed sampling (IDS).
result First efficient kernelized algorithm with regret guarantees.
Algorithm allocates budgets to tasks with semi-bandit feedback, achieving near-optimal regret bounds.
problem Stochastic budget allocation with censored semi-bandit feedback.
method Optimism-based algorithm operating under censored semi-bandit feedback.
result Regret scales polylogarithmically with horizon T in diminishing-returns regimes.
GAMBITTS uses GenAI for adaptive interventions, improving decision-making.
problem Adaptive interventions with GenAI-generated content.
method Generator-mediated bandit-Thompson sampling (GAMBITTS).
result GAMBITTS outperforms standard bandit methods in mobile health interventions.
New MAB model for online caching costs.
problem Learning costs of cached items online.
method Synchronization bandits, MirrorSync algorithm.
result Adversarial regret of O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) for MirrorSync. A new framework for structured bandits using influence diagrams and variational Thompson sampling.
problem Complex statistical dependencies in structured bandit problems.
method Influence diagram framework, variational Thompson sampling, tracking structured posterior distribution.
result Empirically evaluated algorithms perform as well as or better than existing baselines.
Paper addresses privacy and robustness in stochastic linear bandits.
problem Stochastic linear bandits with differential privacy and adversarial robustness.
method Logarithmic batch queries, arm elimination algorithm, two privacy models.
result First algorithms providing differential privacy and adversarial robustness.
We consider the stochastic multi-armed bandit problem and the contextual bandit problem with historical observations and pre-clustered arms. The historical observations can contain any number of instances for each arm, and the pre-clustering information is a fixed clustering of arms provided as part of the input. We de…
New method for contextual bandit with missing rewards.
problem Contextual bandit with missing rewards in online settings.
method Combining contextual bandit approach with unsupervised learning (clustering) to estimate missing rewards.
result Promising empirical results on real-life datasets.
The paper tackles bandit problems with biased offline data by using causal methods.
problem Improving bandit algorithms with biased offline data that includes confounding and selection biases.
method Formalizes the problem from a causal perspective, categorizes biases, and derives robust bounds for each arm.
result Causal bounds can guide the bandit agent to learn a nearly-optimal decision policy and consistently reduce asymptotic regret.
New algorithms tackle adversarial combinatorial bandits with switching costs.
problem Adversarial combinatorial bandits with switching costs.
method Design algorithms operating in batches to restrict switches, proving lower bounds and achieving upper bounds on regret.
result Achieved upper bounds on regret for both bandit and semi-bandit feedback settings.
This paper addresses missing covariates in stochastic linear bandits, providing a high-probability regret bound.
problem Effect of missing covariates on regret in stochastic linear bandit algorithms.
method Proposes an algorithm that provides a high-probability upper bound on regret in terms of covariate sampling probabilities.
result Regret degrades due to missingness by at most ζ m i n 2 ζ_{min}^2 ζ min 2 , where ζ m i n ζ_{min} ζ min is the minimum probability of observing covariates. Thompson Sampling improves decision-making in partially observed contexts.
problem Balancing exploration and exploitation in partially observed contextual bandits.
method Thompson Sampling policy for learning optimal arms from noisy linear functions of unobserved context vectors.
result Thompson Sampling achieves poly-logarithmic regret and square-root consistency of parameter estimation.
PAC-Bayesian analysis improves lifelong learning in multi-armed bandits.
problem Improving lifelong learning in multi-armed bandits.
method PAC-Bayesian analysis for deriving lower bounds and proposing lifelong learning algorithms.
result Proposed algorithms outperform baseline methods in lifelong multi-armed bandit problems.
New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
We analyze how batch learning impacts bandit problems.
problem Impact of batch learning in stochastic bandits.
method Policy-agnostic regret analysis, upper and lower bounds demonstration.
result The impact of batch learning can be measured in terms of online behavior.
Study linear contextual bandits with confounded offline data, improving regret bounds.
problem Linear contextual bandits with confounded offline data.
method Construct a linear bandit algorithm that utilizes projected information.
result Proved regret bounds that improve current bounds by a factor related to visible dimensionality.