Improves online learning with expert demonstrations, quality matters.
problem Improving online learning through offline demonstration data.
method Thompson sampling applied to a multi-armed bandit model, informed by expert demonstrations and Bayes' rule.
result Substantial empirical regret reduction with expert demonstrations, improving online performance.
Unified Bayesian framework for efficient off-policy evaluation and learning in large action spaces.
problem Efficient off-policy evaluation and learning in systems with correlated actions.
method Unified Bayesian framework with structured priors and sDM approach.
result sDM leverages action correlations without compromising computational efficiency.
Improved Thompson Sampling reduces regret in contextual bandits and reinforcement learning.
problem Thompson Sampling's exploration is insufficient in some contexts.
method Developed Feel-Good Thompson Sampling to address exploration issues.
result Feel-Good Thompson Sampling reduces regret compared to standard Thompson Sampling.
IDS improves sparse linear bandits by balancing information and regret.
problem Sparse linear bandits in high-dimensional decision-making.
method Information-directed sampling (IDS) with Bayesian regret bounds and empirical Bayesian sparse posterior sampling.
result IDS nearly matches existing lower bounds and significantly reduces regret.
Introduces tensor bandits for multi-dimensional online decision making.
problem Optimal decision making in multi-dimensional online scenarios.
method Stochastic low-rank tensor bandits, tensor elimination, tensor epoch-greedy, tensor ensemble sampling.
result Tensor elimination and tensor epoch-greedy algorithms outperform existing methods.
Survey and compare PAC-Bayes bounds for bandit problems.
problem Designing and evaluating bandit algorithms with strong performance guarantees.
method PAC-Bayes bounds applied to bandit problems.
result PAC-Bayes bounds useful for offline bandit algorithms, but loose for online algorithms.
The paper tackles budget allocation for multiple campaigns using a novel combinatorial bandit approach.
problem Maximizing cumulative returns with limited budgets across various ad lines.
method Formulated as a multi-task combinatorial bandit problem, integrates Bayesian hierarchical models, and uses Thompson sampling.
result Demonstrates robustness and adaptability in maximizing overall cumulative returns.
Deep Bayesian Bandits improve personalized ads by balancing exploration and exploitation.
problem Recommender systems favor items already engaged by users, causing new campaigns to remain unexplored.
method Formulated as a contextual bandit, implemented exploration techniques using posterior distribution and bootstrapped models.
result Demonstrated a positive gain in exploration model in offline simulation and online AB setting.
Develops Bayesian filtering for online learning and related problems.
problem Sequential machine learning challenges, especially non-stationarity, model misspecification, and high dimensionality.
method Modular adaptive framework, provably robust filter, and sequential parameter updates.
result Improved performance in dynamic, high-dimensional, and misspecified models.
A new framework tackles CASH problem with alternating optimization and Rising Bandits.
problem Efficiently solving the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem.
method Alternating optimization framework using BO for HPO and Rising Bandits for algorithm selection.
result Demonstrated superiority over competitive baselines in extensive experiments.
Bayesian reflex models AI learning like the autonomic nervous system.
problem Online learning in dynamic AI environments.
method Bayesian online algorithms with belief maintenance, sequential updating, and uncertainty-driven action balancing.
result Unified framework for adaptive AI learning.
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. …
Unified framework for Bayesian online learning in changing conditions.
problem Probabilistic online learning in non-stationary environments.
method BONE framework with three modelling choices and two algorithmic choices.
result Framework allows for reinterpreting and proposing new methods.
New algorithms for fast online decision making using neural networks and martingale posteriors.
problem Online sequential decision making under uncertainty.
method Martingale posterior neural networks for fast online learning and decision making.
result Achieves competitive performance-speed trade-offs in non-stationary contextual bandits and Bayesian optimization.
Study online multiclass classification under bandit feedback, extending previous results.
problem Online multiclass classification with bandit feedback, focusing on label space unboundedness.
method Extend Daniely and Helbertal's results, show necessity and sufficiency of Bandit Littlestone dimension for learnability.
result Sequential uniform convergence is necessary but not sufficient for bandit online learnability.
BFTS uses Bayesian Additive Regression Trees for improved personalized mobile health interventions.
problem Adapting to complex, non-linear user behaviors in personalized mobile health interventions.
method Bayesian Forest Thompson Sampling (BFTS) integrates Bayesian Additive Regression Trees (BART) into the exploration loop of contextual bandits.
result BFTS achieves state-of-the-art regret on tabular benchmarks and improves engagement rates by over 30% in a behavioral intervention study.
A new algorithm uses concavity in Gaussian processes to optimize decisions in bandit problems.
problem Optimizing decisions in sequential problems with context-dependent rewards.
method Proposes a UCB algorithm using a shape-constrained reward function estimator based on a Gaussian Process model with concavity constraints.
result Derives regret bounds for the proposed UCB algorithm.
Algorithm balances online and offline data for linear bandits.
problem Online learning with an offline dataset in linear bandits.
method Proposes a linear bandit algorithm that uses offline data early and increasingly favors exploration as the horizon grows.
result Establishes regret bounds showing competitive performance with both purely online and offline solutions.
GLCB uses Gated Linear Networks for online contextual bandits.
problem Online learning in contextual bandits with uncertainty estimation.
method Gated Linear Networks (GLNs) for prediction and uncertainty estimation.
result GLCB outperforms state-of-the-art methods in online contextual bandits.
Greedy algorithm nearly outperforms exploration in contextual bandits.
problem Balancing exploration and exploitation in online learning.
method Smoothed analysis of the greedy algorithm in linear contextual bandits.
result Greedy algorithm nearly matches Bayesian regret rate under diversity conditions, with regret at most O ( T 1 / 3 ) O(T^{1/3}) O ( T 1/3 ) . Paper tackles stochastic k k k -submodular bandits with full feedback, achieving sublinear regret.
problem Online optimization of k k k -submodular functions with full-bandit feedback. method Proposes online algorithms for various k k k -submodular stochastic combinatorial multi-armed bandit problems. result Achieves sublinear α α α -regret bounds for multiple k k k -submodular stochastic combinatorial multi-armed bandit problems. New algorithm reduces regret in sequential decision-making problems.
problem Balancing exploration and exploitation in online sequential decision problems.
method Variational Bayesian optimistic sampling (VBOS) for optimizing policies.
result VBOS achieves i l d e O ( A T ) ilde O(\sqrt{AT}) i l d e O ( A T ) Bayesian regret for stochastic multi-armed bandits. The paper identifies network bottlenecks using minimax paths in stochastic networks.
problem Identifying bottlenecks in networks with stochastic weights.
method Modeling as combinatorial semi-bandit problem, applying combinatorial Thompson Sampling, and approximating the original objective due to computational intractability.
result Established an upper bound on Bayesian regret and evaluated Thompson Sampling performance on real-world networks.
Novel algorithm reduces feature inclusion in online decision-making.
problem Optimizing decision-making for personalized user experiences with fairness.
method Online Batched Sequential Inclusion (OBSI) algorithm for sequential feature inclusion.
result OBSI outperforms other algorithms in terms of regret, relevance of features, and compute.
Efficient boosting method for regression with limited feedback.
problem Online boosting for regression tasks with noisy multi-point bandit feedback.
method Efficient regret minimization method with online boosting algorithm and projection-free online convex optimization.
result Improved state-of-the-art guarantees in efficiency.
Study on Bayesian reinforcement learning performance bounds.
problem Achieving optimal performance in model-based Bayesian reinforcement learning.
method Defining minimum Bayesian regret, deriving upper bounds using relative entropy and Wasserstein distance, and applying these to specific MDP cases.
result Upper bounds on minimum Bayesian regret for MDPs, including specific cases like MAB and online optimization with partial feedback.
Efficiently learns neural network parameters from streaming data.
problem Online learning of neural networks from non-stationary data streams.
method Low-rank extended Kalman filtering for approximate Bayesian inference.
result Significantly faster learning and adaptation to changing distributions.
Multi-armed bandits are a quintessential machine learning problem requiring the balancing of exploration and exploitation. While there has been progress in developing algorithms with strong theoretical guarantees, there has been less focus on practical near-optimal finite-time performance. In this paper, we propose an …
Recent advances in deep reinforcement learning have made significant strides in performance on applications such as Go and Atari games. However, developing practical methods to balance exploration and exploitation in complex domains remains largely unsolved. Thompson Sampling and its extension to reinforcement learning…
Banker-OMD improves online learning with delayed feedback.
problem Handling delayed feedback in online learning.
method Generalized Online Mirror Descent (OMD) framework.
result Achieves nearly-optimal performance in three bandit scenarios.
New method uses offline data to improve online bandit learning, even when distributions differ.
problem Improving online bandit learning with different offline and online distributions.
method MIN-UCB policy that adapts to offline data when informative, achieving tight regret bounds.
result MIN-UCB policy outperforms UCB policy with offline data and provides tight regret bounds.
Novel confidence sets improve linear bandit performance by adapting to unknown noise levels.
problem Adapting to unknown noise levels in sequential decision-making.
method Proposed semi-adaptive and variance-adaptive confidence sets.
result Improved regret bounds and better performance in Bayesian optimization tasks.
We study a variant of the stochastic multi-armed bandit (MAB) problem in which the rewards are corrupted. In this framework, motivated by privacy preservation in online recommender systems, the goal is to maximize the sum of the (unobserved) rewards, based on the observation of transformation of these rewards through a…
PyXAB is a Python library for X-armed bandits and online optimization.
problem Efficiently solving X-armed bandit problems and online blackbox optimization.
method Implementation of 10+ X-armed bandit algorithms and synthetic objectives.
result Evaluation of different algorithms' performance on various synthetic objectives.
This paper uses bandit algorithms to reduce the cost of user interface experimentation in online retail.
problem Reducing the cost of user interface experimentation in online retail.
method Modeling user interface experimentation as an opportunistic bandit problem, reducing the cost of exploration.
result Significant regret reduction and improved contextual information for testing.
Proposes online conformal prediction method with adversarial semi-bandit feedback.
problem Online uncertainty quantification with adversarial semi-bandit feedback.
method Formulates online conformal prediction as an adversarial bandit problem and uses regret minimization.
result Achieves long-run coverage guarantee with adversarial semi-bandit feedback.
Algorithm for online decision making with unknown dynamics and aggregate feedback.
problem Online decision making with unknown dynamics and aggregate bandit feedback.
method Developed an algorithm based on online mirror descent with a self-concordant barrier regularization and an increasing learning rate schedule.
result Achieved O ( K ) O(\sqrt{K}) O ( K ) regret for the online Markov Decision Process with K K K episodes. Algorithm provides online learning guarantees against general comparators in full and bandit feedback.
problem Adversarial online learning with data-dependent regret guarantees.
method Completely online algorithm with data-dependent regret guarantees for full and bandit feedback.
result Algorithm achieves expected performance against arbitrary comparator sequences in full and bandit feedback settings.
Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…
Algorithm improves online learning in adversarial bandits.
problem Online learning in adversarial multi-armed bandits with non-uniform best arm distribution.
method Online-within-online setup, inner and outer learners, leveraging non-uniform empirical distribution of best arms.
result Improves regret bounds for non-uniform best arm distributions.
Bayesian bandit algorithms with approximate inference improve regret bounds in stochastic linear bandits.
problem Theoretical justification for Bayesian bandit algorithms with approximate inference in stochastic linear bandits.
method Proposed a theoretical framework to analyze approximate inference impact and conducted frequentist regret analysis on LinTS and LinBUCB.
result LinTS and LinBUCB preserve their original regret upper bounds with larger constant terms in approximate inference settings.
Unified framework for analyzing online convex optimization across various settings.
problem Analyzing online convex optimization in different settings and feedback types.
method Unified framework allowing systematic proposal and analysis of meta-algorithms.
result Comparable regret bounds for various feedback types and adversary types.
RONM method reduces regret in stochastic convex bandits with decreasing noise.
problem Stochastic convex bandit problem with decreasing noise.
method Regularized Online Newton Method (RONM) based on Online Newton Method (ONM).
result RONM achieves polylogarithmic regret in time horizon n.
Study improves online learning with adaptable agents in various settings.
problem Learning with improving agents in online settings.
method Extensive analysis of combinatorial dimensions, multiclass setup, bandit feedback, and agent cost.
result Characterization and analysis of online learnability in the model.
New MAB model for online caching costs.
problem Learning costs of cached items online.
method Synchronization bandits, MirrorSync algorithm.
result Adversarial regret of O ( T 2 / 3 ) O(T^{2/3}) O ( T 2/3 ) for MirrorSync. Paper solves Bayesian bandit problem with continuous-time limit and approximate policy.
problem Finding optimal policy in Bayesian bandit problems with large horizons.
method Reformulates Bayesian bandit problem as continuous Hamilton-Jacobi-Bellman (HJB) equation and proposes approximate Bayes-optimal policy.
result Approximate Bayes-optimal policy for large horizons with constant computational cost.
This article introduces the concepts around Online Bandit Linear Optimization and explores an efficient setup called SCRiBLe (Self-Concordant Regularization in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe setup and algorithm yield a O ( T ) O(\sqrt{T}) O ( T ) regret bound and polynomial run time comple…
Study on collaborative vs. non-collaborative online and bandit convex optimization.
problem Minimizing average regret in distributed online and bandit convex optimization.
method Analyzes the impact of collaboration in adaptive and zeroth-order feedback settings.
result Collaboration is beneficial in high-dimensional federated online optimization with limited feedback.