arXiv research

New method uses correlated auxiliary feedback to reduce regret in parameterized bandits.

FP-UCB algorithm achieves bounded regret for finitely parameterized multi-armed bandits.

Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the reward distributions: the classic $K$-armed bandit and the linearly parameterized bandit…

We introduce Parameterized Exploration (PE), a simple family of methods for model-based tuning of the exploration schedule in sequential decision problems. Unlike common heuristics for exploration, our method accounts for the time horizon of the decision problem as well as the agent's current state of knowledge of the …

New algorithms improve performance guarantees for multi-armed bandits problems.

We present an efficient second-order algorithm with $\tilde{O}(\frac{1}η\sqrt{T})$ regret for the bandit online multiclass problem. The regret bound holds simultaneously with respect to a family of loss functions parameterized by $η$, for a range of $η$ restricted by the norm of the competitor. The family of loss funct…

New meta-learning approach for bandit policies that achieve high average reward.

New algorithm reduces semi-bandit regret using covariance estimates.

Increasing technological sophistication and widespread use of smartphones and wearable devices provide opportunities for innovative and highly personalized health interventions. A Just-In-Time Adaptive Intervention (JITAI) uses real-time data collection and communication capabilities of modern mobile devices to deliver…

Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.

A new adversarial attack method using structured search and contextual bandits.

This work learns exploration policies for unknown distributions using samples and policy gradients.

AGG-UCB uses neural networks to optimize group behaviors in contextual bandits.

Improved Thompson Sampling algorithms for bandits with tighter regret bounds.

We study the linear contextual bandit problem with finite action sets. When the problem dimension is $d$, the time horizon is $T$, and there are $n \leq 2^{d/2}$ candidate actions per time period, we (1) show that the minimax expected regret is $Ω(\sqrt{dT (\log T) (\log n)})$ for every algorithm, and (2) introduce a V…

Cost-efficient distributed learning via combinatorial bandits.

Optimizes sensor usage for detecting abrupt changes in sensor data.

New algorithm identifies best arm efficiently in stochastic bandits.

New algorithm reduces regret in sequential decision-making problems.

The stochastic linear bandit problem proceeds in rounds where at each round the algorithm selects a vector from a decision set after which it receives a noisy linear loss parameterized by an unknown vector. The goal in such a problem is to minimize the (pseudo) regret which is the difference between the total expected …

This study explores how choosing noninformative priors affects Thompson Sampling in multiparameter bandit models.

A new thompson sampling method controls for time-varying effects.

Improves bandits with knapsacks guarantees for partially stochastic workloads.

The goal of data-driven algorithm design is to obtain high-performing algorithms for specific application domains using machine learning and data. Across many fields in AI, science, and engineering, practitioners will often fix a family of parameterized algorithms and then optimize those parameters to obtain good perfo…

Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks. This limitation motivates a need for scalable parameterizations of the noise generation process, in a manner t…

The exploration/exploitation (E/E) dilemma arises naturally in many subfields of Science. Multi-armed bandit problems formalize this dilemma in its canonical form. Most current research in this field focuses on generic solutions that can be applied to a wide range of problems. However, in practice, it is often the case…

Exploration in reinforcement learning (RL) suffers from the curse of dimensionality when the state-action space is large. A common practice is to parameterize the high-dimensional value and policy functions using given features. However existing methods either have no theoretical guarantee or suffer a regret that is ex…

Optimality of TS with noninformative priors proven for Pareto model.

We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters contr…

The paper introduces various canonical parameterizations for 2D-curved shapes.

There are currently two parameterizations used to derive fixed kernels corresponding to infinite width neural networks, the NTK (Neural Tangent Kernel) parameterization and the naive standard parameterization. However, the extrapolation of both of these parameterizations to infinite width is problematic. The standard p…

The paper proposes a method to construct confidence sets using likelihood ratios for sequential decision-making.

A learning algorithm optimizes beamforming for holographic transceivers in far-field communication.

New algorithms tackle RKHS bandits with reduced complexity and improved performance.

New parameterization for $(1,1)$-knots simplifies their study.

Unified formulation bridges adversarial and nonstationary bandits.

Paper tackles LDP bandits learning with improved results and sub-linear regret.

Paper solves stochastic contextual linear bandits using linear bandit algorithms.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

New definition resolves ambiguity in non-stationary bandit classification.

Unified approach for non-stationary and clustered bandits.

A framework for auto-tuning hyper-parameters in contextual bandit algorithms.

Stochastic parameterizations account for uncertainty in the representation of unresolved sub-grid processes by sampling from the distribution of possible sub-grid forcings. Some existing stochastic parameterizations utilize data-driven approaches to characterize uncertainty, but these approaches require significant str…

The current paper discusses some new results about conformal polynomic surface parameterizations. A new theorem is proved: Given a conformal polynomic surface parameterization of any degree it must be harmonic on each component. As a first geometrical application, every surface that admits a conformal polynomic paramet…

Stochastic multi-armed bandits form a class of online learning problems that have important applications in online recommendation systems, adaptive medical treatment, and many others. Even though potential attacks against these learning algorithms may hijack their behavior, causing catastrophic loss in real-world appli…

New algorithm learns optimal exploration parameters for contextual bandits.

Investigates sequential problems on graph structures and large action spaces.

Study on indexability of restless multi-armed bandits and rollout policy performance.