arXiv research

Boosts weak online learners to strong ones with sublinear regret.

Algorithm minimizes regret in non-stationary dueling bandits with unknown parameters.

Unified framework for best arm identification and dueling bandits regret minimization.

Memory-limited learning tackles adversarial bandits with reduced storage.

New definition resolves ambiguity in non-stationary bandit classification.

Research on the multi-armed bandit problem has studied the trade-off of exploration and exploitation in depth. However, there are numerous applications where the cardinal absolute-valued feedback model (e.g. ratings from one to five) is not suitable. This has motivated the formulation of the duelling bandits problem, w…

Boosting improves online decision-making for large expert sets.

New graph feedback model for bandits with improved regret bounds.

New algorithm tackles adversarial corruption in Lipschitz bandits with sub-linear regret.

Study on tradeoffs between mistakes and ERM oracle calls in online and transductive learning.

New mechanism detects overlap density for weak-to-strong generalization.

This paper describes a new online convex optimization method which incorporates a family of candidate dynamical models and establishes novel tracking regret bounds that scale with the comparator's deviation from the best dynamical model in this family. Previous online optimization methods are designed to have a total a…

New algorithm reduces regret in CMDPs without cancellation of errors.

We study the stochastic multi-armed bandit problem in the case when the arm samples are dependent over time and generated from so-called weak $\cC$-mixing processes. We establish a $\cC-$Mix Improved UCB agorithm and provide both problem-dependent and independent regret analysis in two different scenarios. In the first…

Algorithm minimizes regret in dueling bandits with contextualized utilities.

A new federated algorithm reduces regret in X-armed bandit problems.

Framework uses hindsight regret to audit marketing budget allocations.

GP-PSRL achieves sublinear regret for continuous control with unbounded state space.

Paper develops algorithms for PWA systems with polynomial regret.

Paper solves nonparametric contextual bandit with unbounded contexts.

Study non-rectangular robust MDPs for average-reward, finding optimal policies and transient values.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

Thompson Sampling is a well established approach to bandit and reinforcement learning problems. However its use in continuum armed bandit problems has received relatively little attention. We provide the first bounds on the regret of Thompson Sampling for continuum armed bandits under weak conditions on the function cl…

Paper improves RL algorithms with known optimal Q-value predictions.

Study agnostic feature-based dynamic pricing models with linear policies and noisy valuations.

Study shows how to learn optimal policies quickly in stochastic control problems.

New methods combat data poisoning attacks in bandit algorithms using limited verification.

Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.

Contextual bandits study how reward variance affects regret bounds.

Reinforcement learning (RL) in Markov decision processes (MDPs) with large state spaces is a challenging problem. The performance of standard RL algorithms degrades drastically with the dimensionality of state space. However, in practice, these large MDPs typically incorporate a latent or hidden low-dimensional structu…

Algorithm learns to bid optimally in repeated first-price auctions with censored feedback.

Adaptive model selection for RL with unknown function classes.

We consider an original problem that arises from the issue of security analysis of a power system and that we name optimal discovery with probabilistic expert advice. We address it with an algorithm based on the optimistic paradigm and on the Good-Turing missing mass estimator. We prove two different regret bounds on t…

ARL-GEN adapts to the smallest model class in nested families for RL with improved regret.

The paper considers the problem of global optimization in the setup of stochastic process bandits. We introduce an UCB algorithm which builds a cascade of discretization trees based on generic chaining in order to render possible his operability over a continuous domain. The theoretical framework applies to functions u…

AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.

The paper tackles a bandit problem with infinitely many arms per group, aiming to identify the group with the highest quantile reward.

Algorithm learns optimal arm selection in unsupervised sequential selection with contextual information.

Thompson Sampling tackles USS, a sequential selection problem without feedback.

Regret minimization is treated as the golden rule in the traditional study of online learning. However, regret minimization algorithms tend to converge to the static optimum, thus being suboptimal for changing environments. To address this limitation, new performance measures, including dynamic regret and adaptive regr…

The paper analyzes the sliding regret of stochastic bandit algorithms.

This paper analyzes regret bounds for Gaussian process Thompson sampling.

The notion of \emph{policy regret} in online learning is a well defined? performance measure for the common scenario of adaptive adversaries, which more traditional quantities such as external regret do not take into account. We revisit the notion of policy regret and first show that there are online learning settings …

We discuss general notions of metrics and of Finsler structures which we call weak metrics and weak Finsler structures. Any convex domain carries a canonical weak Finsler structure, which we call its tautological weak Finsler structure. We compute distances in the tautological weak Finsler structure of a domain and we …

In many security and healthcare systems, the detection and diagnosis systems use a sequence of sensors/tests. Each test outputs a prediction of the latent state and carries an inherent cost. However, the correctness of the predictions cannot be evaluated since the ground truth annotations may not be available. Our obje…

Online learning algorithms are designed to learn even when their input is generated by an adversary. The widely-accepted formal definition of an online algorithm's ability to learn is the game-theoretic notion of regret. We argue that the standard definition of regret becomes inadequate if the adversary is allowed to a…

Excessive computational cost for learning large data and streaming data can be alleviated by using stochastic algorithms, such as stochastic gradient descent and its variants. Recent advances improve stochastic algorithms on convergence speed, adaptivity and structural awareness. However, distributional aspects of thes…

Balances the regret of different algorithms in bandit and RL problems.