arXiv research

New Q-learning algorithms reduce regret in inventory control problems.

We use online convex optimization (OCO) for setpoint tracking with uncertain, flexible loads. We consider full feedback from the loads, bandit feedback, and two intermediate types of feedback: partial bandit where a subset of the loads are individually observed and the rest are observed in aggregate, and Bernoulli feed…

One-bit feedback suffices for a bandit problem's optimal strategy.

Paper tackles stochastic $k$-submodular bandits with full feedback, achieving sublinear regret.

Algorithm provides online learning guarantees against general comparators in full and bandit feedback.

Unified framework for expert selection with bandit and lower-bound feedback.

We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides. We propose a novel surrogate loss function and unbiased estimator, allowing weak learners to update themselves with limited information. Using…

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

New algorithms handle online prediction with bandit and delayed feedback, improving regret bounds.

Study online multiclass classification under bandit feedback, extending previous results.

We present online boosting algorithms for multiclass classification with bandit feedback, where the learner only receives feedback about the correctness of its prediction. We propose an unbiased estimate of the loss using a randomized prediction, allowing the model to update its weak learners with limited information. …

We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise r…

Study the tradeoffs of bandit feedback in multiclass classification.

New framework tackles submodular welfare with multi-agent combinatorial bandits.

Unified framework for analyzing online convex optimization across various settings.

We consider the problem of learning to play a repeated multi-agent game with an unknown reward function. Single player online learning algorithms attain strong regret bounds when provided with full information feedback, which unfortunately is unavailable in many real-world scenarios. Bandit feedback alone, i.e., observ…

This work focuses on dynamic regret of online convex optimization that compares the performance of online learning to a clairvoyant who knows the sequence of loss functions in advance and hence selects the minimizer of the loss function at each step. By assuming that the clairvoyant moves slowly (i.e., the minimizers c…

We investigate contextual online learning with nonparametric (Lipschitz) comparison classes under different assumptions on losses and feedback information. For full information feedback and Lipschitz losses, we design the first explicit algorithm achieving the minimax regret rate (up to log factors). In a partial feedb…

New techniques improve distributed training with compressed gradients.

Simple algorithms identify best items or full rankings from choice-based feedback.

Study on sample complexity for pure exploration in feedback graph settings.

The paper addresses learner privacy in convex optimization with feedback.

Paper characterizes minimax regret rates for online ranking with top-k feedback.

Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of $k$ out of $n$ arms may be chosen and the sum of the rewards is gained. We address the full-bandit feedback, in which the agent observes only the sum of rewards, in contrast to the semi-bandit feedback, in which the agent obse…

Mutation improves FTRL convergence in zero-sum games.

We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute. Specifically, we study two regret minimisation problems over subsets of a finite ground set $[n]$, with subset-wise relative prefe…

First sample-efficient algorithm for learning EFCE in bandit feedback settings.

New algorithm controls linear systems with bandit feedback, achieving optimal regret.

New algorithms for efficient learning with partial information, reducing regret.

We derive upper and lower bounds for the policy regret of $T$-round online learning problems with graph-structured feedback, where the adversary is nonoblivious but assumed to have a bounded memory. We obtain upper bounds of $\widetilde O(T^{2/3})$ and $\widetilde O(T^{3/4})$ for strongly-observable and weakly-observab…

Study optimal arms in combinatorial bandits with semi-bandit feedback and finite budget.

We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player's performance using a new notion of regret, also known as policy regret, which better captures the adversary's adaptiveness…

Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.

This paper tackles combinatorial optimization under uncertainty with limited feedback.

Optimal investment strategy with expert opinions in uncertain conditions.

Graph-based approach repairs programs from diagnostic feedback.

Algorithm POLO learns low-rank MDPs with adversarial changes in full-info feedback.

Study online learning with delays and capacity constraints, achieving optimal regret bounds.

We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial tra…

This paper extends combinatorial semi-bandits to graph feedback, improving regret bounds.

This paper performs the numerical analysis and the computation of a Spread option in a market with imperfect liquidity. The number of shares traded in the stock market has a direct impact on the stock's price. Thus, we consider a full-feedback model in which price impact is fully incorporated into the model. The price …

Characterizes learnability of multioutput functions in various settings.

We consider the problem of learning in episodic finite-horizon Markov decision processes with an unknown transition function, bandit feedback, and adversarial losses. We propose an efficient algorithm that achieves $\mathcal{\tilde{O}}(L|X|\sqrt{|A|T})$ regret with high probability, where $L$ is the horizon, $|X|$ is t…

Maximizing trading volume in online learning framework between traders.

Algorithm allocates budgets to tasks with semi-bandit feedback, achieving near-optimal regret bounds.

We study the problem of stochastic combinatorial pure exploration (CPE), where an agent sequentially pulls a set of single arms (a.k.a. a super arm) and tries to find the best super arm. Among a variety of problem settings of the CPE, we focus on the full-bandit setting, where we cannot observe the reward of each singl…

New algorithm shows neural networks can learn without full backpropagation.

New algorithm reduces risk in online games with limited feedback.