arXiv research

Paper proposes no-regret algorithms for private GP bandit optimization.

No-regret optimization for time-varying functions using uncertainty injection.

We study online learning settings in which experts act strategically to maximize their influence on the learning algorithm's predictions by potentially misreporting their beliefs about a sequence of binary events. Our goal is twofold. First, we want the learning algorithm to be no-regret with respect to the best fixed …

New insights link no-regret learning to online conformal prediction in adversarial settings.

No-regret learning fails to converge to Nash equilibria in mixed strategies.

Kernel-based function approximation improves reinforcement learning performance.

New research shows no-regret learning is impossible in Markov games under certain assumptions.

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches provide stronger guarantees in this se…

New algorithms achieve no-regret learning even with adversarial transitions and losses.

New framework uses PDE for no-regret generative modeling.

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

Paper proposes OPF policy for fair resource allocation with sublinear regret.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

R2-B2 optimizes game interactions with recursive reasoning.

Paper establishes no-regret property for practical EGO optimization.

Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is…

This paper tackles no-regret learning for fair multi-agent social welfare optimization.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…

Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…

The paper tackles robust policy learning from multiple data sources.

The paper analyzes the sliding regret of stochastic bandit algorithms.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

Efficient algorithms for online learning with changing action sets, achieving no-approximate-regret guarantees.

Online algorithms stabilize in feedback loops of performative prediction.

ECP optimizes expensive functions without knowing Lipschitz constant.

New algorithm for traffic routing in congested conditions.

Algorithm learns from changing zero-sum games with no regret.

We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum games can be solved using online learning dynamics, where a classical technique involves simulating two no-regret algorithms that play agai…

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showin…

This work shows neural networks can solve non-convex constraints problems.

The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.

Paper presents an efficient algorithm for linear MDP with low switching cost.

Algorithm improves wildlife protection patrols.

We study the problem of online clustering where a clustering algorithm has to assign a new point that arrives to one of $k$ clusters. The specific formulation we use is the $k$-means objective: At each time step the algorithm has to maintain a set of k candidate centers and the loss incurred is the squared distance bet…

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is r…

In a recent series of papers it has been established that variants of Gradient Descent/Ascent and Mirror Descent exhibit last iterate convergence in convex-concave zero-sum games. Specifically, \cite{DISZ17, LiangS18} show last iterate convergence of the so called "Optimistic Gradient Descent/Ascent" for the case of \t…

Algorithm learns expert weights to minimize regret in adversarial setting.

Beam search is widely used for approximate decoding in structured prediction problems. Models often use a beam at test time but ignore its existence at train time, and therefore do not explicitly learn how to use the beam. We develop an unifying meta-algorithm for learning beam search policies using imitation learning.…

Paper tackles multiplayer symmetric games, securing equal share for n players.