arXiv research

No-regret learning fails to converge to Nash equilibria in mixed strategies.

New insights link no-regret learning to online conformal prediction in adversarial settings.

We study online learning settings in which experts act strategically to maximize their influence on the learning algorithm's predictions by potentially misreporting their beliefs about a sequence of binary events. Our goal is twofold. First, we want the learning algorithm to be no-regret with respect to the best fixed …

Kernel-based function approximation improves reinforcement learning performance.

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

No-regret optimization for time-varying functions using uncertainty injection.

New research shows no-regret learning is impossible in Markov games under certain assumptions.

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches provide stronger guarantees in this se…

The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.

New framework uses PDE for no-regret generative modeling.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

R2-B2 optimizes game interactions with recursive reasoning.

Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

Paper proposes no-regret algorithms for private GP bandit optimization.

This paper tackles no-regret learning for fair multi-agent social welfare optimization.

The paper tackles robust policy learning from multiple data sources.

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

This work shows neural networks can solve non-convex constraints problems.

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

A new mechanism reduces expert belief regret in online forecasting.

This paper proposes a new portfolio allocation method using LLMs to outperform traditional strategies.

New model-based methods adapt pre-trained policies to unseen environments efficiently.

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

Unified approach to fair online learning with stochastic contexts.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showin…

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…

New method tackles high-dimensional contextual bandits with flexible kernel models.

In a recent series of papers it has been established that variants of Gradient Descent/Ascent and Mirror Descent exhibit last iterate convergence in convex-concave zero-sum games. Specifically, \cite{DISZ17, LiangS18} show last iterate convergence of the so called "Optimistic Gradient Descent/Ascent" for the case of \t…

Paper proposes OPF policy for fair resource allocation with sublinear regret.

A method for safe online classification reduces test costs while maintaining low error rates.

Efficient algorithms for online learning with changing action sets, achieving no-approximate-regret guarantees.

Algorithm learns expert weights to minimize regret in adversarial setting.

Paper establishes no-regret property for practical EGO optimization.

A new Python-C++ framework for agent-based simulation.

Algorithm learns from changing zero-sum games with no regret.

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

An adversarial bandit problem with memory constraints is studied where only the statistics of a subset of arms can be stored. A hierarchical learning policy that requires only a sublinear order of memory space in terms of the number of arms is developed. Its sublinear regret orders with respect to the time horizon are …

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is r…

New concept of proper-calibeating extends classic calibrated forecasts to proper scoring rules.