arXiv research

No-regret learning with strategic experts, incentivized.

No-regret learning fails to converge to Nash equilibria in mixed strategies.

New insights link no-regret learning to online conformal prediction in adversarial settings.

Kernel-based function approximation improves reinforcement learning performance.

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

No-regret optimization for time-varying functions using uncertainty injection.

New research shows no-regret learning is impossible in Markov games under certain assumptions.

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches provide stronger guarantees in this se…

The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.

New framework uses PDE for no-regret generative modeling.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

R2-B2 optimizes game interactions with recursive reasoning.

Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

OMWU shows last iterate convergence in convex-concave games.

Paper proposes no-regret algorithms for private GP bandit optimization.

This paper tackles no-regret learning for fair multi-agent social welfare optimization.

The paper tackles robust policy learning from multiple data sources.

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

This work shows neural networks can solve non-convex constraints problems.

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

A new mechanism reduces expert belief regret in online forecasting.

This paper proposes a new portfolio allocation method using LLMs to outperform traditional strategies.

New model-based methods adapt pre-trained policies to unseen environments efficiently.

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

Unified approach to fair online learning with stochastic contexts.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showin…

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…

New method tackles high-dimensional contextual bandits with flexible kernel models.

Paper proposes OPF policy for fair resource allocation with sublinear regret.

A method for safe online classification reduces test costs while maintaining low error rates.

Efficient algorithms for online learning with changing action sets, achieving no-approximate-regret guarantees.

Algorithm learns expert weights to minimize regret in adversarial setting.

Paper establishes no-regret property for practical EGO optimization.

A new Python-C++ framework for agent-based simulation.

Algorithm learns from changing zero-sum games with no regret.

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

Online Apprenticeship Learning aims to match expert performance without access to cost functions.

Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is r…

New concept of proper-calibeating extends classic calibrated forecasts to proper scoring rules.

Beam search is widely used for approximate decoding in structured prediction problems. Models often use a beam at test time but ignore its existence at train time, and therefore do not explicitly learn how to use the beam. We develop an unifying meta-algorithm for learning beam search policies using imitation learning.…