arXiv research

No-regret optimization for time-varying functions using uncertainty injection.

No-regret learning with strategic experts, incentivized.

No-regret learning fails to converge to Nash equilibria in mixed strategies.

New insights link no-regret learning to online conformal prediction in adversarial settings.

Paper proposes no-regret algorithms for private GP bandit optimization.

New framework uses PDE for no-regret generative modeling.

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

Kernel-based function approximation improves reinforcement learning performance.

New research shows no-regret learning is impossible in Markov games under certain assumptions.

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches provide stronger guarantees in this se…

The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

Paper proposes OPF policy for fair resource allocation with sublinear regret.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

A method for safe online classification reduces test costs while maintaining low error rates.

R2-B2 optimizes game interactions with recursive reasoning.

OMWU shows last iterate convergence in convex-concave games.

Paper establishes no-regret property for practical EGO optimization.

A new mechanism reduces expert belief regret in online forecasting.

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

This paper tackles no-regret learning for fair multi-agent social welfare optimization.

Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…

This paper proposes a new portfolio allocation method using LLMs to outperform traditional strategies.

New concept of proper-calibeating extends classic calibrated forecasts to proper scoring rules.

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

The paper tackles robust policy learning from multiple data sources.

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is…

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

This work shows neural networks can solve non-convex constraints problems.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…

New model-based methods adapt pre-trained policies to unseen environments efficiently.

New method tackles high-dimensional contextual bandits with flexible kernel models.

The paper analyzes the sliding regret of stochastic bandit algorithms.

Unified approach to fair online learning with stochastic contexts.

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showin…

ECP optimizes expensive functions without knowing Lipschitz constant.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

ECPv2 optimizes Lipschitz functions efficiently and scalably.

Extreme multi-label classification (XMLC) is a problem of tagging an instance with a small subset of relevant labels chosen from an extremely large pool of possible labels. Large label spaces can be efficiently handled by organizing labels as a tree, like in the hierarchical softmax (HSM) approach commonly used for mul…

A new Python-C++ framework for agent-based simulation.

Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…

Online algorithms stabilize in feedback loops of performative prediction.