arXiv research

No-regret optimization for time-varying functions using uncertainty injection.

New framework uses PDE for no-regret generative modeling.

No-regret learning with strategic experts, incentivized.

The paper tackles robust policy learning from multiple data sources.

No-regret learning fails to converge to Nash equilibria in mixed strategies.

This work shows neural networks can solve non-convex constraints problems.

New insights link no-regret learning to online conformal prediction in adversarial settings.

Paper proposes no-regret algorithms for private GP bandit optimization.

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

Kernel-based function approximation improves reinforcement learning performance.

New research shows no-regret learning is impossible in Markov games under certain assumptions.

Mirror descent with an entropic regularizer is known to achieve shifting regret bounds that are logarithmic in the dimension. This is done using either a carefully designed projection or by a weight sharing technique. Via a novel unified analysis, we show that these two approaches deliver essentially equivalent bounds …

Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor performance in theory and often in practice. Some recent approaches provide stronger guarantees in this se…

The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.

We design mechanisms for online procurement of data held by strategic agents for machine learning tasks. The challenge is to use past data to actively price future data and give learning guarantees even when an agent's cost for revealing her data may depend arbitrarily on the data itself. We achieve this goal by showin…

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

New insights on active sequential prediction for mean estimation.

New algorithms achieve no-regret learning even with adversarial transitions and losses.

Paper proposes OPF policy for fair resource allocation with sublinear regret.

DORIS algorithm achieves no-regret learning in Markov games with adversarial opponents.

Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…

A method for safe online classification reduces test costs while maintaining low error rates.

We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum games can be solved using online learning dynamics, where a classical technique involves simulating two no-regret algorithms that play agai…

R2-B2 optimizes game interactions with recursive reasoning.

OMWU shows last iterate convergence in convex-concave games.

Paper establishes no-regret property for practical EGO optimization.

A new mechanism reduces expert belief regret in online forecasting.

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

This paper tackles no-regret learning for fair multi-agent social welfare optimization.

Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learni…

This paper proposes a new portfolio allocation method using LLMs to outperform traditional strategies.

New concept of proper-calibeating extends classic calibrated forecasts to proper scoring rules.

Many prediction domains, such as ad placement, recommendation, trajectory prediction, and document summarization, require predicting a set or list of options. Such lists are often evaluated using submodular reward functions that measure both quality and diversity. We propose a simple, efficient, and provably near-optim…

We consider a family of learning strategies for online optimization problems that evolve in continuous time and we show that they lead to no regret. From a more traditional, discrete-time viewpoint, this continuous-time approach allows us to derive the no-regret properties of a large class of discrete-time algorithms i…

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of eac…

Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is…

Optimistic Thompson Sampling reduces regret in unknown multi-player games.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…

New model-based methods adapt pre-trained policies to unseen environments efficiently.

New method tackles high-dimensional contextual bandits with flexible kernel models.

The paper analyzes the sliding regret of stochastic bandit algorithms.

Unified approach to fair online learning with stochastic contexts.

ECP optimizes expensive functions without knowing Lipschitz constant.

TVBO optimizes time-varying functions with asymptotically vanishing regret.