arXiv research

Paper analyzes faster convergence rates for reinforcement learning from offline data.

Algorithm minimizes regret and converges to equilibria in Markov games.

We consider regret minimization in repeated games with non-convex loss functions. Minimizing the standard notion of regret is computationally intractable. Thus, we define a natural notion of regret which permits efficient optimization and generalizes offline guarantees for convergence to an approximate local optimum. W…

Paper explores rate-preserving reductions between Blackwell approachability and no-regret learning.

The notion of \emph{policy regret} in online learning is a well defined? performance measure for the common scenario of adaptive adversaries, which more traditional quantities such as external regret do not take into account. We revisit the notion of policy regret and first show that there are online learning settings …

Improved BO algorithms reduce prediction error under Gaussian noise.

No-regret learning fails to converge to Nash equilibria in mixed strategies.

Distributed Thompson sampling improves regret convergence in constrained communication networks.

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allo…

OMWU shows last iterate convergence in convex-concave games.

Paper proposes algorithms to minimize both dynamic and adaptive regret simultaneously.

Paper analyzes proper losses and their performance in machine learning tasks.

Decentralized algorithm reduces regret and converges to Nash equilibrium in online congestion games.

Paper introduces a new $G^\star$ regret measure for online convex optimization with smooth losses.

New learning dynamics achieve fast convergence in games without needing to know utility scales.

This paper addresses missing covariates in stochastic linear bandits, providing a high-probability regret bound.

Narendra-Shapiro (NS) algorithms are bandit-type algorithms that have been introduced in the sixties (with a view to applications in Psychology or learning automata), whose convergence has been intensively studied in the stochastic algorithm literature. In this paper, we adress the following question: are the Narendra-…

RL-LOW algorithm achieves exponential simple regret in offline RLHF with pairwise comparisons.

New framework analyzes regret in guided diffusion for optimizing structured inputs.

New method achieves both universality and adaptivity in online convex optimization.

The paper optimizes regret using covariance between costs and decisions.

This paper analyzes OCBA algorithms' convergence rates for DEDS optimization.

Study shows how to learn optimal policies quickly in stochastic control problems.

New algorithm reduces regret in graphical bilinear bandits.

This paper improves Thompson Sampling for complex decision-making problems.

Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…

New algorithms minimize regret with global costs in online learning.

Optimistic Hedge achieves optimal regret bounds in two-player zero-sum games.

Online and stochastic learning has emerged as powerful tool in large scale optimization. In this work, we generalize the Douglas-Rachford splitting (DRs) method for minimizing composite functions to online and stochastic settings (to our best knowledge this is the first time DRs been generalized to sequential version).…

We describe a novel algorithm for noisy global optimisation and continuum-armed bandits, with good convergence properties over any continuous reward function having finitely many polynomial maxima. Over such functions, our algorithm achieves square-root regret in bandits, and inverse-square-root error in optimisation, …

The CFR framework has been a powerful tool for solving large-scale extensive-form games in practice. However, the theoretical rate at which past CFR-based algorithms converge to the Nash equilibrium is on the order of $O(T^{-1/2})$, where $T$ is the number of iterations. In contrast, first-order methods can be used to …

Bayesian optimization (BO) based on Gaussian process models is a powerful paradigm to optimize black-box functions that are expensive to evaluate. While several BO algorithms provably converge to the global optimum of the unknown function, they assume that the hyperparameters of the kernel are known in advance. This is…

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

Optimizes nonconvex optimization by converting it to static regret minimization.

The dueling bandit is a learning framework wherein the feedback information in the learning process is restricted to a noisy comparison between a pair of actions. In this research, we address a dueling bandit problem based on a cost function over a continuous space. We propose a stochastic mirror descent algorithm and …

The regret bound of an optimization algorithms is one of the basic criteria for evaluating the performance of the given algorithm. By inspecting the differences between the regret bounds of traditional algorithms and adaptive one, we provide a guide for choosing an optimizer with respect to the given data set and the l…

A learning algorithm achieves logarithmic regret in a market making model.

Adam optimization algorithm can have non-zero average regret under certain conditions.

Paper tackles hidden game problem in AI alignment and language games.

LIBO optimizes repeated bandit tasks without prior knowledge or regret.

We study online learnability of a wide class of problems, extending the results of (Rakhlin, Sridharan, Tewari, 2010) to general notions of performance measure well beyond external regret. Our framework simultaneously captures such well-known notions as internal and general Phi-regret, learning with non-additive global…

Improved online Q-learning for MDPs with concentration bounds.

Paper analyzes GP-EI for Bayesian optimization with no regret and provides guidance on choosing incumbents.

New algorithm achieves sublinear regret in CMDPs without error cancellations.

New algorithms for online learning without boundedness or Lipschitz loss assumptions.

New algorithms use neural networks to minimize regret in contextual bandits.

Dynamic pricing policy converges to Nash equilibrium with low regret.

qEUBO optimizes decision-making with noisy feedback.