Algorithm finds near-optimal strategy in changing zero-sum games.
problem Finding near-optimal strategy in changing zero-sum games.
method Designing an algorithm with small NE regret for online matrix games.
result Achieves near-optimal dependence on the number of rounds and number of actions.
Algorithm learns from changing zero-sum games with no regret.
problem Learning in time-varying zero-sum games.
method Developed a single parameter-free algorithm with three performance measures.
result Algorithm recovers best known results for fixed games and adapts to non-stationarity.
New bounds show simple predictors can learn complex concepts online.
problem When can simple predictors learn complex concepts in online learning?
method Characterized optimal mistake bounds for online learning with simple predictors.
result Achieved nearly optimal mistake bounds for online learning using sparse majority-vote of proper predictors.
Study online learning in unknown Markov games with sublinear regret.
problem Online learning in unknown Markov games with unobservable opponents.
method Introduced an algorithm achieving sublinear regret against the minimax value.
result First sublinear regret bound for unknown Markov games, independent of action spaces size.
Algorithm solves online binary classification and infinite games using ERM oracle.
problem Online learning and solving infinite games with computationally inefficient oracles.
method Proposes an algorithm relying solely on ERM oracle calls for online binary classification and nonparametric games.
result Achieves finite and sublinearly growing regret in various settings.
We introduce CSE for MLSF games and devise online learning algorithms for achieving no-external Stackelberg-regret.
problem Learning equilibrium in leader-follower games with noisy bandit feedback.
method Proposed Correlated Stackelberg Equilibrium (CSE) and online learning algorithms balancing exploration and exploitation.
result Achieves no-external Stackelberg-regret, converging to approximate CSE.
Efficient reinforcement learning for simultaneous-move zero-sum games using optimistic value iteration.
problem Learning optimal strategies in simultaneous-move zero-sum Markov games with function approximation.
method Developed an optimistic variant of least-squares minimax value iteration algorithm for offline and online settings.
result Achieved an upper bound of i l d e O ( d 3 H 3 T ) ilde O(\sqrt{d^3 H^3 T}) i l d e O ( d 3 H 3 T ) on duality gap and regret. New algorithms achieve near-optimal cumulative loss in nonparametric online learning and games.
problem Fast rates of convergence in nonparametric online regression and classification.
method Randomized proper learning algorithms, hierarchical aggregation, multi-scale extension, stability proof.
result Achieved near-optimal cumulative loss bounds for real-valued and binary games.
New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.
problem Improving performance in matrix games with unknown payoff matrices and bandit feedback.
method Regret analyses of variants of UCB and K-learning.
result New algorithms achieve lower regret compared to adversarial bandit algorithms.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
New framework connects online learning to statistical learning for better generalization bounds.
problem Deriving generalization bounds for statistical learning algorithms.
method Constructing an online learning game and showing a connection to statistical learning.
result Established a connection between online and statistical learning, leading to new generalization bounds.
Decentralized algorithm reduces regret and converges to Nash equilibrium in online congestion games.
problem Online congestion games with exponential action sets and strict Nash equilibria.
method CongestEXP algorithm using exponential weights method.
result CongestEXP achieves O ( k F T ) O(kF\sqrt{T}) O ( k F T ) regret bound and almost exponential convergence to strict Nash equilibrium. Paper analyzes convergence rates for multi-agent learning in games.
problem Convergence rates for multi-agent learning in games.
method Characterizes finite-time convergence rates for joint OGD learning on λ λ λ -cocoercive games and develops adaptive algorithms. result Adaptive algorithms achieve same convergence rates as non-adaptive counterparts.
Survey of algorithms to correct past mistakes in prediction.
problem Improving prediction accuracy by correcting past errors.
method Defensive Forecasting as a sequential game theory approach to minimize prediction metrics.
result Simple, near-optimal algorithms for various prediction tasks.
We describe an approximate dynamic programming (ADP) approach to compute approximations of the optimal strategies and of the minimal losses that can be guaranteed in discounted repeated games with vector-valued losses. Such games prominently arise in the analysis of regret in repeated decision-making in adversarial env…
Optimal online learning algorithms for label-efficient prediction and bandits.
problem Efficient prediction in online learning with limited information.
method Optimistic online mirror descent with second order corrections and hybrid regularizers.
result Improved regret bounds for label-efficient prediction and bandits.
Algorithm learns NE in imperfect information games with imperfect feedback.
problem Learning Nash equilibrium in imperfect information games with bandit feedback.
method IXOMD algorithm for model-free learning with 1 / T 1/\sqrt{T} 1/ T convergence rate. result IXOMD achieves 1 / T 1/\sqrt{T} 1/ T convergence rate to NE. New algorithm reduces risk in online games with limited feedback.
problem Risk-averse learning in repeated unknown games with bandit feedback.
method Proposes a momentum-based algorithm to estimate CVaR using historical cost values.
result Achieves sub-linear regret and outperforms existing methods in numerical experiments.
Neural MMO simulates MMOs to study multiagent intelligence.
problem Limited research environments for multiagent intelligence.
method Developed a new game environment inspired by MMOs.
result Standard methods can learn interesting behaviors in MMOs.
New algorithm learns from noisy and correlated game outcomes.
problem Learning to play a repeated multi-agent game with unknown reward function.
method GP-MW algorithm using Gaussian processes and multiplicative weight method.
result Novel kernel-dependent regret bounds comparable to full information settings.
The in-game economies of massively multi-player online games (MMOGs) are complex systems that have to be carefully designed and managed. This paper presents the results of an analysis of auction house data from the MMOG Glitch, across a 14 month time period, the entire lifetime of the game. The data comprise almost 3 m…
New algorithm reduces online learning regret for bounded recall games.
problem Reducing regret in online learning with limited past information.
method Constructing a stationary bounded-recall algorithm with O ( 1 / M ) O(1/\sqrt{M}) O ( 1/ M ) regret. result Any low regret bounded-recall algorithm must be aware of past losses' order.
GAME improves matrix completion by considering subgroup-specific latent structures.
problem Heterogeneous data with overlapping categories, smoothing away subgroup-specific variation.
method Group-Aware Matrix Estimation (GAME) with overlapping nuclear-norm penalties.
result GAME outperforms global low-rank estimators in structured missingness regimes.
This work establishes always-valid risk bounds for online matrix completion.
problem Challenges in establishing always-valid concentration inequalities for online matrix completion.
method Combines non-asymptotic martingale concentration and regularized low-rank matrix regression.
result Establishes always-valid risk bound process for online matrix completion.
Algorithm learns to play against unknown opponents in sequential games.
problem Designing strategies for a learner to interact with an unknown opponent in repeated sequential games.
method Kernel-based regularity assumptions and a novel algorithm combining bilevel optimization and online learning.
result Algorithm achieves sublinear regret guarantees and is effective in specific game settings.
Online learning algorithms have impressive convergence properties when it comes to risk minimization and convex games on very large problems. However, they are inherently sequential in their design which prevents them from taking advantage of modern multi-core architectures. In this paper we prove that online learning …
New method accelerates smooth games using spectral shape analysis.
problem Accelerating optimization in smooth games with complex numerical challenges.
method Matrix iteration theory and spectral shape analysis to characterize and manipulate acceleration.
result Identified a continuum of optimization strategies from convex minimization to gradient descent.
We consider online learning in an adversarial, non-convex setting under the assumption that the learner has an access to an offline optimization oracle. In the general setting of prediction with expert advice, Hazan et al. (2016) established that in the optimization-oracle model, online learning requires exponentially …
Framework for multi-agent RL with human feedback in a Snake game.
problem Improving multi-agent reinforcement learning with human feedback.
method Developed a simulated game environment for offline model training and online competitions. Introduced HILL methods and reward manipulation heuristics.
result Agents with HILL methods outperform those without in online competitions.
Two new RL methods enable deep learning of MFG equilibria.
problem Efficiently learning equilibria in Mean Field Games using RL.
method Two novel RL methods: distillation and online mixing.
result Deep RL algorithms can now solve various MFGs.
Educational game on crypto investment helps students grasp macroeconomics.
problem Weak connections between microeconomic decision-making and macroeconomic concepts in classroom games.
method Design and study of an educational game on cryptocurrency investment.
result Engages students in understanding macroeconomics through incentivized individual investment decisions.
A simplified Bayesian approach for online sports rating.
problem Rating players/teams based on game outcomes.
method Bayesian approach using Kalman filter principles.
result Known algorithms like Elo and TrueSkill are special cases.
New RL algorithms find SNE in Markov games with myopic followers.
problem Finding SNE in Markov games with myopic followers.
method Optimistic and pessimistic variants of least-squares value iteration, incorporating function approximation.
result First provably efficient RL algorithms for SNEs in general-sum Markov games with myopic followers.
The Interactive Minority Game (IMG) is an online version of the traditional Minority Game in which human players can enter into competition with the traditional computer-controlled agents. Through the rich (and, importantly, analytically understood) behaviour of the MG, we can explore humans' behaviour in different kin…
We consider the problem of strongly-convex online optimization in presence of adversarial delays; in a T-iteration online game, the feedback of the player's query at time t is arbitrarily delayed by an adversary for d_t rounds and delivered before the game ends, at iteration t+d_t-1. Specifically for \algo{online-gradi…
Algorithm finds Nash equilibria in complex games with function approximation.
problem Learning Nash equilibria in two-player zero-sum Markov Games with nonlinear function approximation.
method Online learning algorithm using upper and lower confidence bounds derived from optimism in the face of uncertainty.
result Achieves O ( T ) O(\sqrt{T}) O ( T ) regret with polynomial complexity, under mild assumptions. Paper proposes an online estimator for covariance matrix of SGD iterates.
problem Quantifying variability and randomness of SGD-based estimates in online learning.
method Proposes a fully online estimator for covariance matrix of ASGD using SGD iterates.
result Establishes consistency of the online estimator and shows comparable convergence rate to offline methods.
Paper addresses inefficiency in converting EFGs to NFGs for learning.
problem Inefficiency in converting Extensive-Form Games to Normal-Form Games.
method Uses Φ Φ Φ -Hedge algorithm and Online Mirror Descent (OMD) for polynomial-time learning of EFGs. result Achieves O ~ ( X A T ) \widetilde{\mathcal{O}}(\sqrt{XAT}) O ( X A T ) EFCE-regret, matching information-theoretic lower bound. Paper analyzes algorithms for nonstationary saddle-point optimization problems.
problem Nonstationary saddle-point optimization problems in game theory, reinforcement learning, and machine learning.
method Proposes extragradient and Frank-Wolfe algorithms for online and bandit settings.
result Establishes sub-linear regret bounds for the proposed algorithms.
Machine learning detects regime shifts in online game-experiments with high accuracy.
problem Detecting regime shifts in online social systems.
method Gradient-boosted decision trees with memory-retaining features.
result Significantly outperforms standard early warning indicators.
Graphon game model simplifies stochastic interactions among agents.
problem Complex interactions among heterogeneous agents in stochastic games.
method Introduced a discrete-time graphon game formulation with a representative player.
result Existence and uniqueness of graphon equilibrium proven with mild assumptions.
We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu \cite{NeelAaronRoth2018} asks whether this implication is {\it efficient}…
Improved FTPL algorithm reduces regret in predictable minimax games.
problem Online learning and minimax games with predictable loss sequences.
method Optimistic modification of FTPL with dual regularization view.
result Tighter regret bounds for predictable sequences, O ( T − 1 / 2 ) O(T^{-1/2}) O ( T − 1/2 ) accuracy. New model for high rank matrix completion with online and batch methods.
problem Matrix completion for high rank matrices with latent structure.
method Kernel trick to map data into a high dimensional feature space, explicit parametrization of low dimensional subspace, online fitting procedure.
result Online method can handle streaming data and adapt to non-stationary latent structure.
Algorithm learns robust equilibrium in online Markov games with interactive data.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust RL with minimum value assumption, least square value iteration.
result Sample-efficient algorithm for robust equilibrium in online Markov games.
New algorithms achieve logarithmic regret in KL-regularized Markov games.
problem Improving sample efficiency in game-theoretic settings with KL regularization.
method Developed OMG and SOMG algorithms for matrix and Markov games, using best response sampling and superoptimistic bonuses.
result Logarithmic regret in T T T that scales inversely with KL regularization strength β β β . Adaptive OMD reduces variance in learning optimal strategies for imperfect information games.
problem High variance in learning optimal strategies for imperfect information games.
method Fixed sampling approach with locally applied Online Mirror Descent (OMD) algorithm.
result Convergence rate of i l d e O ( T − 1 / 2 ) ilde{\mathcal{O}}(T^{-1/2}) i l d e O ( T − 1/2 ) with high probability. This thesis presents some geometric insights into three different types of two player prediction games -- namely general learning task, prediction with expert advice, and online convex optimization. These games differ in the nature of the opponent (stochastic, adversarial, or intermediate), the order of the players' mo…