New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.
problem Improving performance in matrix games with unknown payoff matrices and bandit feedback.
method Regret analyses of variants of UCB and K-learning.
result New algorithms achieve lower regret compared to adversarial bandit algorithms.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
Playing repeated matrix games (RMG) while maximizing the cumulative returns is a basic method to evaluate multi-agent learning (MAL) algorithms. Previous work has shown that U C B UCB U C B , M 3 M3 M 3 , S S S or E x p 3 Exp3 E x p 3 algorithms have good behaviours on average in RMG. Besides, hedging algorithms have been shown to be effective on predi…
New insights into query complexity for Nash equilibrium learning.
problem Characterizing the number of queries needed to learn approximate Nash equilibria in matrix games.
method Introduced a new technique to prove lower bounds on query complexity, improving previous techniques.
result Lower bounds of order Ω ( log ( 1 K ε ) ) Ω(\log(\frac{1}{Kε})) Ω ( log ( K ε 1 )) for any ε ≤ 1 / ( c K 4 ) ε\leq 1 / (cK^4) ε ≤ 1/ ( c K 4 ) , where c c c is a constant. Improved CFR achieves faster convergence to Nash equilibrium.
problem Theoretical slow convergence rate of CFR algorithms.
method Combining predictive and stable regret minimizers within CFR.
result Achieves O ( T − 3 / 4 ) O(T^{-3/4}) O ( T − 3/4 ) convergence rate, faster than O ( T − 1 / 2 ) O(T^{-1/2}) O ( T − 1/2 ) . New algorithms achieve logarithmic regret in KL-regularized Markov games.
problem Improving sample efficiency in game-theoretic settings with KL regularization.
method Developed OMG and SOMG algorithms for matrix and Markov games, using best response sampling and superoptimistic bonuses.
result Logarithmic regret in T T T that scales inversely with KL regularization strength β β β . New algorithm improves self-play reinforcement learning for competitive games.
problem Inefficient opponent selection in self-play reinforcement learning.
method Intelligently selects opponents based on adversarial rules derived from saddle point optimization.
result Algorithm converges to approximate equilibrium with high probability in convex-concave games.
Algorithm finds near-optimal strategy in changing zero-sum games.
problem Finding near-optimal strategy in changing zero-sum games.
method Designing an algorithm with small NE regret for online matrix games.
result Achieves near-optimal dependence on the number of rounds and number of actions.
New algorithms reduce variance in solving complex mathematical problems.
problem Solving convex-concave saddle point problems, variational inequalities, and inclusions.
method Stochastic variance reduction for extragradient, forward-backward-forward, and forward-reflected-backward methods.
result All proposed methods converge with complexities matching or improving deterministic counterparts.
New algorithms converge faster to Nash equilibrium in zero-sum games with bandit feedback.
problem Learning in zero-sum games with bandit feedback without communication.
method Developed two uncoupled algorithms achieving optimal rate of Ω ( T − 1 / 4 ) Ω(T^{-1/4}) Ω ( T − 1/4 ) . result Achieved optimal rate of Ω ( T − 1 / 4 ) Ω(T^{-1/4}) Ω ( T − 1/4 ) for convergence of policy profiles to Nash equilibrium. Unified framework for estimating reward functions in competitive games.
problem Estimating unknown reward functions in competitive games.
method Unified framework with entropy regularization for reward function recovery.
result Strong theoretical guarantees and practical effectiveness demonstrated.
New algorithm reduces regret in sequential decision-making problems.
problem Balancing exploration and exploitation in online sequential decision problems.
method Variational Bayesian optimistic sampling (VBOS) for optimizing policies.
result VBOS achieves i l d e O ( A T ) ilde O(\sqrt{AT}) i l d e O ( A T ) Bayesian regret for stochastic multi-armed bandits. Humans are capable of attributing latent mental contents such as beliefs or intentions to others. The social skill is critical in daily life for reasoning about the potential consequences of others' behaviors so as to plan ahead. It is known that humans use such reasoning ability recursively by considering what others …
Increasing iterate averaging improves convergence rates for saddle-point problems.
problem Solving saddle-point problems efficiently.
method Increasing iterate averaging schemes applied to various first-order methods.
result Increasing iterate averaging preserves the O ( 1 / T ) O(1/T) O ( 1/ T ) convergence rate with no additional assumptions or overhead. New metrics needed for better understanding emergent communication in multi-agent systems.
problem Current metrics for emergent communication are insufficient for complex environments.
method Training deep reinforcement learning agents in simple games to analyze communication effectiveness.
result Messages in emergent communication can be misleading and do not always impact the environment.
New algorithm learns from noisy and correlated game outcomes.
problem Learning to play a repeated multi-agent game with unknown reward function.
method GP-MW algorithm using Gaussian processes and multiplicative weight method.
result Novel kernel-dependent regret bounds comparable to full information settings.
Develops accelerated fixed-point methods with delayed oracles for scientific computing.
problem Approximating fixed points of nonexpansive operators.
method Combines Nesterov's acceleration and KM iteration with delayed inexact oracles.
result Establishes improved convergence rates for fixed-point approximation.
Research tackles alliance formation in many-player zero-sum games, showing reinforcement learning fails but a contract mechanism can help.
problem Tackles the challenge of alliance formation in many-player zero-sum games.
method Demonstrates the social dilemma aspect of alliance formation, introduces a contract mechanism to augment reinforcement learning.
result Naïve reinforcement learning fails to form alliances, but a contract mechanism can help.
Efficient reinforcement learning for simultaneous-move zero-sum games using optimistic value iteration.
problem Learning optimal strategies in simultaneous-move zero-sum Markov games with function approximation.
method Developed an optimistic variant of least-squares minimax value iteration algorithm for offline and online settings.
result Achieved an upper bound of i l d e O ( d 3 H 3 T ) ilde O(\sqrt{d^3 H^3 T}) i l d e O ( d 3 H 3 T ) on duality gap and regret.