A new multi-agent learning method improves performance in complex games.
problem Performance gap between MAPG and value-based multi-agent approaches.
method Introduces value function decomposition into multi-agent actor-critic framework for off-policy learning.
result DOP significantly outperforms state-of-the-art multi-agent reinforcement learning algorithms.
The paper formalizes and analyzes multi-agent Q-learning with value factorization.
problem Understanding and improving the convergence of multi-agent Q-learning with value factorization.
method Formalized a multi-agent fitted Q-iteration framework for analyzing factorized multi-agent Q-learning.
result Multi-agent Q-learning with linear value factorization can converge under certain conditions.
PPO algorithm converges to global optimality in multi-agent reinforcement learning.
problem Designing statistical guarantees for policy optimization methods in multi-agent reinforcement learning.
method Leveraging a multi-agent performance difference lemma, a localized action value function is used as a descent direction for each local policy, leading to a multi-agent PPO algorithm.
result The multi-agent PPO algorithm converges to the globally optimal policy at a sublinear rate under standard regularity conditions.
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
This work benchmarks MARL algorithms in cooperative tasks.
problem Lack of evaluation tasks and criteria for comparing MARL algorithms.
method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.
result Insights into the effectiveness of different learning approaches.
This paper extends off-policy reinforcement learning to the multi-agent case in which a set of networked agents communicating with their neighbors according to a time-varying graph collaboratively evaluates and improves a target policy while following a distinct behavior policy. To this end, the paper develops a multi-…
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
Improved exploration in cooperative multi-agent reinforcement learning.
problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.
method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.
result Consistently improved performance on various MARL benchmarks.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
problem Challenges in cooperative multi-agent reinforcement learning, especially credit assignment and large action spaces.
method Decomposed Soft Actor-Critic (mSAC) method with Q network architecture, discrete probabilistic policy, and counterfactual advantage function.
result Significantly outperforms policy-based approach COMA and achieves competitive results with SOTA value-based approach Qmix.
NeuroMAS treats multi-agent systems as neural networks for scalable, trainable coordination.
problem Designing multi-agent systems as hand-designed workflows is inefficient and inflexible.
method NeuroMAS treats multi-agent systems as a neural network architecture with reinforcement learning for scalable coordination.
result NeuroMAS improves significantly over multi-agent baselines and can be scaled progressively.
Reinforcement learning in multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in single-agent settings. We present an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism…
In this paper, we explore using deep reinforcement learning for problems with multiple agents. Most existing methods for deep multi-agent reinforcement learning consider only a small number of agents. When the number of agents increases, the dimensionality of the input and control spaces increase as well, and these met…
A variety of cooperative multi-agent control problems require agents to achieve individual goals while contributing to collective success. This multi-goal multi-agent setting poses difficulties for recent algorithms, which primarily target settings with a single global reward, due to two new challenges: efficient explo…
The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.
problem Cooperative multi-agent reinforcement learning with function approximation.
method Careful message-passing and cooperative value iteration.
result Achieving near-optimal no-regret learning with limited communication in cooperative multi-agent settings.
Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
While multi-agent interactions can be naturally modeled as a graph, the environment has traditionally been considered as a black box. We propose to create a shared agent-entity graph, where agents and environmental entities form vertices, and edges exist between the vertices which can communicate with each other. Agent…
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
Recent developments in deep reinforcement learning are concerned with creating decision-making agents which can perform well in various complex domains. A particular approach which has received increasing attention is multi-agent reinforcement learning, in which multiple agents learn concurrently to coordinate their ac…
Study differential privacy in multi-agent RL, achieving efficient and private learning.
problem Protecting sensitive data in multi-agent reinforcement learning.
method Extending DP definitions to two-player games, designing an efficient algorithm with privatized bonuses.
result Achieved trajectory-wise differential privacy in multi-agent RL, improving regret bounds.
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
VPP learns joint policies for multi-agent RL through interactions.
problem Learning effective joint policies for multi-agent reinforcement learning.
method VPP integrates variational inference into policy layers for efficient sampling and differentiability.
result VPP outperforms previous methods on large-scale multi-agent tasks.
QTRAN++ improves MARL performance in complex environments.
problem Poor empirical performance of QTRAN in complex environments.
method Stabilizing training objective, removing role separation, and introducing a multi-head mixing network.
result QTRAN++ achieves state-of-the-art performance in the Starcraft Multi-Agent Challenge (SMAC).
Fairness is essential for human society, contributing to stability and productivity. Similarly, fairness is also the key for many multi-agent systems. Taking fairness into multi-agent learning could help multi-agent systems become both efficient and stable. However, learning efficiency and fairness simultaneously is a …
Central to all machine learning algorithms is data representation. For multi-agent systems, selecting a representation which adequately captures the interactions among agents is challenging due to the latent group structure which tends to vary depending on context. However, in multi-agent systems with strong group stru…
A scalable MARL algorithm using local rewards for cooperative multi-agent learning.
problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.
method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.
result LOMAQ scales well compared to other methods, improving performance and convergence speed.
The behavioral dynamics of multi-agent systems have a rich and orderly structure, which can be leveraged to understand these systems, and to improve how artificial agents learn to operate in them. Here we introduce Relational Forward Models (RFM) for multi-agent learning, networks that can learn to make accurate predic…
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
New NPG variants ensure parameter convergence in multi-agent learning.
problem Non-convergence of parameters in NPG for multi-agent learning.
method Proposed variants of NPG for multi-agent learning scenarios.
result Global last-iterate parameter convergence guarantees in various multi-agent learning settings.
Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…
Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms however have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal pol…
We present the MagNet, a neural network-based multi-agent interaction model to discover the governing dynamics and predict evolution of a complex multi-agent system from observations. We formulate a multi-agent system as a coupled non-linear network with a generic ordinary differential equation (ODE) based state evolut…
New method uses LP to achieve optimal sample complexity in multi-agent reinforcement learning.
problem Achieving global optimality in multi-agent reinforcement learning with average-cost criterion.
method Randomized Linear Programming and Stochastic Primal-Dual Methods for multi-agent saddle point problems.
result Sample complexity matches tight dependencies on state and action spaces, and scales with network size.
Study collaborative learning among multi-agents in multi-armed bandits.
problem Minimizing group cumulative regret in a heterogeneous multi-agent setting.
method Developed decentralized algorithms for collaboration between N agents learning M stochastic multi-armed bandits. result Proved near-optimal behavior of proposed algorithms for group regret.
Study compares MAPF and MARL algorithms for warehouse automation.
problem Optimizing multi-agent pickup and delivery in warehouse settings.
method Compared conflict-based search (MAPF) and shared experience actor-critic (MARL).
result Comprehensive benchmarking of MAPF and MARL in a simulated warehouse environment.
Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (…
CollaQ improves multi-agent performance in StarCraft by 40% with fewer samples.
problem Limited generalization and high training rounds in multi-agent reinforcement learning.
method Formulates multi-agent collaboration as joint optimization on reward assignment, decomposes Q-function into self and interactive terms, and uses MARA loss.
result Improves win rate by 40% in StarCraft maps with same number of samples compared to state-of-the-art techniques.
Many real world tasks require multiple agents to work together. Multi-agent reinforcement learning (RL) methods have been proposed in recent years to solve these tasks, but current methods often fail to efficiently learn policies. We thus investigate the presence of a common weakness in single-agent RL, namely value fu…
New methods improve multi-agent reinforcement learning by addressing rotational dynamics.
problem Reproducibility crisis in multi-agent reinforcement learning.
method Reframing MARL approaches using Variational Inequalities (VIs) and proposing gradient-based VI methods.
result Significant performance improvements across benchmarks, including better convergence to equilibrium strategies in zero-sum games.
AgensFlow learns multi-agent coordination policies from experience.
problem Difficult coordination choices in multi-agent systems built on LLMs.
method Online policy learning from repeated trajectories, treating decisions as learnable.
result Learned routing improves coordination-heavy workflows over static wiring.
The paper develops an economic foundation for multi-agent learning in markets.
problem Learning dynamics in markets with strategic externalities.
method A two-phase incentive mechanism that estimates and uses implementable transfers to steer long-run dynamics.
result The mechanism achieves sublinear social-welfare regret and asymptotically optimal welfare under mild rationality and exploration conditions.
Novel approach models opponent learning dynamics in multi-agent reinforcement learning.
problem Adaptation and learning of other agents in multi-agent settings cause non-stationarity, challenging existing algorithms.
method Develops a novel approach called Learning to Model Opponent Learning (LeMOL) to accurately model opponent learning dynamics.
result Structured opponent model is more accurate and stable than naive baselines.
Multi-agent Q-learning untangles braids, improving over training.
problem Untangling braids using reinforcement learning.
method Two competing players learn to tangle and untangle braids in the OpenAI Gym environment.
result The system improves at untangling braids with more training, producing good tangled examples.
Framework for multi-agent RL with human feedback in a Snake game.
problem Improving multi-agent reinforcement learning with human feedback.
method Developed a simulated game environment for offline model training and online competitions. Introduced HILL methods and reward manipulation heuristics.
result Agents with HILL methods outperform those without in online competitions.