Paper develops a new multi-agent reinforcement learning algorithm.
problem Improving policies in a network of communicating agents.
method Develops a multi-agent off-policy actor-critic algorithm using emphatic temporal difference learning.
result Proves convergence of the algorithm under linear function approximation.
A new multi-agent learning method improves performance in complex games.
problem Performance gap between MAPG and value-based multi-agent approaches.
method Introduces value function decomposition into multi-agent actor-critic framework for off-policy learning.
result DOP significantly outperforms state-of-the-art multi-agent reinforcement learning algorithms.
The paper formalizes and analyzes multi-agent Q-learning with value factorization.
problem Understanding and improving the convergence of multi-agent Q-learning with value factorization.
method Formalized a multi-agent fitted Q-iteration framework for analyzing factorized multi-agent Q-learning.
result Multi-agent Q-learning with linear value factorization can converge under certain conditions.
New algorithm for multi-agent reinforcement learning with attention mechanism.
problem Challenges in training decentralized policies in multi-agent settings.
method Actor-attention-critic algorithm with centrally computed critics and attention mechanism.
result More effective and scalable learning in complex multi-agent environments.
This paper surveys methods to handle non-stationarity in multi-agent deep reinforcement learning.
problem Non-stationarity in multi-agent reinforcement learning environments.
method Modifications in training procedures, opponent policy representation learning, meta-learning, communication, and decentralized learning.
result A comprehensive review of recent works on addressing non-stationarity in multi-agent deep reinforcement learning.
PPO algorithm converges to global optimality in multi-agent reinforcement learning.
problem Designing statistical guarantees for policy optimization methods in multi-agent reinforcement learning.
method Leveraging a multi-agent performance difference lemma, a localized action value function is used as a descent direction for each local policy, leading to a multi-agent PPO algorithm.
result The multi-agent PPO algorithm converges to the globally optimal policy at a sublinear rate under standard regularity conditions.
Paper presents a method to efficiently learn ordered representations of multi-agent data.
problem Challenges in learning consistent representations of multi-agent interactions.
method Dynamic alignment method to order multi-agent data for faster representation learning.
result Representation learning of multi-agent data is significantly accelerated.
CM3 learns multi-agent cooperation by first achieving individual goals.
problem Cooperative multi-agent control with multiple goals and interactions.
method Two-stage curriculum: first learn individual goals, then cooperation; new policy gradient with credit function.
result CM3 learns faster on multi-goal multi-agent problems than existing algorithms.
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
This work benchmarks MARL algorithms in cooperative tasks.
problem Lack of evaluation tasks and criteria for comparing MARL algorithms.
method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.
result Insights into the effectiveness of different learning approaches.
FEN model learns fairness and efficiency in multi-agent systems.
problem Fairness in multi-agent systems for stability and productivity.
method Hierarchical reinforcement learning model with fair-efficient reward and sub-policies.
result FEN model easily learns both fairness and efficiency in multi-agent scenarios.
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
RFM models predict multi-agent behavior, offering insights and faster learning.
problem Understanding and improving multi-agent systems learning.
method Relational Forward Models (RFM) that learn to predict future agent behavior.
result RFM modules inside agents lead to faster learning compared to baseline methods.
Reduces overestimation bias in multi-agent RL, improving performance.
problem Value function overestimation bias in multi-agent RL.
method Double centralized critics to reduce overestimation bias.
result Significant improvement in performance on mixed tasks.
This paper reviews deep RL methods for multi-agent systems.
problem Challenges in high-dimensional environments for RL algorithms.
method Survey of different MADRL approaches, including non-stationarity, partial observability, etc.
result Insights into robust multi-agent learning methods.
A new framework for multi-agent inverse reinforcement learning.
problem Finding suitable reward functions for multi-agent scenarios.
method MA-AIRL, based on a new solution concept and maximum pseudolikelihood estimation.
result MA-AIRL can recover highly correlated reward functions and significantly outperforms prior methods.
Improved exploration in cooperative multi-agent reinforcement learning.
problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.
method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.
result Consistently improved performance on various MARL benchmarks.
New algorithms tackle multi-agent problems with hybrid action spaces.
problem Applying deep reinforcement learning to multi-agent problems with discrete-continuous hybrid action spaces.
method Proposed two novel algorithms: Deep MAPQN and Deep MAHHQN, using centralized training and decentralized execution.
result Empirical results show both algorithms significantly outperform existing methods.
Agents learn to cooperate by exchanging messages in a shared graph model.
problem Creating effective multi-agent cooperation in unknown environments.
method Shared agent-entity graph, multi-agent reinforcement learning, invariant to team size and permutation.
result Decentralized multi-agent systems can quickly transfer learned policies to different team sizes.
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
problem Challenges in cooperative multi-agent reinforcement learning, especially credit assignment and large action spaces.
method Decomposed Soft Actor-Critic (mSAC) method with Q network architecture, discrete probabilistic policy, and counterfactual advantage function.
result Significantly outperforms policy-based approach COMA and achieves competitive results with SOTA value-based approach Qmix.
NeuroMAS treats multi-agent systems as neural networks for scalable, trainable coordination.
problem Designing multi-agent systems as hand-designed workflows is inefficient and inflexible.
method NeuroMAS treats multi-agent systems as a neural network architecture with reinforcement learning for scalable coordination.
result NeuroMAS improves significantly over multi-agent baselines and can be scaled progressively.
In this paper, we explore using deep reinforcement learning for problems with multiple agents. Most existing methods for deep multi-agent reinforcement learning consider only a small number of agents. When the number of agents increases, the dimensionality of the input and control spaces increase as well, and these met…
New algorithm for multi-agent reinforcement learning with reduced communication.
problem Cooperative learning among multiple agents with limited communication.
method Randomized multi-agent actor-critic algorithm for directed graphs.
result Algorithm solves problem for strongly connected graphs with reduced communication.
MagNet uses neural networks to predict multi-agent dynamics from observations.
problem Predicting the evolution of complex multi-agent systems.
method Formulated a coupled non-linear network with ODE-based state evolution, trained a neural network to discover dynamics from observations.
result Orders of magnitude improvement in prediction accuracy over traditional models.
Graph Convolutional Reinforcement Learning improves cooperation in dynamic multi-agent environments.
problem Learning cooperation in dynamic multi-agent environments is challenging.
method Graph Convolutional Reinforcement Learning adapts to dynamic graphs and captures interplay between agents.
result Our method substantially outperforms existing methods in cooperative scenarios.
The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.
problem Cooperative multi-agent reinforcement learning with function approximation.
method Careful message-passing and cooperative value iteration.
result Achieving near-optimal no-regret learning with limited communication in cooperative multi-agent settings.
Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
The paper introduces a health-informed policy gradient method for multi-agent reinforcement learning.
problem Optimizing joint reward functions in multi-agent systems with varying agent health.
method Health-informed credit assignment in a multi-agent proximal policy optimization algorithm.
result Significant improvement in learning performance compared to traditional methods.
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
SMAC benchmarks StarCraft II multi-agent micromanagement challenges.
problem Benchmarking cooperative multi-agent reinforcement learning.
method Proposes StarCraft Multi-Agent Challenge (SMAC) for evaluation.
result SMAC provides a standard benchmark for multi-agent RL.
New neural policies learn multi-agent relationships directly, improving coordination in dynamic environments.
problem Training coordination among varying numbers of agents in reinforcement learning.
method Attentional architecture for shared policies that adapt to each agent's context.
result Superior performance on multi-agent vehicle coordination problem, especially with many agents.
Study differential privacy in multi-agent RL, achieving efficient and private learning.
problem Protecting sensitive data in multi-agent reinforcement learning.
method Extending DP definitions to two-player games, designing an efficient algorithm with privatized bonuses.
result Achieved trajectory-wise differential privacy in multi-agent RL, improving regret bounds.
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
VPP learns joint policies for multi-agent RL through interactions.
problem Learning effective joint policies for multi-agent reinforcement learning.
method VPP integrates variational inference into policy layers for efficient sampling and differentiability.
result VPP outperforms previous methods on large-scale multi-agent tasks.
A novel framework for adaptive multi-agent communication in reinforcement learning.
problem Manual specification of communication structures in multi-agent reinforcement learning.
method Learning Structured Communication (LSC) framework using hierarchical graph neural networks.
result Adaptive hierarchical formations and efficient message propagation among agents.
QTRAN++ improves MARL performance in complex environments.
problem Poor empirical performance of QTRAN in complex environments.
method Stabilizing training objective, removing role separation, and introducing a multi-head mixing network.
result QTRAN++ achieves state-of-the-art performance in the Starcraft Multi-Agent Challenge (SMAC).
A scalable MARL algorithm using local rewards for cooperative multi-agent learning.
problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.
method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.
result LOMAQ scales well compared to other methods, improving performance and convergence speed.
Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
New framework for multi-agent reinforcement learning improves coordination and efficiency.
problem Coordination and effective learning in complex multi-agent systems.
method Centralized training and decentralized execution via policy distillation.
result Significantly better performance and higher sample efficiency.
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
New NPG variants ensure parameter convergence in multi-agent learning.
problem Non-convergence of parameters in NPG for multi-agent learning.
method Proposed variants of NPG for multi-agent learning scenarios.
result Global last-iterate parameter convergence guarantees in various multi-agent learning settings.
Global convergence proved for multi-agent LQRs with hierarchical actor-critic.
problem Challenges in understanding multi-agent reinforcement learning algorithms.
method Developed a hierarchical actor-critic algorithm for partially exchangeable agents.
result Global linear convergence to optimal policy proved.