AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
Graph Convolutional Reinforcement Learning improves cooperation in dynamic multi-agent environments.
problem Learning cooperation in dynamic multi-agent environments is challenging.
method Graph Convolutional Reinforcement Learning adapts to dynamic graphs and captures interplay between agents.
result Our method substantially outperforms existing methods in cooperative scenarios.
Adaptive policies for multi-agent RL improve adaptiveness in changing environments.
problem Non-stationarity in multi-agent reinforcement learning where other agents may alter their policies.
method Train multiple adaptive policies for each agent and a policy predictor to select the best policy at execution time.
result Agents trained with our method outperform state-of-the-art methods in all tested environments.
Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
TarMAC targets and coordinates multi-agent communication for cooperative tasks.
problem Coordinating multi-agent reinforcement learning in partially observable environments.
method Targeted multi-round communication approach without supervision.
result Improved performance and sample efficiency in diverse environments.
This work uses reinforcement learning to optimize task scheduling and execution in a dynamic multi-agent warehouse environment.
problem Optimizing task scheduling and execution in a dynamic multi-agent warehouse environment with limited observability.
method Deep reinforcement learning to solve both high-level scheduling and low-level multi-agent execution problems.
result Demonstrates the effectiveness of reinforcement learning in optimizing task scheduling and execution in a dynamic multi-agent environment.
Deep RL for multi-agent autonomous driving in dynamic environments.
problem Adapting to dynamic, multi-agent driving environments.
method Formulated Partially Observable Markov Games (POSG) for multi-agent learning.
result Demonstrated successful training of multi-agent control policies.
RFM models predict multi-agent behavior, offering insights and faster learning.
problem Understanding and improving multi-agent systems learning.
method Relational Forward Models (RFM) that learn to predict future agent behavior.
result RFM modules inside agents lead to faster learning compared to baseline methods.
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
New algorithm for multi-agent reinforcement learning with attention mechanism.
problem Challenges in training decentralized policies in multi-agent settings.
method Actor-attention-critic algorithm with centrally computed critics and attention mechanism.
result More effective and scalable learning in complex multi-agent environments.
QTRAN++ improves MARL performance in complex environments.
problem Poor empirical performance of QTRAN in complex environments.
method Stabilizing training objective, removing role separation, and introducing a multi-head mixing network.
result QTRAN++ achieves state-of-the-art performance in the Starcraft Multi-Agent Challenge (SMAC).
A new multi-agent learning method improves performance in complex games.
problem Performance gap between MAPG and value-based multi-agent approaches.
method Introduces value function decomposition into multi-agent actor-critic framework for off-policy learning.
result DOP significantly outperforms state-of-the-art multi-agent reinforcement learning algorithms.
New neural policies learn multi-agent relationships directly, improving coordination in dynamic environments.
problem Training coordination among varying numbers of agents in reinforcement learning.
method Attentional architecture for shared policies that adapt to each agent's context.
result Superior performance on multi-agent vehicle coordination problem, especially with many agents.
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
The paper introduces a health-informed policy gradient method for multi-agent reinforcement learning.
problem Optimizing joint reward functions in multi-agent systems with varying agent health.
method Health-informed credit assignment in a multi-agent proximal policy optimization algorithm.
result Significant improvement in learning performance compared to traditional methods.
Agents learn social skills from each other, improving performance.
problem Independent RL agents fail to use social learning.
method Imposed constraints, model-based auxiliary loss, mixed training.
result Agents trained with social learning outperform solo-trained agents.
This paper uses counterfactual thinking to improve multi-agent reinforcement learning.
problem Improving decision-making in multi-agent environments.
method Proposes a deep reinforcement learning model with counterfactual thinking to generate and evaluate multiple actions.
result Counterfactual thinking enhances agents' performance in multi-agent environments, leading to higher rewards and fair competition.
This paper reviews deep RL methods for multi-agent systems.
problem Challenges in high-dimensional environments for RL algorithms.
method Survey of different MADRL approaches, including non-stationarity, partial observability, etc.
result Insights into robust multi-agent learning methods.
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
ReF-ER algorithm improved performance in multi-agent reinforcement learning.
problem Improving performance in multi-agent reinforcement learning environments.
method Extended ReF-ER algorithm to include dependencies between agents and modeled environment dynamics.
result ReF-ER MARL outperforms state-of-the-art algorithms in collaborative environments.
Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…
INTAGS uses interactive simulation to improve realism in multi-agent systems.
problem Challenges in developing realistic agent-based simulators for multi-agent systems.
method INTAGS introduces a novel metric to evaluate the difference between real and synthetic multi-agent systems, optimizing a stochastic policy in reinforcement learning to adapt to interactive sequential decision-making environments.
result INTAGS generates more realistic market data compared to state-of-the-art approaches.
Algorithm improves multi-agent learning with noisy observations.
problem Challenges in learning optimal policies with noisy, weakly correlated observations.
method Enhanced multi-agent deep deterministic policy gradient algorithm (MADDPG-M) with a communication medium.
result Algorithm performs well in complex, non-stationary environments, offering significant performance gains.
Proposes ABIDES-Gym for financial markets simulation.
problem Training multi-agent systems in financial markets.
method Wrapping DEMAS into OpenAI Gym framework.
result Developed two financial markets OpenAI Gym environments.
AEC Games model represents software MARL environments better than POSGs.
problem POSGs are conceptually unsuitable for software MARL environments.
method Introduced AEC Games model as an equivalent to POSGs.
result AEC Games model is more representative of software MARL environments.
New algorithm learns efficiently in multi-agent settings.
problem Efficient learning in multi-agent Markov decision processes.
method Cooperative Prioritized Sweeping: model-based reinforcement learning with sample efficiency.
result Outperforms state-of-the-art on SysAdmin and randomized environments.
A simple DQN-based multi-agent RL system for binary actions.
problem Complexity and training issues in multi-agent reinforcement learning.
method Shared state and rewards, agent-specific actions, experience replay pool.
result Better performance and faster convergence compared to conventional methods.
Novel approach models opponent learning dynamics in multi-agent reinforcement learning.
problem Adaptation and learning of other agents in multi-agent settings cause non-stationarity, challenging existing algorithms.
method Develops a novel approach called Learning to Model Opponent Learning (LeMOL) to accurately model opponent learning dynamics.
result Structured opponent model is more accurate and stable than naive baselines.
Agents learn to give rewards to others in a shared learning environment.
problem How to encourage cooperation among RL agents in a shared environment.
method Each agent learns a reward function to influence others, optimizing for its own and others' extrinsic objectives.
result Agents significantly outperform standard RL in Markov games, often finding near-optimal division of labor.
MACC learns communication protocols by adapting counterfactual reasoning.
problem Credit assignment and non-stationarity in communication environments.
method Adapts counterfactual reasoning to overcome credit assignment and uses action policy and Q-function of other agents to handle non-stationarity.
result MACC outperforms state-of-the-art baselines in four scenarios.
Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
Paper proposes DEMER to reconstruct hidden confounders for better reinforcement learning in recommendation.
problem Reinforcement learning in real-world applications is costly due to exploration in the environment.
method DEMER uses a multi-agent generative adversarial imitation learning framework to learn the environment and hidden confounder.
result DEMER effectively reconstructs hidden confounders and improves recommendation policy performance.
Study compares MAPF and MARL algorithms for warehouse automation.
problem Optimizing multi-agent pickup and delivery in warehouse settings.
method Compared conflict-based search (MAPF) and shared experience actor-critic (MARL).
result Comprehensive benchmarking of MAPF and MARL in a simulated warehouse environment.
SMAC benchmarks StarCraft II multi-agent micromanagement challenges.
problem Benchmarking cooperative multi-agent reinforcement learning.
method Proposes StarCraft Multi-Agent Challenge (SMAC) for evaluation.
result SMAC provides a standard benchmark for multi-agent RL.
Adversarial policies can defeat RL agents in multi-agent environments.
problem Vulnerability of RL policies to adversarial perturbations.
method Demonstrated adversarial policies in a multi-agent, high-dimensional environment.
result Adversarial policies reliably win against RL agents but generate seemingly random behavior.
Google Research Football: A new 3D physics-based game for reinforcement learning.
problem Training reinforcement learning algorithms in complex, realistic environments.
method Developed a new 3D physics-based football simulator environment.
result Reported baseline results for various reinforcement algorithms.
Study models opaque financial markets using multi-agent simulation.
problem Challenges in financial markets with obscured data availability.
method Multi-agent simulation with small-scale meta-heuristic methods.
result Captures bilateral market dynamics of OTC trading.
This work benchmarks MARL algorithms in cooperative tasks.
problem Lack of evaluation tasks and criteria for comparing MARL algorithms.
method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.
result Insights into the effectiveness of different learning approaches.
Framework improves self-play for cooperative multi-agent learning.
problem Evolutionary learning converges to bad local optima in multi-agent RL.
method Add imaginary rewards using peer prediction method to elicit truthful signals.
result State-of-the-art performance in predator prey, traffic junction and StarCraft tasks.
A novel algorithm minimizes regret in a multi-agent bandit problem with time-varying random graphs and heterogeneous rewards.
problem Minimizing regret in a multi-agent multi-armed bandit problem with time-varying random graphs and heterogeneous rewards.
method Introduces a novel algorithmic framework combining averaging-based consensus with a weighting technique and upper confidence bound.
result Derives optimal instance-dependent regret upper bounds of order logT in both sub-gaussian and sub-exponential environments. This paper surveys methods to handle non-stationarity in multi-agent deep reinforcement learning.
problem Non-stationarity in multi-agent reinforcement learning environments.
method Modifications in training procedures, opponent policy representation learning, meta-learning, communication, and decentralized learning.
result A comprehensive review of recent works on addressing non-stationarity in multi-agent deep reinforcement learning.
This paper uses deep reinforcement learning to optimize stock liquidation strategies.
problem Optimizing liquidation strategies in the stock market considering market impact and trader risk.
method Multi-agent deep reinforcement learning model to learn optimal selling decisions.
result Reinforcement learning models can effectively solve realistic liquidation problems.
Modeling and learning turn-taking behaviors in multi-agent systems.
problem Modeling and predicting turn-taking behaviors in dynamic multi-agent systems.
method Individual behavior models (WFSTs) and multi-agent fusion model (logistic regression classifier).
result Accurately models and predicts turn-taking behaviors with high precision.
Agents learn sophisticated tool use and coordination in hide-and-seek.
problem Training agents to perform complex tasks in multi-agent environments.
method Multi-agent competition with hide-and-seek as a simple objective.
result Agents develop emergent strategies requiring sophisticated tool use and coordination.
CM3 learns multi-agent cooperation by first achieving individual goals.
problem Cooperative multi-agent control with multiple goals and interactions.
method Two-stage curriculum: first learn individual goals, then cooperation; new policy gradient with credit function.
result CM3 learns faster on multi-goal multi-agent problems than existing algorithms.
Multi-agent Q-learning untangles braids, improving over training.
problem Untangling braids using reinforcement learning.
method Two competing players learn to tangle and untangle braids in the OpenAI Gym environment.
result The system improves at untangling braids with more training, producing good tangled examples.
FEN model learns fairness and efficiency in multi-agent systems.
problem Fairness in multi-agent systems for stability and productivity.
method Hierarchical reinforcement learning model with fair-efficient reward and sub-policies.
result FEN model easily learns both fairness and efficiency in multi-agent scenarios.