This paper surveys methods to handle non-stationarity in multi-agent deep reinforcement learning.
problem Non-stationarity in multi-agent reinforcement learning environments.
method Modifications in training procedures, opponent policy representation learning, meta-learning, communication, and decentralized learning.
result A comprehensive review of recent works on addressing non-stationarity in multi-agent deep reinforcement learning.
A method for efficient reinforcement learning query reformulation.
problem Efficiently learn diverse strategies for query reformulation.
method A framework with specialized sub-agents and a meta-agent trained on full data.
result Improved generalization performance and diversity of reformulation strategies.
In this paper, we explore using deep reinforcement learning for problems with multiple agents. Most existing methods for deep multi-agent reinforcement learning consider only a small number of agents. When the number of agents increases, the dimensionality of the input and control spaces increase as well, and these met…
A new framework for multi-agent inverse reinforcement learning.
problem Finding suitable reward functions for multi-agent scenarios.
method MA-AIRL, based on a new solution concept and maximum pseudolikelihood estimation.
result MA-AIRL can recover highly correlated reward functions and significantly outperforms prior methods.
New algorithm for multi-agent reinforcement learning with attention mechanism.
problem Challenges in training decentralized policies in multi-agent settings.
method Actor-attention-critic algorithm with centrally computed critics and attention mechanism.
result More effective and scalable learning in complex multi-agent environments.
Reinforcement learning optimizes market making decisions.
problem Optimizing market making decisions in cryptocurrency markets.
method Multi-agent reinforcement learning framework with two agents: macro-agent and micro-agent.
result The proposed framework leads to successful trades in the Bitcoin market.
Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
This paper uses deep reinforcement learning to optimize stock liquidation strategies.
problem Optimizing liquidation strategies in the stock market considering market impact and trader risk.
method Multi-agent deep reinforcement learning model to learn optimal selling decisions.
result Reinforcement learning models can effectively solve realistic liquidation problems.
A bandit framework selects the best reinforcement learning agent for costly interactions.
problem Optimal reinforcement learning agent selection in environments without simulators.
method Multi-arm bandit framework with surrogate rewards to select and learn from reinforcement learning agents.
result The framework consistently selects the optimal agent with more cumulative reward.
New algorithm for multi-agent reinforcement learning with reduced communication.
problem Cooperative learning among multiple agents with limited communication.
method Randomized multi-agent actor-critic algorithm for directed graphs.
result Algorithm solves problem for strongly connected graphs with reduced communication.
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
Paper develops a new multi-agent reinforcement learning algorithm.
problem Improving policies in a network of communicating agents.
method Develops a multi-agent off-policy actor-critic algorithm using emphatic temporal difference learning.
result Proves convergence of the algorithm under linear function approximation.
Study models crypto markets using multi-agent reinforcement learning.
problem Emulating crypto market dynamics and behaviors.
method Multi-agent reinforcement learning (MARL) with RL techniques.
result Model accurately emulates crypto market microstructure and behaviors.
Mobile agents classify images via reinforcement learning and consensus.
problem Image classification using multiple mobile agents.
method Proposed network architecture for local belief formation and feature extraction. Decentralized consensus protocol using reinforcement learning.
result Effectiveness of the proposed framework demonstrated on MNIST dataset.
Deep learning agents negotiate contracts with prosocial or selfish behaviors.
problem Training agents to negotiate contracts with varying behaviors.
method Multi-Agent Reinforcement Learning, modeling prosocial and selfish behaviors, training a meta agent.
result Trained agents hold their own against human players and emulate human behavior.
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.
problem Robustness of cooperative multi-agent reinforcement learning to adversaries.
method Novel attack method involving training a policy network and using targeted adversarial examples.
result Reduces team reward from 20 to 9.4 by attacking a single agent, reducing winning rate from 98.9% to 0%.
New method makes deep RL agents more understandable.
problem Incomprehensible decision-making in NN-based RL agents limits their applications.
method Derives a secondary comprehensible agent from a NN-based RL agent.
result Empirical evaluation supports the possibility of building a comprehensible agent.
A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.
problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.
Graph Convolutional Reinforcement Learning improves cooperation in dynamic multi-agent environments.
problem Learning cooperation in dynamic multi-agent environments is challenging.
method Graph Convolutional Reinforcement Learning adapts to dynamic graphs and captures interplay between agents.
result Our method substantially outperforms existing methods in cooperative scenarios.
Two policy regularization methods improve coordination in multi-agent reinforcement learning.
problem Challenging to discover successful collective behaviors in multi-agent reinforcement learning.
method Proposes TeamReg and CoachReg methods to promote coordination.
result Improved performance across cooperative multi-agent problems.
New algorithm tackles multi-agent reinforcement learning issues.
problem Multi-agent reinforcement learning suffers from the curse of many agents.
method Proposes MF-FQI algorithm based on mean embeddings of distributions.
result Establishes a non-asymptotic analysis for MF-FQI algorithm.
This paper uses counterfactual thinking to improve multi-agent reinforcement learning.
problem Improving decision-making in multi-agent environments.
method Proposes a deep reinforcement learning model with counterfactual thinking to generate and evaluate multiple actions.
result Counterfactual thinking enhances agents' performance in multi-agent environments, leading to higher rewards and fair competition.
Deep RL agents vary significantly in Atari environments.
problem Challenges in reproducibility due to stochasticity.
method Experiments with OpenAI Baselines agents.
result Variability in agent performance is significant and underreported.
New snooping attacks exploit deep RL without access to environment.
problem Security vulnerabilities in deep reinforcement learning.
method Proposes snooping threat models and attacks on RL agents.
result Adversaries can launch attacks without interacting with the environment.
Framework for multi-agent RL with human feedback in a Snake game.
problem Improving multi-agent reinforcement learning with human feedback.
method Developed a simulated game environment for offline model training and online competitions. Introduced HILL methods and reward manipulation heuristics.
result Agents with HILL methods outperform those without in online competitions.
New algorithm reduces complexity in multi-agent reinforcement learning.
problem High computational complexity in exact computations for multi-agent reinforcement learning.
method Design of a scalable algorithm based on Natural Policy Gradient, using local information and limited communication.
result Converges to globally optimal policy with dimension-free complexity and localization error.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…
Paper tackles RL for power grid topology optimization.
problem Managing large action spaces in growing power networks.
method Hierarchical multi-agent reinforcement learning (MARL) framework.
result MARL framework outperforms single-agent RL methods.
A lifelong learning architecture improves reinforcement learning policies using simulations and a DNC model.
problem Improving reinforcement learning policies in dynamic environments.
method Iterative training of a Reinforcement Learning agent and a DNC model in conjunction.
result DNC models can continually learn from pixels alone to simulate new tasks.
MARLA uses deep reinforcement learning for multi-agent AHT, reducing Bayes risk.
problem Decentralized active hypothesis testing among multiple agents.
method MARLA algorithm using deep multi-agent reinforcement learning.
result MARLA outperforms single-agent learning in AHT problems.
Adversaries can manipulate cooperative MARL networks.
problem Fragility of consensus-based MARL algorithms to adversarial attacks.
method Study of adversarial attacks on consensus-based MARL networks.
result Consensus-based MARL algorithms are vulnerable to adversarial manipulation.
Solves reinforcement learning reward design for user-aligned agents.
problem Designing effective reward functions for reinforcement learning.
method Reward modeling: learn from user interaction and optimize.
result Agents can be aligned with user intentions through learned reward functions.
Reinforcement learning crypto agent achieves high returns on Bitcoin derivatives.
problem Maximizing returns on volatile cryptocurrency markets.
method Online transfer learning with an echo state network and recurrent reinforcement learning.
result Achieves a total return of 350%, net of transaction costs, over five years.
Deep RL fails on deceptive games, revealing algorithm weaknesses.
problem Characterizing and understanding failures of deep reinforcement learning.
method Testing A2C on four deceptive games using a game framework.
result Deep RL fails in specific ways that differ from planning-based agents.
Improved PAC guarantees for multi-agent reinforcement learning with noisy communication.
problem Improving exploration in cooperative multi-agent reinforcement learning with communication constraints.
method Develops PAC guarantees for multiple concurrent MDPs with noisy and resource-limited communication.
result Theoretical and empirical improvements in sample complexity for information fusion.
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
This work formalizes and extends parameter sharing in multi-agent reinforcement learning.
problem Parameter sharing limits multi-agent learning to a single policy, preventing different tasks or action spaces.
method Introduces agent indication and extends parameter sharing to heterogeneous observation and action spaces.
result Proves convergence to optimal policies for parameter sharing in heterogeneous environments.
This paper proposes a decentralized reinforcement learning method for multi-agent resource allocation.
problem Allocating heterogeneous resources among multiple agents in a decentralized manner.
method Liquid-Graph-Time Clustering-IPPO, integrating dynamic cluster consensus.
result LGTC-IPPO achieves more stable rewards, better coordination, and robust performance.
A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.
problem Sparse-reward tasks where demonstrations are used as sources of causal knowledge.
method Develops causal models through observation and reasons from this knowledge to decompose tasks.
result A basic implementation of Reasoning from Demonstration (RfD) is effective in sparse-reward tasks.
PPO algorithm converges to global optimality in multi-agent reinforcement learning.
problem Designing statistical guarantees for policy optimization methods in multi-agent reinforcement learning.
method Leveraging a multi-agent performance difference lemma, a localized action value function is used as a descent direction for each local policy, leading to a multi-agent PPO algorithm.
result The multi-agent PPO algorithm converges to the globally optimal policy at a sublinear rate under standard regularity conditions.
Distributed learning method for multi-agent reinforcement learning with policy coordination.
problem Solving multi-agent reinforcement learning problems with coordination.
method Distributed off-policy actor critic with policy consensus.
result The proposed algorithm achieves asymptotic agreement on the global optimal policy function.
This work makes deep reinforcement learning more transparent by visualizing the decision-making process.
problem Making deep reinforcement learning models transparent and explaining their decisions.
method Proposes to visualize the evidence on which the agent bases its decision.
result Makes deep reinforcement learning more transparent and interpretable.
REFIL learns from imagined sub-group interactions to improve multi-agent reinforcement learning.
problem Learning efficiency in multi-agent reinforcement learning settings.
method Randomized Entity-wise Factorization (REFIL) approach.
result REFIL outperforms all strong baselines in challenging multi-task StarCraft settings.
Method models other agents' behaviors without requiring direct observation.
problem Understanding and interacting effectively with other agents in reinforcement learning.
method Extracts representations from local observations of the controlled agent using encoder-decoder architectures.
result The method achieves higher returns than baseline methods in multi-agent environments.
Efficient communication improves MARL performance in StarCraft II.
problem Achieving efficient communication among agents in multi-agent reinforcement learning.
method Variance Based Control (VBC) to limit message variance and preserve useful information.
result VBC reduces communication overhead by 2-10 times compared to state-of-the-art MARL algorithms.
Assisted by neural networks, reinforcement learning agents have been able to solve increasingly complex tasks over the last years. The simulation environment in which the agents interact is an essential component in any reinforcement learning problem. The environment simulates the dynamics of the agents' world and henc…