Agents learn to give rewards to others in a shared learning environment.
problem How to encourage cooperation among RL agents in a shared environment.
method Each agent learns a reward function to influence others, optimizing for its own and others' extrinsic objectives.
result Agents significantly outperform standard RL in Markov games, often finding near-optimal division of labor.
Interactive agent modeling by learning to probe improves understanding of other agents' behaviors.
problem Understanding and predicting the behaviors of other agents in interactive scenarios.
method An interactive agent modeling scheme enabled by encouraging the agent to learn to probe, combining imitation learning and curiosity-driven reinforcement learning.
result The agent model learned by the proposed approach generalizes better and enhances performance in multiple applications.
Method models other agents' behaviors without requiring direct observation.
problem Understanding and interacting effectively with other agents in reinforcement learning.
method Extracts representations from local observations of the controlled agent using encoder-decoder architectures.
result The method achieves higher returns than baseline methods in multi-agent environments.
Deep learning agents negotiate contracts with prosocial or selfish behaviors.
problem Training agents to negotiate contracts with varying behaviors.
method Multi-Agent Reinforcement Learning, modeling prosocial and selfish behaviors, training a meta agent.
result Trained agents hold their own against human players and emulate human behavior.
New method makes deep RL agents more understandable.
problem Incomprehensible decision-making in NN-based RL agents limits their applications.
method Derives a secondary comprehensible agent from a NN-based RL agent.
result Empirical evaluation supports the possibility of building a comprehensible agent.
New algorithm for multi-agent reinforcement learning scales with number of agents.
problem Existing methods for deep multi-agent reinforcement learning struggle with increasing number of agents.
method Proposes a distributed optimization approach assuming policies of agents are close in parameter space.
result Demonstrates superior performance on co-operative and competitive tasks compared to existing methods.
This paper surveys methods to handle non-stationarity in multi-agent deep reinforcement learning.
problem Non-stationarity in multi-agent reinforcement learning environments.
method Modifications in training procedures, opponent policy representation learning, meta-learning, communication, and decentralized learning.
result A comprehensive review of recent works on addressing non-stationarity in multi-agent deep reinforcement learning.
Agents learn social skills from each other, improving performance.
problem Independent RL agents fail to use social learning.
method Imposed constraints, model-based auxiliary loss, mixed training.
result Agents trained with social learning outperform solo-trained agents.
RFM models predict multi-agent behavior, offering insights and faster learning.
problem Understanding and improving multi-agent systems learning.
method Relational Forward Models (RFM) that learn to predict future agent behavior.
result RFM modules inside agents lead to faster learning compared to baseline methods.
A method for efficient reinforcement learning query reformulation.
problem Efficiently learn diverse strategies for query reformulation.
method A framework with specialized sub-agents and a meta-agent trained on full data.
result Improved generalization performance and diversity of reformulation strategies.
Enactive learning shows agents can learn from their environment, but limited by action choices.
problem Learning and interaction of autonomous agents in complex environments.
method Simulation of artificial agents in maze environments, comparing enactive learning to classical reinforcement learning.
result Enactive agents can learn to avoid unfavorable interactions but performance is limited by action choices.
New algorithm reduces learning regret in multi-agent systems with unknown dynamics.
problem Challenges in decentralized learning due to unknown dynamics and lack of communication.
method Proposed MARL algorithm for two-agent LQ systems with unknown dynamics and one-directional communication.
result Achieved O ( T ) O(\sqrt{T}) O ( T ) regret bound for multi-agent LQ systems with certain communication patterns. I2C enables agents to learn efficient communication without redundancy.
problem Redundant broadcast communication in multi-agent cooperation.
method I2C learns a prior for agent-agent communication via causal inference and reinforcement learning.
result I2C reduces communication overhead and improves multi-agent cooperative performance.
A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.
problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.
A simple learning agent learns to trade in an agent-based market model.
problem Optimal execution of trades in an agent-based financial market model.
method Asynchronous trading through a matching engine, varying initial order sizes and state spaces, calibration of empirical stylized facts and price impact curves.
result Smaller state space agents converge faster in learning and can trade intuitively using spread and volume states.
New algorithm for multi-agent reinforcement learning with attention mechanism.
problem Challenges in training decentralized policies in multi-agent settings.
method Actor-attention-critic algorithm with centrally computed critics and attention mechanism.
result More effective and scalable learning in complex multi-agent environments.
Many learning agents impact a financial market model, showing complex dynamics.
problem Understanding the dynamics of financial markets with multiple learning agents.
method Agent-based model of financial market with multiple reinforcement learning agents interacting.
result Inclusion of learning agents changes market dynamics to match empirical data.
Modeling and learning turn-taking behaviors in multi-agent systems.
problem Modeling and predicting turn-taking behaviors in dynamic multi-agent systems.
method Individual behavior models (WFSTs) and multi-agent fusion model (logistic regression classifier).
result Accurately models and predicts turn-taking behaviors with high precision.
Agents are rewarded for influencing others' actions in MARL, improving coordination and communication.
problem Achieving effective coordination and communication in Multi-Agent Reinforcement Learning.
method Rewarding agents for having causal influence over other agents' actions, assessed through counterfactual reasoning.
result Influence rewards lead to enhanced coordination and communication in challenging social dilemma environments.
MARLA uses deep reinforcement learning for multi-agent AHT, reducing Bayes risk.
problem Decentralized active hypothesis testing among multiple agents.
method MARLA algorithm using deep multi-agent reinforcement learning.
result MARLA outperforms single-agent learning in AHT problems.
Human strategies improve deep reinforcement learning for multiple agents.
problem Limited exploration capacity of deep reinforcement learning agents.
method Integrates human strategies into multiple deep reinforcement learning agents.
result Significant performance improvement of multiple agents working together.
Algorithm helps first agent learn to collaborate with adaptive second agent in MDPs.
problem Designing a learning algorithm for the first agent to collaborate with an adaptive second agent in MDPs.
method Novel online learning algorithms for the first agent with a specific regret bound.
result Sub-linear regret of the first agent implies near-optimality of the joint return for smooth MDPs.
Paper develops a new multi-agent reinforcement learning algorithm.
problem Improving policies in a network of communicating agents.
method Develops a multi-agent off-policy actor-critic algorithm using emphatic temporal difference learning.
result Proves convergence of the algorithm under linear function approximation.
Study improves online learning with adaptable agents in various settings.
problem Learning with improving agents in online settings.
method Extensive analysis of combinatorial dimensions, multiclass setup, bandit feedback, and agent cost.
result Characterization and analysis of online learnability in the model.
RL agents outperform baselines in asset allocation.
problem Optimizing asset allocation using reinforcement learning.
method Model-free deep RL agents trained on real-world stock prices.
result RL agents significantly outperformed random and uniform allocation.
This work formalizes and extends parameter sharing in multi-agent reinforcement learning.
problem Parameter sharing limits multi-agent learning to a single policy, preventing different tasks or action spaces.
method Introduces agent indication and extends parameter sharing to heterogeneous observation and action spaces.
result Proves convergence to optimal policies for parameter sharing in heterogeneous environments.
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
Graph Convolutional Reinforcement Learning improves cooperation in dynamic multi-agent environments.
problem Learning cooperation in dynamic multi-agent environments is challenging.
method Graph Convolutional Reinforcement Learning adapts to dynamic graphs and captures interplay between agents.
result Our method substantially outperforms existing methods in cooperative scenarios.
EPC curriculum improves MARL performance as agent population grows.
problem Challenges in learning good policies for large multi-agent systems.
method Evolutionary Population Curriculum (EPC) for scaling MARL.
result EPC consistently outperforms baselines as agent population increases.
Novel approach models opponent learning dynamics in multi-agent reinforcement learning.
problem Adaptation and learning of other agents in multi-agent settings cause non-stationarity, challenging existing algorithms.
method Develops a novel approach called Learning to Model Opponent Learning (LeMOL) to accurately model opponent learning dynamics.
result Structured opponent model is more accurate and stable than naive baselines.
This paper introduces dynamic safe interruptibility for multi-agent reinforcement learning.
problem Preventing dangerous situations in decentralized multi-agent reinforcement learning.
method Introduces dynamic safe interruptibility, studies it in two learning frameworks, and gives sufficient conditions for its implementation.
result Dynamic safe interruptibility can be enabled for joint action learners but not for independent learners.
Algorithm improves learning by integrating diverse agents' behaviors.
problem Lack of social learning in reinforcement learning algorithms.
method Free energy approach for social bandit learning.
result Algorithm converges to optimal policy and enhances learning.
A framework learns agent policies from interaction data.
problem Modeling complex multiagent behavior.
method Representation learning approach using imitation and agent identification.
result Demonstrated utility in diverse multiagent tasks.
Algorithm improves multi-agent learning with noisy observations.
problem Challenges in learning optimal policies with noisy, weakly correlated observations.
method Enhanced multi-agent deep deterministic policy gradient algorithm (MADDPG-M) with a communication medium.
result Algorithm performs well in complex, non-stationary environments, offering significant performance gains.
New framework for multi-agent imitation learning in complex environments.
problem Multi-agent settings with multiple Nash equilibria and non-stationary environments.
method Generalized inverse reinforcement learning and actor-critic algorithm.
result Practical multi-agent imitation learning with good empirical performance.
Agents learn to cooperate by exchanging messages in a shared graph model.
problem Creating effective multi-agent cooperation in unknown environments.
method Shared agent-entity graph, multi-agent reinforcement learning, invariant to team size and permutation.
result Decentralized multi-agent systems can quickly transfer learned policies to different team sizes.
Paper presents a method to efficiently learn ordered representations of multi-agent data.
problem Challenges in learning consistent representations of multi-agent interactions.
method Dynamic alignment method to order multi-agent data for faster representation learning.
result Representation learning of multi-agent data is significantly accelerated.
Study of repeated principal-agent bandit game with self-interested and exploratory learning agents.
problem Interaction between principal and agent in unknown environments with learning and exploration behaviors.
method Developed algorithms for self-interested and exploratory learning agents with bandit feedback, achieving regret bounds.
result Achieved O ~ ( T 2 / 3 ) \widetilde{O}(T^{2/3}) O ( T 2/3 ) regret bound for exploratory learning agent in i.i.d. reward setup. New neural policies learn multi-agent relationships directly, improving coordination in dynamic environments.
problem Training coordination among varying numbers of agents in reinforcement learning.
method Attentional architecture for shared policies that adapt to each agent's context.
result Superior performance on multi-agent vehicle coordination problem, especially with many agents.
Paper investigates if RL can learn from observing others without explicit modeling.
problem Can RL learn from observing others without explicit modeling?
method Simple scenarios with RL agents observing and learning from another agent's actions.
result RL agents can learn from observing others, leveraging environmental effects.
CM3 learns multi-agent cooperation by first achieving individual goals.
problem Cooperative multi-agent control with multiple goals and interactions.
method Two-stage curriculum: first learn individual goals, then cooperation; new policy gradient with credit function.
result CM3 learns faster on multi-goal multi-agent problems than existing algorithms.
PCL tackles collaborative learning for diverse agents, reducing sample complexity.
problem Balancing collaborative speedup with personalization for heterogeneous agents.
method AffPCL, with bias and importance correction mechanisms.
result AffPCL reduces sample complexity by a factor of max { n − 1 , δ } \max\{n^{-1}, δ\} max { n − 1 , δ } , where n n n is the number of agents and δ ∈ [ 0 , 1 ] δ\in[0,1] δ ∈ [ 0 , 1 ] measures heterogeneity. Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
New framework for multi-agent reinforcement learning improves coordination and efficiency.
problem Coordination and effective learning in complex multi-agent systems.
method Centralized training and decentralized execution via policy distillation.
result Significantly better performance and higher sample efficiency.
New MAS model learns stock price formation through reinforcement learning.
problem Understanding complex stock price formation processes.
method Multi-agent reinforcement learning model to simulate stock markets.
result Agents develop diverse trading strategies impacting market stability and performance.
This paper proposes using variational autoencoders to model opponents in multi-agent systems.
problem Understanding and interacting with opponents in multi-agent systems.
method Variational autoencoders for opponent modeling, with a modification to use local information.
result Our opponent modeling methods achieve equal or greater episodic returns.
Agents learn to play a first-person multiplayer game at human level performance.
problem Training AI agents for complex, multi-agent, real-time environments.
method Population-based deep reinforcement learning with concurrent training of multiple agents.
result Achieved human-level performance in a first-person multiplayer game.