CSAC enables cooperative reinforcement learning for multi-stage tasks.
problem Coordinating consecutive reinforcement learning agents for long-term multi-stage tasks.
method CSAC modifies each agent's policy to maximize both current and next agent's critic.
result CSAC outperforms uncooperative policies and single-agent training in multi-room maze domain.
Improved cooperation between levels boosts reinforcement learning performance.
problem Training multi-level policies in hierarchical reinforcement learning.
method Modeling policy optimization as a multi-agent process and inducing cooperation between sub-policies.
result Inducing cooperation between sub-policies leads to stronger and more sample-efficient policies.
Introduces LTQL for factored policies in cooperative MARL.
problem Learning optimal joint policies in collaborative MARL scenarios.
method Logical Team Q-learning (LTQL) as a stochastic approximation to dynamic programming.
result LTQL provides factored policies for optimal joint behavior in cooperative MARL.
Improved exploration in cooperative multi-agent reinforcement learning.
problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.
method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.
result Consistently improved performance on various MARL benchmarks.
A new approach for cooperative multi-agent reinforcement learning with limited communication, reducing the number of communication rounds.
problem Limited communication in decentralized MARL systems leads to outdated information and unstable learning.
method Base policy prediction technique to estimate gradients and collect samples for a sequence of base policies.
result The proposed algorithm converges to an ε-Nash equilibrium with significantly fewer communication rounds and samples.
LICA learns credit assignment for cooperative agents without explicit formulation.
problem Credit assignment among cooperative agents.
method Centralized critic as hypernetwork, adaptive entropy regularization.
result Significantly outperforms previous methods on benchmarks.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
problem Challenges in cooperative multi-agent reinforcement learning, especially credit assignment and large action spaces.
method Decomposed Soft Actor-Critic (mSAC) method with Q network architecture, discrete probabilistic policy, and counterfactual advantage function.
result Significantly outperforms policy-based approach COMA and achieves competitive results with SOTA value-based approach Qmix.
Most previous studies on multi-agent reinforcement learning focus on deriving decentralized and cooperative policies to maximize a common reward and rarely consider the transferability of trained policies to new tasks. This prevents such policies from being applied to more complex multi-agent tasks. To resolve these li…
A variety of cooperative multi-agent control problems require agents to achieve individual goals while contributing to collective success. This multi-goal multi-agent setting poses difficulties for recent algorithms, which primarily target settings with a single global reward, due to two new challenges: efficient explo…
Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
DSAC improves cooperative MARL with general utilities, converging faster than existing methods.
problem Improving cooperation in multi-agent reinforcement learning with nonlinear utilities.
method Decentralized Shadow Reward Actor-Critic (DSAC) that estimates local occupancy measures and derivatives.
result DSAC converges to ε-stationarity in O(1/ε^2.5) steps with high probability, finding globally optimal policies.
Bayesian network approach for efficient cooperative MARL.
problem Leveraging inter-agent coupling information for scalable MARL algorithms.
method Modeling cooperative MARL via Bayesian networks, identifying value dependency sets, proposing P-DTDE paradigm.
result P-DTDE policy gradient estimator has lower total variance than CTDE.
Adversaries can manipulate cooperative MARL networks.
problem Fragility of consensus-based MARL algorithms to adversarial attacks.
method Study of adversarial attacks on consensus-based MARL networks.
result Consensus-based MARL algorithms are vulnerable to adversarial manipulation.
Learning to cooperate with friends and compete with foes is a key component of multi-agent reinforcement learning. Typically to do so, one requires access to either a model of or interaction with the other agent(s). Here we show how to learn effective strategies for cooperation and competition in an asymmetric informat…
New algorithm reduces individual regret and communication costs in cooperative bandits.
problem Optimal individual and group regret in cooperative multi-agent bandits.
method Integrates a new communication policy into a learning algorithm.
result Achieves optimal individual regret and constant communication costs.
We study the problem of designing AI agents that can robustly cooperate with people in human-machine partnerships. Our work is inspired by real-life scenarios in which an AI agent, e.g., a virtual assistant, has to cooperate with new users after its deployment. We model this problem via a parametric MDP framework where…
ComEx protocol reduces communication costs in cooperative bandits.
problem Minimizing communication costs in cooperative bandits while maintaining optimal performance.
method Developed ComEx protocol to reduce communication from Θ(T) to O(logT) messages. result Achieves state-of-the-art performance with significantly reduced communication cost.
This work benchmarks MARL algorithms in cooperative tasks.
problem Lack of evaluation tasks and criteria for comparing MARL algorithms.
method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.
result Insights into the effectiveness of different learning approaches.
Traffic signal control has long been considered as a critical topic in intelligent transportation systems. Most existing learning methods mainly focus on isolated intersections and suffer from inefficient training. This paper aims at the cooperative control for large scale multi-intersection traffic signal, in which a …
This paper improves transportation efficiency by teaching automated vehicles to cooperate.
problem Improving efficiency and safety of transportation systems with automated vehicles.
method Multi-agent graph reinforcement learning with attention mechanism.
result Automated vehicles can achieve better performance when learning to cooperate with each other.
Analyzes gaming in federated learning systems and provides design principles.
problem Gaming and cooperation in federated learning systems under partial observability.
method Developed an analytical framework to separate welfare-improving behavior from metric gaming.
result Derived threshold conditions for deterring harmful gaming and preserving cooperation.
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
Modelling and exploiting teammates' policies in cooperative multi-agent systems have long been an interest and also a big challenge for the reinforcement learning (RL) community. The interest lies in the fact that if the agent knows the teammates' policies, it can adjust its own policy accordingly to arrive at proper c…
Agents learn to communicate and solve navigation tasks efficiently.
problem Understanding and developing communication in artificial agents.
method Multi-agent navigation tasks in gridworld environments.
result Agents develop an interpretable communication protocol that optimally solves tasks.
I2C enables agents to learn efficient communication without redundancy.
problem Redundant broadcast communication in multi-agent cooperation.
method I2C learns a prior for agent-agent communication via causal inference and reinforcement learning.
result I2C reduces communication overhead and improves multi-agent cooperative performance.
Hierarchical MARL learns complementary skills for team coordination.
problem Creating intelligent agents that coordinate like human sports teams.
method Two-level hierarchical MARL with unsupervised skill discovery.
result Emergence of useful and complementary skills in team games.
The paper analyzes a game where players must balance short-term and long-term interests, leading to cooperative or competitive outcomes.
problem Analyzing time inconsistency in inter-personal decision-making under non-exponential discounting.
method Iterative procedures and Zorn's lemma to find Nash equilibria between players' intra-personal equilibria.
result Inter-personal equilibria exist and depend on the impatience levels of the players.
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
A new MARL framework for community-based cooperation with transfer and active exploration.
problem Flexible coordination patterns in multi-agent systems with community structures.
method Community-based multi-agent reinforcement learning with transfer and active exploration.
result Provably convergent actor-critic algorithms for structured information sharing and transfer learning.
Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.
problem Robustness of cooperative multi-agent reinforcement learning to adversaries.
method Novel attack method involving training a policy network and using targeted adversarial examples.
result Reduces team reward from 20 to 9.4 by attacking a single agent, reducing winning rate from 98.9% to 0%.
Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to non-cooperative approaches in the strongly-convex setting. More re…
In the present work we propose an original analytical model of coopetitive game. We try to apply this analytical model of coopetition - based on game theory and conceived at a macro level - to the Greek crisis, suggesting feasible solutions in a cooperative perspective for the divergent interests which drive the econom…
Efficient algorithms for planning in cooperative multi-agent reinforcement learning with combinatorial action spaces.
problem Planning in cooperative multi-agent reinforcement learning with a combinatorial action space.
method Efficient algorithms using local access to a simulator and linear function approximation, with improvements for additive feature decomposition and kernelized settings.
result Polynomial compute and query complexity in relevant problem parameters.
PPO algorithm converges to global optimality in multi-agent reinforcement learning.
problem Designing statistical guarantees for policy optimization methods in multi-agent reinforcement learning.
method Leveraging a multi-agent performance difference lemma, a localized action value function is used as a descent direction for each local policy, leading to a multi-agent PPO algorithm.
result The multi-agent PPO algorithm converges to the globally optimal policy at a sublinear rate under standard regularity conditions.
Develops methods for cooperative Bayesian inference.
problem Cooperation between learning agents.
method Sequential Bayesian inference approaches.
result Theoretical foundation for cooperative inference.
Many potential applications of reinforcement learning in the real world involve interacting with other agents whose numbers vary over time. We propose new neural policy architectures for these multi-agent problems. In contrast to other methods of training an individual, discrete policy for each agent and then enforcing…
Modeling agent behavior is central to understanding the emergence of complex phenomena in multiagent systems. Prior work in agent modeling has largely been task-specific and driven by hand-engineering domain-specific prior knowledge. We propose a general learning framework for modeling agent behavior in any multiagent …
In this paper, we present own point of view how the unexpected fluctuations of the long-term real interest rate can be explained. We describe a macroeconomic environment by the modification of the fundamental macroeconomic equilibrium model called the IS-LM model. Last but not least, we suggest a possible cooperation b…
Multi-cell cooperative processing with limited backhaul traffic is studied for cellular uplinks. Aiming at reduced backhaul overhead, a sparsity-regularized multi-cell receive-filter design problem is formulated. Both unstructured distributed cooperation as well as clustered cooperation, in which base station groups ar…
In this study, we investigate the use of global information to speed up the learning process and increase the cumulative rewards of reinforcement learning (RL) in competition tasks. Within the actor-critic RL, we introduce multiple cooperative critics from two levels of the hierarchy and propose a reinforcement learnin…
Develops an equilibrium model for securities pricing in a mixed cooperative and non-cooperative market.
problem Equilibrium pricing of securities in a market with cooperative and non-cooperative agents.
method Conditional extended mean-field control for cooperative agents, mean-field model for both cooperative and non-cooperative agents.
result Existence of a unique equilibrium for both finite-agent and mean-field models under certain conditions.
This paper addresses the problem of multi-agent inverse reinforcement learning (MIRL) in a two-player general-sum stochastic game framework. Five variants of MIRL are considered: uCS-MIRL, advE-MIRL, cooE-MIRL, uCE-MIRL, and uNE-MIRL, each distinguished by its solution concept. Problem uCS-MIRL is a cooperative game in…
New k-step policy gradient method avoids local optima in restricted policy classes.
problem Suboptimal local optima in policy gradient methods for restricted policy classes.
method Proposes a k-step policy gradient method to escape myopic local optima. result The method converges to near optimal solutions exponentially close to the optimal deterministic policy.
New algorithm reduces complexity in multi-agent reinforcement learning.
problem High computational complexity in exact computations for multi-agent reinforcement learning.
method Design of a scalable algorithm based on Natural Policy Gradient, using local information and limited communication.
result Converges to globally optimal policy with dimension-free complexity and localization error.
Stable cooperation emerges in fluctuating environments.
problem Evolutionary stability of cooperation in fluctuating conditions.
method Agents with fluctuating wealth share public goods, leading to cooperation.
result Agents with cooperation produce an advantage in fluctuating environments.
Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
MaxMax Q-Learning improves coordination in multi-agent reinforcement learning by refining action selection.
problem Relative over-generalization in decentralized multi-agent reinforcement learning.
method MaxMax Q-Learning employs iterative sampling and evaluation of potential next states to refine approximations of ideal state transitions.
result MaxMax Q-Learning frequently outperforms existing baselines, demonstrating enhanced convergence and sample efficiency.
The paper tackles cooperative RL with function approximation, achieving near-optimal learning with limited communication.
problem Cooperative multi-agent reinforcement learning with function approximation.
method Careful message-passing and cooperative value iteration.
result Achieving near-optimal no-regret learning with limited communication in cooperative multi-agent settings.