Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
Paper develops a private algorithm for multi-agent learning in bandits.
problem Private cooperative learning in decentralized systems.
method Developed extsc{FedUCB} algorithm for multi-agent learning.
result Improves pseudoregret bounds and empirical performance.
Multi-cell cooperative processing with limited backhaul traffic is studied for cellular uplinks. Aiming at reduced backhaul overhead, a sparsity-regularized multi-cell receive-filter design problem is formulated. Both unstructured distributed cooperation as well as clustered cooperation, in which base station groups ar…
A decentralized algorithm minimizes regret in a network of agents playing stochastic bandits.
problem Minimizing regret in a network of agents playing stochastic bandits with delayed information.
method Fully decentralized algorithm using accelerated consensus and UCB for delayed estimates.
result Regret bound is the optimal centralized regret plus a term depending on spectral gap of communication matrix.
Improved exploration in cooperative multi-agent reinforcement learning.
problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.
method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.
result Consistently improved performance on various MARL benchmarks.
Study uses multi-agent reinforcement learning to control self-assembly with high-resolution external control.
problem Designing effective external control protocols for self-assembly with high-resolution control.
method Investigated a multi-agent reinforcement learning approach, comparing fully decentralized and partially decentralized strategies.
result Partially decentralized approach outperforms fully decentralized in controlling self-assembly towards target structures.
LICA learns credit assignment for cooperative agents without explicit formulation.
problem Credit assignment among cooperative agents.
method Centralized critic as hypernetwork, adaptive entropy regularization.
result Significantly outperforms previous methods on benchmarks.
Bayesian network approach for efficient cooperative MARL.
problem Leveraging inter-agent coupling information for scalable MARL algorithms.
method Modeling cooperative MARL via Bayesian networks, identifying value dependency sets, proposing P-DTDE paradigm.
result P-DTDE policy gradient estimator has lower total variance than CTDE.
Linear speedup achieved in non-convex optimization for decentralized systems.
problem Achieving optimal performance in decentralized non-convex optimization.
method Examined the dependence of convergence guarantees on spectral properties of combination policies.
result Linear speedup in saddle-point escape time for symmetric combination policies.
QTRAN factorizes joint action-value function for cooperative MARL without structural constraints.
problem Value-based solutions for multi-agent reinforcement learning with structural constraints.
method QTRAN: transforming joint action-value function into easily factorizable form.
result QTRAN covers a wider class of MARL tasks than previous methods.
Decentralized learning ensures stability in online queuing systems with packet rates above 1.
problem Ensuring stability in online queuing systems with decentralized learning.
method Proposed cooperative queues and a learning algorithm for packet rates above 1.
result Decentralized learning strategies guarantee stability in queuing systems with packet rates above 1.
Decentralized learning reduces regret by sharing model updates, especially with stochastic components.
problem Achieve better online problem solving without sharing private data.
method Characterize loss functions as adversarial and stochastic components, analyze DOG algorithm's regret bound.
result Decentralized online gradient (DOG) achieves a new regret bound with communication, benefiting from private data randomness.
A new algorithm reduces regret in cooperative multi-agent bandits with heavy-tailed data.
problem Cooperative multi-agent bandits with heavy-tailed data.
method MP-UCB algorithm incorporating robust estimation with message-passing protocol.
result Optimal regret bounds for MP-UCB in various settings.
Algorithm for decentralized competition among adaptive agents.
problem Decentralized competition among adaptive networks.
method Developed an algorithm for decentralized competition among teams of adaptive agents.
result Algorithm enables decentralized competition among adaptive agents.
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD. However, a rigorous convergence analysis and comparative study of different communication-reduction strategies rem…
Semi-decentralized federated learning combines device-to-server and device-to-device communications for faster convergence.
problem Faster convergence in federated learning with decentralized model training.
method Two timescale hybrid federated learning (TT-HF) with cooperative D2D model aggregations.
result Achieves sublinear convergence rate of O(1/t) with adaptive control algorithm.
A scalable MARL algorithm using local rewards for cooperative multi-agent learning.
problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.
method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.
result LOMAQ scales well compared to other methods, improving performance and convergence speed.
Study cooperative bandit learning with imperfect communication, achieving near-optimal performance.
problem Real-world distributed decision-making with imperfect communication.
method Proposed decentralized algorithms for three communication scenarios: stochastic networks, random delays, and adversarially corrupted rewards.
result Achieved competitive performance and near-optimal guarantees on group regret.
Framework improves self-play for cooperative multi-agent learning.
problem Evolutionary learning converges to bad local optima in multi-agent RL.
method Add imaginary rewards using peer prediction method to elicit truthful signals.
result State-of-the-art performance in predator prey, traffic junction and StarCraft tasks.
pFedGame uses game theory for decentralized federated learning in dynamic networks.
problem Performance bottlenecks, data bias, model convergence issues, and model poisoning attacks in federated learning.
method pFedGame employs game theory to decentralize federated learning, avoiding a central aggregation server and addressing dynamic network challenges.
result pFedGame achieves higher accuracy (over 70%) in heterogeneous data compared to existing methods.
New algorithm reduces learning regret in multi-agent systems with unknown dynamics.
problem Challenges in decentralized learning due to unknown dynamics and lack of communication.
method Proposed MARL algorithm for two-agent LQ systems with unknown dynamics and one-directional communication.
result Achieved O ( T ) O(\sqrt{T}) O ( T ) regret bound for multi-agent LQ systems with certain communication patterns. Bayesian algorithms minimize cumulative regret in decentralized multi-agent bandits.
problem Minimizing cumulative regret in a decentralized multi-agent multi-armed bandit problem.
method Proposed decentralized Bayesian multi-armed bandit framework, including Thompson Sampling and Bayes-UCB algorithms.
result Regret scales logarithmically with constants matching those of an optimal centralized agent.
Hierarchical MARL learns complementary skills for team coordination.
problem Creating intelligent agents that coordinate like human sports teams.
method Two-level hierarchical MARL with unsupervised skill discovery.
result Emergence of useful and complementary skills in team games.
A decentralized approach for agents to learn and optimize collectively.
problem Challenges in coordinating non-cooperative agents to solve complex sequential decision problems.
method Designing a learning environment where agents learn by trading and optimizing local objectives, leading to a Nash equilibrium.
result Decentralized reinforcement learning algorithms that can handle various decision-making scenarios.
Agents learn to cooperate by exchanging messages in a shared graph model.
problem Creating effective multi-agent cooperation in unknown environments.
method Shared agent-entity graph, multi-agent reinforcement learning, invariant to team size and permutation.
result Decentralized multi-agent systems can quickly transfer learned policies to different team sizes.
Decentralized learning over graphs improves model accuracy.
problem Training machine learning models in a fully decentralized manner.
method Nodes aggregate information from one-hop neighbors to learn a model fitting the entire network.
result The proposed algorithm reduces error probability for every node in the network.
This paper presents novel Gaussian process decentralized data fusion algorithms exploiting the notion of agent-centric support sets for distributed cooperative perception of large-scale environmental phenomena. To overcome the limitations of scale in existing works, our proposed algorithms allow every mobile sensing ag…
The paper addresses private and Byzantine-proof cooperative decision-making in multi-agent systems.
problem Designing algorithms for multi-agent decision-making that are private and resilient to faulty agents.
method Upper-confidence bound algorithms for stochastic bandit problems under privacy and Byzantine conditions.
result Optimal regret achieved in both private and Byzantine-tolerant settings.
We consider the problem of decentralized clustering and estimation over multi-task networks, where agents infer and track different models of interest. The agents do not know beforehand which model is generating their own data. They also do not know which agents in their neighborhood belong to the same cluster. We prop…
Partner-aware algorithms improve AI collaboration in multi-agent settings.
problem Improving AI cooperation in teams with shared rewards.
method Proposed Partner-Aware strategy extending Upper Confidence Bound for decentralized MAB.
result Achieves logarithmic regret in collaborative decision-making.
Reinforcement Learning (RL) is a learning paradigm concerned with learning to control a system so as to maximize an objective over the long term. This approach to learning has received immense interest in recent times and success manifests itself in the form of human-level performance on games like \textit{Go}. While R…
DSPG improves SPSA for distributed optimization with wireless delays.
problem Optimizing global functions in multi-agent systems with wireless delays and errors.
method Cross-entropy based distributed stochastic approximation algorithm (DSPG) using simultaneous perturbation.
result DSPG reduces biases due to communication delays and maintains convergence rate.
Thanks to the advances in machine learning, data-driven analysis tools have become valuable solutions for various applications. However, there still remain essential challenges to develop effective data-driven methods because of the need to acquire a large amount of data and to have sufficient computing power to handle…
A fully decentralized multi-agent algorithm converges linearly with minimal memory.
problem Efficiently evaluating policies in multi-agent settings with limited exploration.
method Fully decentralized, combining off-policy learning, eligibility traces, and linear function approximation.
result Achieves linear convergence with minimal memory requirements.
DSAC improves cooperative MARL with general utilities, converging faster than existing methods.
problem Improving cooperation in multi-agent reinforcement learning with nonlinear utilities.
method Decentralized Shadow Reward Actor-Critic (DSAC) that estimates local occupancy measures and derivatives.
result DSAC converges to ε-stationarity in O(1/ε^2.5) steps with high probability, finding globally optimal policies.
Proposes a model for multi-agent reinforcement learning with hierarchical graph attention network.
problem Limited transferability of trained policies to new multi-agent tasks.
method Uses hierarchical graph attention network for representation learning and multi-agent actor-critic for policy learning.
result Demonstrates superior performance in mixed cooperative and competitive tasks compared to existing methods.
MaxMax Q-Learning improves coordination in multi-agent reinforcement learning by refining action selection.
problem Relative over-generalization in decentralized multi-agent reinforcement learning.
method MaxMax Q-Learning employs iterative sampling and evaluation of potential next states to refine approximations of ideal state transitions.
result MaxMax Q-Learning frequently outperforms existing baselines, demonstrating enhanced convergence and sample efficiency.
A new approach for cooperative multi-agent reinforcement learning with limited communication, reducing the number of communication rounds.
problem Limited communication in decentralized MARL systems leads to outdated information and unstable learning.
method Base policy prediction technique to estimate gradients and collect samples for a sequence of base policies.
result The proposed algorithm converges to an ε-Nash equilibrium with significantly fewer communication rounds and samples.
A decentralized policy achieves logarithmic regret for multi-agent MAB problems with communication constraints.
problem Decentralized policy for multi-agent MAB problems with option availability and communication constraints.
method Upper Confidence Bound (UCB) algorithms with non-stationary stochastic communication protocol.
result Guaranteed logarithmic regret for non-fully connected spatial graphs with communication constraints.
A decentralized algorithm minimizes cumulative regret in stochastic linear bandits with safety constraints.
problem Efficiently solving a linear bandit-optimization problem over a network of agents with safety constraints.
method DLUCB: a fully decentralized algorithm that minimizes cumulative regret through UCB strategy and consensus procedure.
result Near-optimal regret performance of O ( d log N T N T ) \mathcal{O}(d\log{NT}\sqrt{NT}) O ( d log N T N T ) with O ( d N 2 ) \mathcal{O}(dN^2) O ( d N 2 ) communication rate. RIVCoin stabilizes cryptocurrency portfolios through a DAO and redistributes income.
problem Stabilizing cryptocurrency value and aligning incentives for all users.
method Decentralized DAO, diversified reserves, and income redistribution.
result Aligns incentives for wealthier users to stabilize smaller users' risk.
Paper tackles delays in multi-agent reinforcement learning, improving performance.
problem Challenges in reinforcement learning due to delays in real-world systems.
method Proposes a novel framework for multi-agent reinforcement learning with delays, using Delay-Aware Markov Games and centralized-decentralized training.
result Demonstrates significant improvement in performance with delay-aware multi-agent reinforcement learning.
Floating Gossip improves continuous machine learning in a decentralized manner.
problem Continuous learning in infrastructure-less environments.
method Mean field approach to analyze the impact of communication and computing in Floating Gossip.
result Floating Gossip can effectively incorporate large amounts of data into machine learning models.
A new algorithm RESYNC for defenders against malicious attackers in multi-player bandits.
problem Malicious players colliding with cooperative players to prevent rewards.
method Decentralized and robust algorithm RESYNC for defenders.
result RESYNC algorithm is order-optimal, performing gracefully as the number of collisions increases.
Many machine learning problems can be formulated as consensus optimization problems which can be solved efficiently via a cooperative multi-agent system. However, the agents in the system can be unreliable due to a variety of reasons: noise, faults and attacks. Providing erroneous updates leads the optimization process…
Co-DQL improves traffic signal control using multi-agent reinforcement learning.
problem Optimizing signal timing for large-scale traffic control.
method Cooperative double Q-learning (Co-DQL) with mean field approximation and reward allocation.
result Co-DQL reduces average waiting time for vehicles in the road system.
Improves data efficiency in multi-agent control tasks using model-based reinforcement learning.
problem Limited data efficiency in reinforcement learning for multi-agent tasks.
method Decentralized model-based policy optimization (DMPO) framework.
result DMPO achieves superior data efficiency and matches model-free methods using true models.
QR-MIX models joint state-action values as a distribution to handle randomness in MARL.
problem Randomness in rewards and observations leads to randomness in long-term returns in MARL.
method QR-MIX uses quantile regression and combines it with QMIX and IQN to model joint state-action values as a distribution.
result QR-MIX outperforms QMIX in the StarCraft Multi-Agent Challenge (SMAC) environment.