Extracts StarCraft II tournament data for AI and ML studies.
problem Lack of accessible esports data for scientific use.
method Gathered and processed StarCraft II tournament replays using an API parser library.
result The largest publicly available StarCraft II esports dataset.
Study improves StarCraft bot's strategy selection with partial observations.
problem Selecting effective strategies in real-time strategy games with limited information.
method Utilized full game state information during training to predict opponent strategies.
result Substantial win rate improvements over a fixed-strategy baseline.
SMAC benchmarks StarCraft II multi-agent micromanagement challenges.
problem Benchmarking cooperative multi-agent reinforcement learning.
method Proposes StarCraft Multi-Agent Challenge (SMAC) for evaluation.
result SMAC provides a standard benchmark for multi-agent RL.
DefogGAN predicts hidden RTS game information to aid strategic decision-making.
problem Predicting hidden information in real-time strategy games like StarCraft.
method Conditional Generative Adversarial Network (GAN) with pyramidal reconstruction loss.
result DefogGAN predicts enemy buildings and combat units as accurately as professional players.
Paper presents TG and TL for efficient RL in StarCraft II.
problem Efficient reinforcement learning for StarCraft II.
method Abstract forward model (TG) combined with transfer learning (TL).
result 99% win-rate on 64x64 map with 1.08 hours of training.
Efficient communication improves MARL performance in StarCraft II.
problem Achieving efficient communication among agents in multi-agent reinforcement learning.
method Variance Based Control (VBC) to limit message variance and preserve useful information.
result VBC reduces communication overhead by 2-10 times compared to state-of-the-art MARL algorithms.
CollaQ improves multi-agent performance in StarCraft by 40% with fewer samples.
problem Limited generalization and high training rounds in multi-agent reinforcement learning.
method Formulates multi-agent collaboration as joint optimization on reward assignment, decomposes Q-function into self and interactive terms, and uses MARA loss.
result Improves win rate by 40% in StarCraft maps with same number of samples compared to state-of-the-art techniques.
New metric for video generation models, addressing quality, coherence, and diversity.
problem Lack of metrics for evaluating generative models of video.
method Proposed Fréchet Video Distance (FVD) and StarCraft 2 Videos (SCV) benchmark.
result FVD correlates well with human judgment of generated videos.
StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learning approach for StarCraft II. The hierarchy involves two levels of abstraction. One is the macro-acti…
Growing action spaces accelerates learning in complex tasks.
problem Efficient learning in tasks with large combinatorial action spaces.
method Curriculum of progressively growing action spaces, off-policy reinforcement learning, data transfer.
result Efficacy demonstrated in control tasks and large-scale StarCraft tasks.
QTRAN++ improves MARL performance in complex environments.
problem Poor empirical performance of QTRAN in complex environments.
method Stabilizing training objective, removing role separation, and introducing a multi-head mixing network.
result QTRAN++ achieves state-of-the-art performance in the Starcraft Multi-Agent Challenge (SMAC).
Robot learns from human demonstrations to work autonomously.
problem Leveraging human and robotic strengths in human-robot systems.
method Bayesian inference for detecting and classifying human heterogeneity.
result Bayesian approach outperforms conventional methods by up to 12.8. The paper formalizes and analyzes multi-agent Q-learning with value factorization.
problem Understanding and improving the convergence of multi-agent Q-learning with value factorization.
method Formalized a multi-agent fitted Q-iteration framework for analyzing factorized multi-agent Q-learning.
result Multi-agent Q-learning with linear value factorization can converge under certain conditions.
A new multi-agent learning method improves performance in complex games.
problem Performance gap between MAPG and value-based multi-agent approaches.
method Introduces value function decomposition into multi-agent actor-critic framework for off-policy learning.
result DOP significantly outperforms state-of-the-art multi-agent reinforcement learning algorithms.
The paper uses causal models to explain reinforcement learning agent behavior.
problem Understanding and explaining the behavior of reinforcement learning agents.
method Learning a structural causal model during reinforcement learning, generating explanations based on counterfactual analysis.
result Causal model explanations outperform other models in terms of task prediction, explanation satisfaction, and trust.
EMIX minimizes surprise in multi-agent reinforcement learning.
problem Surprise and approximation bias in multi-agent reinforcement learning.
method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.
result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.
New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.
problem Challenges in cooperative multi-agent reinforcement learning, especially credit assignment and large action spaces.
method Decomposed Soft Actor-Critic (mSAC) method with Q network architecture, discrete probabilistic policy, and counterfactual advantage function.
result Significantly outperforms policy-based approach COMA and achieves competitive results with SOTA value-based approach Qmix.
QPLEX learns efficient multi-agent Q-values by enforcing IGM principle.
problem Scalable multi-agent reinforcement learning with IGM consistency.
method Dueling duplex network architecture to enforce IGM principle.
result QPLEX achieves high sample efficiency and benefits from offline data.
QR-MIX models joint state-action values as a distribution to handle randomness in MARL.
problem Randomness in rewards and observations leads to randomness in long-term returns in MARL.
method QR-MIX uses quantile regression and combines it with QMIX and IQN to model joint state-action values as a distribution.
result QR-MIX outperforms QMIX in the StarCraft Multi-Agent Challenge (SMAC) environment.
REFIL learns from imagined sub-group interactions to improve multi-agent reinforcement learning.
problem Learning efficiency in multi-agent reinforcement learning settings.
method Randomized Entity-wise Factorization (REFIL) approach.
result REFIL outperforms all strong baselines in challenging multi-task StarCraft settings.
This paper introduces a new framework for learning nearly decomposable Q-functions via communication minimization.
problem Challenges in multi-agent reinforcement learning, especially scalability and non-stationarity.
method Learning nearly decomposable Q-functions (NDQ) via communication minimization, introducing two information-theoretic regularizers.
result Significantly outperforms baseline methods on the StarCraft unit micromanagement benchmark.
We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning. It uses self-attention to iteratively reason about the relations between entities in a scene a…
Framework improves self-play for cooperative multi-agent learning.
problem Evolutionary learning converges to bad local optima in multi-agent RL.
method Add imaginary rewards using peer prediction method to elicit truthful signals.
result State-of-the-art performance in predator prey, traffic junction and StarCraft tasks.
IC3Net improves communication efficiency in multiagent tasks.
problem Efficient communication in multiagent tasks, especially in semi-cooperative and competitive settings.
method IC3Net uses a gating mechanism and individualized rewards to control continuous communication, improving scalability and performance.
result IC3Net yields improved performance and convergence rates compared to baselines as task scale increases.
Study shows Elo models fail to accurately measure transitive strength in competitive games.
problem Elo models fail to correctly identify the transitive component in real-world competitive games.
method Investigated the challenge of identifying the transitive component in games, proposed an extension of the Elo score.
result Disc ranking system assigns two scores: skill and consistency.
Meta-learner infers subtask graph to adapt quickly to unknown tasks.
problem Adapting to unknown tasks with unknown subtask dependencies in few-shot RL.
method Meta-learner with Subtask Graph Inference (MSGI) and UCB-inspired intrinsic reward.
result Accurately infers latent task parameters and adapts more efficiently.
NeuPL learns diverse policies in strategy games efficiently.
problem Iterative training of policies in strategy games leads to under-trained good-responses and wasteful repetition.
method NeuPL uses a single conditional model to represent a population of policies, offering convergence guarantees and transfer learning.
result NeuPL achieves better performance and efficiency across various domains, enabling access to novel strategies.
Deep RL improves Diplomacy performance, outperforming previous methods.
problem Applying RL to complex, many-agent, simultaneous-move games like Diplomacy.
method Best response policy iteration and fictitious play approximation.
result Deep RL agents convincingly outperform previous Diplomacy agents.
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…
Improved exploration in cooperative multi-agent reinforcement learning.
problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.
method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.
result Consistently improved performance on various MARL benchmarks.
The paper reveals a spinning top geometry in real-world games.
problem Understanding the structure of real-world games.
method Developed a spinning top geometric model and used Nash clustering.
result Real-world games exhibit a spinning top structure with transitive and non-transitive dimensions.
New theorem guarantees approximate equilibrium in non-convex games.
problem No guarantee of equilibrium in non-convex games.
method Introduced a minimax theorem for non-convex games involving neural networks.
result Provided an approximate minimax theorem for non-convex games.
Neural network learns expert knowledge to improve reinforcement learning performance.
problem Lack of domain knowledge in reinforcement learning.
method Integrates expert knowledge into neural network architecture and uses policy gradient updates.
result Novel approach outperforms existing methods in various tasks.
RODE learns roles to simplify multi-agent tasks.
problem Efficiently discovering roles for complex multi-agent tasks.
method Clustering actions based on effects, bi-level learning hierarchy, integrating action effects into role policies.
result RODE outperforms state-of-the-art MARL algorithms on StarCraft II benchmarks.
LICA learns credit assignment for cooperative agents without explicit formulation.
problem Credit assignment among cooperative agents.
method Centralized critic as hypernetwork, adaptive entropy regularization.
result Significantly outperforms previous methods on benchmarks.
QMIX combines per-agent values to create decentralised policies.
problem Training decentralised policies from centralised learning.
method QMIX uses a mixing network to estimate joint action-values as a monotonic combination of per-agent values.
result QMIX significantly outperforms existing methods on the StarCraft Multi-Agent Challenge (SMAC).
Paper introduces a structured prediction approach for multi-agent reinforcement learning.
problem Scaling and generalization in cooperative multi-agent reinforcement learning.
method Structured prediction approach using learned scoring models for task assignment.
result The approach can generalize to larger problems with more agents and tasks.
Equivariant MuZero improves generalization in procedurally generated environments.
problem Leveraging world models for robust generalization in real-world scenarios.
method Incorporates symmetries of the environment into MuZero's world-model architecture, making MuZero equivariant to certain groups of transformations.
result Improves data efficiency and generalization capabilities, demonstrated on procedurally generated MiniPacman and Chaser.
Efficiently samples from large resolution images for faster inference.
problem Inferential and memory costs for large resolution images.
method Proposes a smaller proxy distribution to learn co-ordinates for regions of interest.
result Produces comparable results with ~10x faster inference and lower memory consumption.
Flexible decentralized MARL framework for cooperative multi-agent learning.
problem Complexity and impracticality of centralized MARL in complicated applications.
method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.
result Competitive performance in large-scale cooperative multi-agent environments.
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
This work tackles large action spaces in RL by binarizing actions.
problem Large action spaces in reinforcement learning cause significant challenges.
method Sequentializing actions and binarizing the action space.
result Binarizing the action space can significantly improve RL algorithms and reduce state space size.
A novel memory mechanism for reinforcement learning agents that stores past events in human-readable language.
problem Lack of interpretability in reinforcement learning agent's memory mechanisms.
method Uses CLIP to associate visual inputs with language tokens, then feeds these tokens to a pretrained language model.
result Significantly faster convergence on challenging continuous recognition tasks.
Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.
problem QMIX restricts Q-values to monotonic mixtures, limiting complex value functions. method Introduced weighted projection to recover optimal policies, improving performance.
result CW QMIX and OW QMIX outperform baseline QMIX on multi-agent tasks.
RL enhances cloud autoscaling by learning optimal resource allocation.
problem Optimizing resource allocation in dynamic, uncertain cloud environments.
method Reinforcement Learning for dynamic, transparent, and adaptable resource management policies.
result RL enables transparent, dynamic, and adaptable resource management policies.