On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “MARL”

MARLeME: A Multi-Agent Reinforcement Learning Model Extraction Librarycs.LG

MARLeME extracts MARL models into symbolic models for better interpretability.

problem Improving interpretability and explainability of MARL systems.

method Develops a MARL model extraction library that approximates MARL systems with symbolic models.

result Enhanced understanding and inspection of MARL systems and agents.

MARL Reinforcement Learning Symbolic Models Interpretability Explainability

2020-04-16abs ↗pdf ↗

Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learningcs.LG

New algorithm tackles multi-agent reinforcement learning issues.

problem Multi-agent reinforcement learning suffers from the curse of many agents.

method Proposes MF-FQI algorithm based on mean embeddings of distributions.

result Establishes a non-asymptotic analysis for MF-FQI algorithm.

multi-agent reinforcement learning mean-field MARL mean embedding MF-FQI algorithm blessing of many agents

2020-06-21abs ↗pdf ↗

Greedy UnMixing for Q-Learning in Multi-Agent Reinforcement Learningcs.LG

GUM tackles MARL by avoiding overestimation through state-marginal restriction.

problem Overestimation of values in large joint state-action spaces.

method Greedy UnMixing through state-marginal restriction and unmixing.

result Superior performance compared to existing Q-learning and general MARL algorithms.

multi-agent reinforcement learning Q-learning overestimation state-marginal unmixing

2021-09-19abs ↗pdf ↗

Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscs.LG

Paper develops MFC approach to reduce MARL complexity.

problem Curse of dimensionality in cooperative MARL.

method Mean-field control (MFC) and Q-learning.

result Linear convergence rate for MFC-K-Q, independent of

N

MARL Mean-field control Q-learning Convergence Complexity

2020-02-10abs ↗pdf ↗

Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agentscs.LG

Despite the increasing interest in multi-agent reinforcement learning (MARL) in multiple communities, understanding its theoretical foundation has long been recognized as a challenging problem. In this work, we address this problem by providing a finite-sample analysis for decentralized batch MARL with networked agents…

2018-12-06abs ↗pdf ↗

Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARLcs.LG

Paper proves efficiency of MARL with transformers, addressing agent complexity.

problem Theoretical understanding of MARL with many agents and limited relational reasoning.

method Set Transformer for relational reasoning, model-free and model-based MARL algorithms.

result Provable efficiency of MARL algorithms, suboptimality gaps independent of number of agents.

MARL Transformer Relational Reasoning Model-Free Model-Based

2022-09-20abs ↗pdf ↗

Causal Multi-Agent Reinforcement Learning: Review and Open Problemscs.LG

The paper reviews MARL and its causal challenges, advocating for a 'causality first' approach.

problem Challenges in multi-agent reinforcement learning (MARL) and lack of theoretical guarantees for emergent behavior.

method Discussion of how causal methods can improve safety, interpretability, and robustness in MARL.

result Causal methods can provide strong theoretical guarantees for emergent behavior in MARL.

multi-agent reinforcement learning causality MARL reinforcement learning theoretical guarantees

2021-11-12abs ↗pdf ↗

Adversarial attacks in consensus-based multi-agent reinforcement learningeess.SY

Adversaries can manipulate cooperative MARL networks.

problem Fragility of consensus-based MARL algorithms to adversarial attacks.

method Study of adversarial attacks on consensus-based MARL networks.

result Consensus-based MARL algorithms are vulnerable to adversarial manipulation.

adversarial attacks multi-agent reinforcement learning consensus-based fragile cooperative

2021-03-11abs ↗pdf ↗

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscs.LG

Recent years have witnessed significant advances in reinforcement learning (RL), which has registered great success in solving various sequential decision-making problems in machine learning. Most of the successful RL applications, e.g., the games of Go and Poker, robotics, and autonomous driving, involve the participa…

2019-11-24abs ↗pdf ↗

ABIDES-MARL: A Multi-Agent Reinforcement Learning Environment for Endogenous Price Formation and Execution in a Limit Order Bookq-fin.TR

ABIDES-MARL uses MARL to study market behavior in a realistic financial simulation.

problem Understanding equilibrium behavior in complex financial market games.

method Combines MARL with a realistic LOB simulation to study market behavior.

result Validated approach by solving an extended Kyle model and showing how execution strategies shape market dynamics.

MARL LOB simulation market behavior equilibrium financial markets

2025-11-03abs ↗pdf ↗

Logical Team Q-learning: An approach towards factored policies in cooperative MARLcs.LG

Introduces LTQL for factored policies in cooperative MARL.

problem Learning optimal joint policies in collaborative MARL scenarios.

method Logical Team Q-learning (LTQL) as a stochastic approximation to dynamic programming.

result LTQL provides factored policies for optimal joint behavior in cooperative MARL.

Logical Team Q-learning Cooperative MARL Factored Policies Dynamic Programming

2020-06-05abs ↗pdf ↗

The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applicationscs.LG

Study compares MAPF and MARL algorithms for warehouse automation.

problem Optimizing multi-agent pickup and delivery in warehouse settings.

method Compared conflict-based search (MAPF) and shared experience actor-critic (MARL).

result Comprehensive benchmarking of MAPF and MARL in a simulated warehouse environment.

warehouse automation multi-agent systems pickup and delivery conflict-based search shared experience actor-critic

2022-03-14abs ↗pdf ↗

Addressing Rotational Learning Dynamics in Multi-Agent Reinforcement Learningstat.ML

New methods improve multi-agent reinforcement learning by addressing rotational dynamics.

problem Reproducibility crisis in multi-agent reinforcement learning.

method Reframing MARL approaches using Variational Inequalities (VIs) and proposing gradient-based VI methods.

result Significant performance improvements across benchmarks, including better convergence to equilibrium strategies in zero-sum games.

multi-agent reinforcement learning reproducibility crisis rotational dynamics variational inequalities gradient-based methods

2024-10-10abs ↗pdf ↗

Agent Environment Cycle Gamescs.LG

AEC Games model represents software MARL environments better than POSGs.

problem POSGs are conceptually unsuitable for software MARL environments.

method Introduced AEC Games model as an equivalent to POSGs.

result AEC Games model is more representative of software MARL environments.

Reinforcement Learning Multi-Agent Systems Software Implementation Game Theory Partial Observability

2020-09-28abs ↗pdf ↗

Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcs.LG

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a sim…

2019-09-06abs ↗pdf ↗

Multi-agent Reinforcement Learning for Networked System Controlcs.LG

This paper improves MARL for networked systems through new protocols and discount factors.

problem Improving control in networked systems using multi-agent reinforcement learning.

method Formulated as a spatiotemporal Markov decision process, introduced a spatial discount factor, and proposed NeurComm.

result Appropriate spatial discount factor enhances learning curves of non-communicative MARL algorithms.

multi-agent reinforcement learning networked systems spatiotemporal Markov decision process spatial discount factor neurcomm

2020-04-03abs ↗pdf ↗

Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Explorationcs.LG

Exploration efficiency is a challenging problem in multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the collaborative approach among multiple agents. Another important problem is the less informative reward restricts the learning speed of MARL compared with the informative…

2019-07-18abs ↗pdf ↗

PettingZoo: Gym for Multi-Agent Reinforcement Learningcs.LG

PettingZoo library accelerates multi-agent reinforcement learning research.

problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.

method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.

result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.

multi-agent reinforcement learning PettingZoo library reinforcement learning game models reproducibility

2020-09-30abs ↗pdf ↗

Optimal Lattice Boltzmann Closures through Multi-Agent Reinforcement Learningphysics.flu-dyn

MARL improves LBM stability and accuracy across scales.

problem Stability and accuracy issues in under-resolved LBM simulations.

method Multi-Agent Reinforcement Learning (MARL) to dynamically control local relaxation parameters.

result MARL closures stabilize simulations and recover spectra of fully resolved models.

Lattice Boltzmann Method Reinforcement Learning Hydrodynamics Stability Accuracy

2025-04-19abs ↗pdf ↗

A Review of Cooperative Multi-Agent Deep Reinforcement Learningcs.LG

Deep Reinforcement Learning has made significant progress in multi-agent systems in recent years. In this review article, we have focused on presenting recent approaches on Multi-Agent Reinforcement Learning (MARL) algorithms. In particular, we have focused on five common approaches on modeling and solving cooperative …

2019-08-11abs ↗pdf ↗

QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcs.LG

We explore value-based solutions for multi-agent reinforcement learning (MARL) tasks in the centralized training with decentralized execution (CTDE) regime popularized recently. However, VDN and QMIX are representative examples that use the idea of factorization of the joint action-value function into individual ones f…

2019-05-14abs ↗pdf ↗

F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learningcs.LG

Flexible decentralized MARL framework for cooperative multi-agent learning.

problem Complexity and impracticality of centralized MARL in complicated applications.

method Flexible fully-decentralized actor-critic MARL framework using primal-dual hybrid gradient descent.

result Competitive performance in large-scale cooperative multi-agent environments.

Reinforcement Learning Multi-Agent Systems Decentralized Learning Parameter Sharing Policy Improvement

2020-04-17abs ↗pdf ↗

Remember and Forget Experience Replay for Multi-Agent Reinforcement Learningcs.LG

ReF-ER algorithm improved performance in multi-agent reinforcement learning.

problem Improving performance in multi-agent reinforcement learning environments.

method Extended ReF-ER algorithm to include dependencies between agents and modeled environment dynamics.

result ReF-ER MARL outperforms state-of-the-art algorithms in collaborative environments.

Reinforcement Learning Multi-Agent Systems Experience Replay Neural Networks Stanford Intelligent Systems Laboratory

2022-03-24abs ↗pdf ↗

Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspectivecs.MA

Bayesian network approach for efficient cooperative MARL.

problem Leveraging inter-agent coupling information for scalable MARL algorithms.

method Modeling cooperative MARL via Bayesian networks, identifying value dependency sets, proposing P-DTDE paradigm.

result P-DTDE policy gradient estimator has lower total variance than CTDE.

Bayesian networks cooperative MARL value dependency set policy gradient scalable algorithms

2025-10-11abs ↗pdf ↗

Energy-based Surprise Minimization for Multi-Agent Value Factorizationcs.LG

EMIX minimizes surprise in multi-agent reinforcement learning.

problem Surprise and approximation bias in multi-agent reinforcement learning.

method Energy-based MIXer (EMIX) for minimizing surprise across multiple agents.

result EMIX demonstrates consistent stable performance in challenging StarCraft II scenarios.

multi-agent reinforcement learning surprise minimization energy-based approach StarCraft II overestimation bias

2020-09-16abs ↗pdf ↗

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscs.LG

This work benchmarks MARL algorithms in cooperative tasks.

problem Lack of evaluation tasks and criteria for comparing MARL algorithms.

method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.

result Insights into the effectiveness of different learning approaches.

multi-agent reinforcement learning cooperative tasks algorithm comparison evaluation benchmarks

2020-06-14abs ↗pdf ↗

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcs.LG

Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action sp…

2019-03-11abs ↗pdf ↗

REMAX: Relational Representation for Multi-Agent Explorationcs.LG

A method to train multi-agent reinforcement learning models without intrinsic rewards.

problem Training multi-agent reinforcement learning models with sparse rewards is challenging.

method A learning-based exploration strategy using variational graph autoencoder to generate initial states.

result The method improves the training and performance of multi-agent reinforcement learning models.

multi-agent reinforcement learning sparse rewards exploration strategy variational graph autoencoder latent representation

2020-08-12abs ↗pdf ↗

Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoTcs.LG

The recent success of single-agent reinforcement learning (RL) in Internet of things (IoT) systems motivates the study of multi-agent reinforcement learning (MARL), which is more challenging but more useful in large-scale IoT. In this paper, we consider a voting-based MARL problem, in which the agents vote to make grou…

2019-07-02abs ↗pdf ↗

MARL with General Utilities via Decentralized Shadow Reward Actor-Criticstat.ML

DSAC improves cooperative MARL with general utilities, converging faster than existing methods.

problem Improving cooperation in multi-agent reinforcement learning with nonlinear utilities.

method Decentralized Shadow Reward Actor-Critic (DSAC) that estimates local occupancy measures and derivatives.

result DSAC converges to ε-stationarity in O(1/ε^2.5) steps with high probability, finding globally optimal policies.

Multi-Agent Reinforcement Learning Cooperative Learning General Utilities Shadow Reward Actor-Critic

2021-05-29abs ↗pdf ↗

Randomized Exploration in Cooperative Multi-Agent Reinforcement Learningcs.LG

Unified framework for randomized exploration in cooperative MARL.

problem Efficient exploration in cooperative multi-agent reinforcement learning.

method Unified algorithm framework with two Thompson Sampling algorithms, CoopTS-PHE and CoopTS-LMC.

result Theoretical

\widetilde{\mathcal{O}}(d^{3/2}H^2\sqrt{MK})

regret bound for parallel MDPs with linear transition.

Cooperative MARL Randomized Exploration Thompson Sampling Regret Bound Federated Learning

2024-04-16abs ↗pdf ↗

Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learningcs.AI

A scalable MARL algorithm using local rewards for cooperative multi-agent learning.

problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.

method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.

result LOMAQ scales well compared to other methods, improving performance and convergence speed.

MARL locality scalability reinforcement learning multi-agent systems

2021-09-22abs ↗pdf ↗

On the Robustness of Cooperative Multi-Agent Reinforcement Learningcs.LG

Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.

problem Robustness of cooperative multi-agent reinforcement learning to adversaries.

method Novel attack method involving training a policy network and using targeted adversarial examples.

result Reduces team reward from 20 to 9.4 by attacking a single agent, reducing winning rate from 98.9% to 0%.

cooperative multi-agent reinforcement learning robustness adversarial attacks reinforcement learning team reward

2020-03-08abs ↗pdf ↗

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advancescs.LG

Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control. With the recent development of (single-agent) deep RL, there is a resurgence of interests in developing new MARL algorithms, especially those that are backed by theoretical analysis…

2019-12-09abs ↗pdf ↗

Multi-Agent Reinforcement Learning in Stochastic Networked Systemscs.LG

New scalable MARL framework for dynamic networked systems.

problem Scalability in multi-agent reinforcement learning with dynamic dependencies.

method Scalable Actor Critic framework for non-local and stochastic dependencies.

result Finite-time error bound showing convergence rate dependence on information spread speed.

multi-agent reinforcement learning scalable algorithms stochastic networks temporal difference learning finite-time convergence

2020-06-11abs ↗pdf ↗

The Emergence of Individualitycs.LG

EOI enhances individuality in multi-agent systems.

problem Improving multi-agent cooperation through individuality.

method A probabilistic classifier that predicts agent identity based on observations, with intrinsic rewards and regularizers.

result EOI significantly outperforms existing methods in multi-agent cooperative scenarios.

multi-agent reinforcement learning individuality intrinsic reward classifier discriminability MARL algorithms

2020-06-10abs ↗pdf ↗

Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycs.LG

This paper optimizes model-based RL for two-player zero-sum games with near-optimal sample complexity.

problem Optimizing model-based reinforcement learning for two-player zero-sum games with minimal samples.

method Model-based reinforcement learning approach for two-player discounted zero-sum Markov games with a generative model.

result Achieves a sample complexity of

ilde O(|S||A||B|(1-γ)^{-3}ε^{-2})

for finding the Nash equilibrium and ε-NE policies.

reinforcement learning multi-agent systems zero-sum games sample complexity Nash equilibrium

2020-07-15abs ↗pdf ↗

Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraintscs.LG

We study the problem of multi-agent reinforcement learning (MARL) with adaptivity constraints -- a new problem motivated by real-world applications where deployments of new policies are costly and the number of policy updates must be minimized. For two-player zero-sum Markov Games, we design a (policy) elimination base…

2024-02-02abs ↗pdf ↗

Regularize! Don't Mix: Multi-Agent Reinforcement Learning without Explicit Centralized Structurescs.LG

MARL algorithm uses regularization to avoid explicit structures, improving performance.

problem Lack of effective reinforcement learning methods for multi-agent systems.

method MARQ uses regularization to promote structured exploration without explicit centralized structures.

result MARQ outperforms existing methods in multi-agent environments.

Multi-Agent Reinforcement Learning Regularization Structured Exploration Reinforcement Learning Multi-Agent Systems

2021-09-19abs ↗pdf ↗

Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learningcs.LG

EPC curriculum improves MARL performance as agent population grows.

problem Challenges in learning good policies for large multi-agent systems.

method Evolutionary Population Curriculum (EPC) for scaling MARL.

result EPC consistently outperforms baselines as agent population increases.

Multi-Agent Reinforcement Learning Curriculum Learning Evolutionary Algorithms Population Scaling Adaptability

2020-03-23abs ↗pdf ↗

Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcs.LG

New MARL algorithms resolve the curse of multiagency with function approximation.

problem Challenges in Multi-Agent Reinforcement Learning (MARL) due to the curse of multiagency.

method V-Learning with Policy Replay and Decentralized Optimistic Policy Mirror Descent.

result First polynomial sample complexity results for learning approximate Coarse Correlated Equilibria (CCEs) of Markov Games under decentralized linear function approximation.

Multi-Agent Reinforcement Learning Decentralized Learning Function Approximation Coarse Correlated Equilibria Markov Games

2023-02-13abs ↗pdf ↗

Community-based Multi-Agent Reinforcement Learning with Transfer and Active Explorationcs.LG

A new MARL framework for community-based cooperation with transfer and active exploration.

problem Flexible coordination patterns in multi-agent systems with community structures.

method Community-based multi-agent reinforcement learning with transfer and active exploration.

result Provably convergent actor-critic algorithms for structured information sharing and transfer learning.

multi-agent reinforcement learning community-based transfer learning active exploration provable guarantees

2025-05-14abs ↗pdf ↗

Decentralized Diffusion Policy Learning for Enhanced Exploration in Cooperative Multi-agent Reinforcement Learningcs.MA

Improved exploration in cooperative multi-agent reinforcement learning.

problem Limited expressiveness of Gaussian policies in DecSPG hinders effective exploration.

method Proposes decentralized diffusion policy learning (DDPL) with denoising diffusion probabilistic models.

result Consistently improved performance on various MARL benchmarks.

cooperative multi-agent reinforcement learning diffusion policy learning exploration enhancement denoising diffusion score matching

2026-05-08abs ↗pdf ↗

QPLEX: Duplex Dueling Multi-Agent Q-Learningcs.LG

QPLEX learns efficient multi-agent Q-values by enforcing IGM principle.

problem Scalable multi-agent reinforcement learning with IGM consistency.

method Dueling duplex network architecture to enforce IGM principle.

result QPLEX achieves high sample efficiency and benefits from offline data.

multi-agent reinforcement learning IGM principle value function Q-learning StarCraft II

2020-08-03abs ↗pdf ↗

Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimizationcs.LG

Traffic congestion in metropolitan areas is a world-wide problem that can be ameliorated by traffic lights that respond dynamically to real-time conditions. Recent studies applying deep reinforcement learning (RL) to optimize single traffic lights have shown significant improvement over conventional control. However, o…

2019-09-23abs ↗pdf ↗

Multi-Agent Reinforcement Learning for Power Grid Topology Optimizationcs.LG

Paper tackles RL for power grid topology optimization.

problem Managing large action spaces in growing power networks.

method Hierarchical multi-agent reinforcement learning (MARL) framework.

result MARL framework outperforms single-agent RL methods.

reinforcement learning power grid topology optimization multi-agent systems hierarchical structure

2023-10-04abs ↗pdf ↗

Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcs.LG

Finding the optimal signal timing strategy is a difficult task for the problem of large-scale traffic signal control (TSC). Multi-Agent Reinforcement Learning (MARL) is a promising method to solve this problem. However, there is still room for improvement in extending to large-scale problems and modeling the behaviors …

2019-08-10abs ↗pdf ↗

Breaking the Curse of Multiagency in Robust Multi-Agent Reinforcement Learningcs.LG

New algorithm breaks multiagency gap in robust MARL.

problem Vulnerability of MARL to sim-to-real gaps.

method Distributionally robust Markov games (RMGs) with a new uncertainty set formulation.

result First algorithm to break the curse of multiagency for RMGs.

Multiagent reinforcement learning Robustness Behavioral economics Sample efficiency Curse of multiagency

2024-09-30abs ↗pdf ↗