This paper uses a path integral approach to model complex economic systems with many agents.
problem Modeling economic systems with a large number of interacting agents.
method Develops a path integral formalism to describe the behavior of a large number of agents in an economic system.
result The method provides an analytical treatment of business cycle models with many agents, revealing various phases and interactions.
We consider the dynamics of player's strategies in repeated market games, where the selection of strategies is determined by a learning model. Prior theoretical analysis and experimental data show that after large number of plays the average number of agents who decide to enter, per round of the game, approaches the ma…
EPC curriculum improves MARL performance as agent population grows.
problem Challenges in learning good policies for large multi-agent systems.
method Evolutionary Population Curriculum (EPC) for scaling MARL.
result EPC consistently outperforms baselines as agent population increases.
Reduced models derived from agent-based systems using Koopman theory.
problem Time-consuming simulations of large agent-based systems.
method Koopman operator theory applied to simulation data.
result Derived reduced models match known analytical results.
A game environment simulates competition among many agents for resources.
problem Understanding large-scale multiagent interactions and resource competition.
method Developed a persistent, massively multiplayer AI environment.
result Population size affects the development of skillful behaviors and niche differentiation.
Market equilibrium price proven in a large-agent model.
problem Proving market equilibrium in a large-agent setting.
method Proved existence of equilibrium price in a complete, continuous time market with infinite agents.
result The equilibrium price dynamics decouple as the number of agents increases.
New algorithms for fair item allocation with limited copies.
problem Fair division of numerous items with few copies.
method Modeling as a contextual bandit problem with sub-linear regret guarantees.
result Proposed algorithms achieve sub-linear regret in fair item allocation.
New neural policies learn multi-agent relationships directly, improving coordination in dynamic environments.
problem Training coordination among varying numbers of agents in reinforcement learning.
method Attentional architecture for shared policies that adapt to each agent's context.
result Superior performance on multi-agent vehicle coordination problem, especially with many agents.
The dynamics of many socioeconomic systems is determined by the decision making process of agents. The decision process depends on agent's characteristics, such as preferences, risk aversion, behavioral biases, etc.. In addition, in some systems the size of agents can be highly heterogeneous leading to very different i…
We consider a financial market model which consists of a financial asset and a large number of interacting agents classified into many types. Different types of agents are heterogeneous in their price expectations. Each agent can change its type based on the current empirical distribution of the types and the equilibri…
We consider the problem of \emph{fully decentralized} multi-agent reinforcement learning (MARL), where the agents are located at the nodes of a time-varying communication network. Specifically, we assume that the reward functions of the agents might correspond to different tasks, and are only known to the corresponding…
The paper models financial markets and real economy interactions using a large agent framework.
problem Understanding capital allocation and accumulation in financial markets and real economy interactions.
method Developed a field-formalism model to analyze interactions between financial markets and real economy with a large number of heterogeneous agents.
result The number of firms in each sector depends on the aggregate financial capital invested and expected long-term returns.
Study optimal treatment assignment policies under strategic agent responses.
problem Learning optimal treatment policies with strategic agents complicates estimation.
method Dynamic model with threshold convergence to mean-field equilibrium, consistent estimator for policy gradient.
result Threshold for treatment assignment converges to mean-field equilibrium threshold under large but finite number of agents.
Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a str…
We present a novel microscopic stock market model consisting of a large number of random agents modeling traders in a market. Each agent is characterized by a set of parameters that serve to make iterated predictions of two successive returns. The future price is determined according to the offer and the demand of all …
Collective behavior of the complex socio-economic systems is heavily influenced by the herding, group, behavior of individuals. The importance of the herding behavior may enable the control of the collective behavior of the individuals. In this contribution we consider a simple agent-based herding model modified to inc…
This paper attempts to find out numerically the distribution of the queue-length ratio in the context of a model of preferential attachment. Here we consider two restaurants only and a large number of customers (agents) who come to these restaurants. Each day the same number of agents sequentially arrives and decides w…
Crowdsourced predictions from microservices improve supply chain efficiency.
problem Improving supply chain efficiency through high-quality predictions.
method Trials of a multi-agent system with microservices and economic incentives.
result Empirical lessons suggest potential for a Prediction Web.
This paper finds an MFE in large stochastic games using reinforcement learning.
problem Finding an equilibrium in large stochastic games is difficult.
method Lower-myopic best response dynamics and posterior sampling for reinforcement learning.
result Policy and action distributions converge to optimal strategies in an MFE.
Model explains herding and volatility in urban housing prices.
problem Understanding non-linear price dynamics in urban housing markets.
method Agent-based model with rational households and trend-following behavior.
result Model accurately predicts price variability and herding behavior.
Efficient algorithms for planning in cooperative multi-agent reinforcement learning with combinatorial action spaces.
problem Planning in cooperative multi-agent reinforcement learning with a combinatorial action space.
method Efficient algorithms using local access to a simulator and linear function approximation, with improvements for additive feature decomposition and kernelized settings.
result Polynomial compute and query complexity in relevant problem parameters.
Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large…
New AI platform screens portfolios for desirable firms and news.
problem Optimizing portfolio selection with AI.
method Two LLM agents screen for firm fundamentals and news sentiment. Agents deliberate to generate buy/sell signals. High-dimensional estimation determines optimal weights.
result Screened portfolio's Sharpe ratio consistently estimates target, superior to baseline and conventional approaches.
JaxMARL-HFT accelerates MARL for HFT with 240x speedup.
problem Heavy computational cost in MARL for HFT.
method GPU-accelerated JAX framework for multi-agent RL.
result Agents learn to outperform benchmarks in HFT.
Develops variational framework for LQG risk-sensitive MFGs with major-minor interactions.
problem Risk-sensitive optimal control in LQG systems with major-minor interactions.
method Variational approach, nonlinear necessary and sufficient condition of optimality, equivalent risk-neutral measure, Markovian closed-loop best-response strategies.
result Derives optimal control strategies for LQG risk-sensitive MFGs with major-minor interactions, establishing Nash and ε-Nash equilibria. Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
The large majority of risk-sharing transactions involve few agents, each of whom can heavily influence the structure and the prices of securities. This paper proposes a game where agents' strategic sets consist of all possible sharing securities and pricing kernels that are consistent with Arrow-Debreu sharing rules. F…
New algorithm reduces regret in collaborative multi-agent bandit problems.
problem Optimizing decisions in a network of agents with communication delays.
method Follow-the-Regularized-Leader (FTRL) algorithm with suitable regularizers and communication protocols.
result Upper bound on individual regret matches lower bound up to a constant factor.
We explore the effects of social influence in a simple market model in which a large number of agents face a binary choice: 'to buy/not to buy' a single unit of a product at a price posted by a single seller (the monopoly case). We consider the case of 'positive externalities': an agent is more willing to buy if the ot…
Derives mean field game equations from microscopic agent dynamics.
problem Modeling behavior of large interacting agents in games.
method Derives mean field game PDEs from deterministic agent dynamics using Nash equilibria and dynamic programming.
result Derives mean field game limits and scales from agent-based financial market model.
New algorithms for RL in Markov games with independent linear function approximation, breaking the curse of multiagents.
problem Tackles the challenge of learning Markov equilibria in large state space Markov games with multiple agents.
method Proposes independent linear Markov games and designs new algorithms for learning Markov coarse correlated equilibria and Markov correlated equilibria with polynomial sample complexity.
result Breaks the curse of multiagents by achieving sample complexity bounds that scale polynomially with each agent's function class complexity.
New algorithm finds Nash equilibrium in multi-agent games.
problem Extending RL theory to multi-agent settings with large state spaces.
method Proposes an algorithm using an exploiter to find Nash equilibrium policies.
result Proves finding Nash equilibrium is possible with polynomial samples.
Bayesian network approach for efficient cooperative MARL.
problem Leveraging inter-agent coupling information for scalable MARL algorithms.
method Modeling cooperative MARL via Bayesian networks, identifying value dependency sets, proposing P-DTDE paradigm.
result P-DTDE policy gradient estimator has lower total variance than CTDE.
We formulate a stochastic game of mean field type where the agents solve optimal stopping problems and interact through the proportion of players that have already stopped. Working with a continuum of agents, typical equilibria become functions of the common noise that all agents are exposed to, whereas idiosyncratic r…
A novel algorithm reduces regret in multi-agent multi-armed bandits through gossip-based communication.
problem Minimizing average cumulative regret in a large number of agents solving the same MAB problem.
method A gossip-based protocol for decentralized decision-making and communication among agents.
result Achieves significant reduction in per-agent regret and communication complexity.
MIDAS learns to adaptively control other cars in urban driving scenarios.
problem Autonomous vehicles need to interact with other agents on the road.
method Reinforcement learning with attention mechanism to handle multiple agents.
result MIDAS policies are adaptive and robust to external changes.
We study a simple model of an asset market with informed and non-informed agents. In the absence of non-informed agents, the market becomes information efficient when the number of traders with different private information is large enough. Upon introducing non-informed agents, we find that the latter contribute signif…
New algorithm tackles multi-agent reinforcement learning with optimal convergence rate.
problem Multi-agent reinforcement learning with large state spaces and linear function approximations.
method Refined AVLPR framework with data-dependent pessimistic estimation and action-dependent bonuses.
result First algorithm with optimal O(T−1/2) convergence rate and no poly(Amax) dependency. Agent learns complex web tasks by breaking them into simpler sub-tasks.
problem Learning from large, sparse reward environments like web navigation.
method Decompose complex instructions into sub-instructions, schedule a curriculum, and use meta-learning for new tasks.
result DQN agent achieves 100% success rate on complex web tasks without human demonstrations.
We formulate and analyze a multi-agent model for the evolution of individual and systemic risk in which the local agents interact with each other through a central agent who, in turn, is influenced by the mean field of the local agents. The central agent is stabilized by a bistable potential, the only stabilizing force…
Model financial network dynamics to avoid systemic risk.
problem Emergence of systemic risk in financial networks.
method Derive solutions of random fixed point equations, analyze replicator dynamics, derive conditions for evolutionary stable strategies, verify with simulations.
result Emerging strategies converge to an attractor of an ODE, avoiding systemic risk.
A scalable MARL algorithm using local rewards for cooperative multi-agent learning.
problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.
method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.
result LOMAQ scales well compared to other methods, improving performance and convergence speed.
Modified Perceptron handles strategic agents with limited position changes.
problem Learning linear classifiers in the presence of strategic agents that can manipulate their positions.
method Developed a modified Perceptron algorithm with bounded mistakes under various manipulation costs.
result The modified Perceptron achieves bounded mistakes even when manipulation costs are unknown.
Developed scalable ABM for complex financial markets.
problem Simulating large-scale agent-based financial markets.
method Agent-based modeling, distributed computing, continuous double auction.
result Captures statistical properties of real financial markets.
Algorithm maximizes total reward in multi-agent bandits with adversarial corruptions.
problem Maximizing total reward in multi-agent bandits with adversarial corruptions.
method Proposes a cooperative learning algorithm robust to adversarial corruptions.
result Demonstrates an additive O((L/Lmin)C) regret term for an adversary with unknown corruption budget. CausalGame benchmarks LLM agents' causal thinking in games.
problem Evaluating causal thinking in AI Scientists with LLMs.
method Interactive games with 14 scenarios incorporating selection bias, measurement error, and hidden confounders.
result None of the 30 LLM agents demonstrated reliable causal thinking, with the best model achieving only 68.0% survival.
Market confidence is essential for successful investing. By incorporating multi-market into the evolutionary minority game, we investigate the effects of investor beliefs on the evolution of collective behaviors and asset prices. When there exists another investment opportunity, market confidence, including overconfide…
Model analyzes competitive pricing strategies in large markets of perishable products.
problem Maximizing profits in a competitive market of perishable products.
method Mean-field competition model, Hamilton-Jacobi-Bellman equation, iterative numerical algorithm.
result Properties of equilibrium pricing strategies and market dynamics.