Generative thermal design learns optimal shapes using multi-agent reinforcement learning.
problem Complex thermal design challenges due to convection-diffusion equation and boundary interactions.
method Cooperative multi-agent deep reinforcement learning with continuous geometric representation.
result Framework learns optimal design strategies without shape derivation or differentiable objectives.
In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the ag…
This research tackles information design in multi-agent reinforcement learning.
problem Designing information to influence other adaptive agents in a non-stationary environment.
method Formulated Markov signaling game, introduced signaling gradient and extended obedience constraints.
result Developed efficient algorithm for mixed-motive tasks in multi-agent reinforcement learning.
AutoDIME automates design of multi-agent environments for RL.
problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.
Agent-based model helps design financial markets.
problem Designing financial markets that work well is complex and risky.
method Agent-based model to simulate and analyze financial markets.
result Tick size reduction impacts market efficiency.
ARCO-BO optimizes multi-agent design under heterogeneity, improving efficiency and performance.
problem Heterogeneous multi-agent optimization challenges in resource use and information sharing.
method ARCO-BO integrates a consensus mechanism, budget-aware sampling, and partial input sharing for heterogeneous design spaces.
result ARCO-BO outperforms independent and collaborative BO methods in complex multi-agent settings.
Study evaluates reinforcement learning algorithms for sequential experimental design.
problem Lack of generalization in reinforcement learning for experimental design.
method Investigated several reinforcement learning algorithms for sequential experimental design.
result Certain algorithms, using dropout or ensemble approaches, show attractive generalization properties.
NeuroMAS treats multi-agent systems as neural networks for scalable, trainable coordination.
problem Designing multi-agent systems as hand-designed workflows is inefficient and inflexible.
method NeuroMAS treats multi-agent systems as a neural network architecture with reinforcement learning for scalable coordination.
result NeuroMAS improves significantly over multi-agent baselines and can be scaled progressively.
Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.
problem Designing incentives for an adapting population in multi-agent systems without prior knowledge of their learning dynamics.
method Introduces a model-based non-episodic Reinforcement Learning (RL) formulation for steering Markovian agents towards desired policies, focusing on history-dependent strategies to handle model uncertainty.
result Identifies conditions for the existence of steering strategies to guide agents to desired policies and provides empirical algorithms to approximately solve the objective.
Study designs steering rewards for MFGs with unknown dynamics and model uncertainty.
problem Designing incentives for large populations of agents in MFGs with uncertain model details.
method Developed optimistic exploration algorithms for agents with no-adaptive regret behaviors.
result Sub-linear regret guarantees for cumulative gaps between agent behaviors and desired outcomes.
Study shows market volatility affects optimal communication design for trading strategies.
problem Investigating how communication impacts trading strategy performance in multi-agent systems.
method 5-agent LLM-based trading systems across 450 experiments spanning 21 months, comparing 5 organizational structures.
result Communication improves performance but depends on market characteristics, with competitive conversation excelling in volatile tech stocks.
In many sequential decision making tasks, it is challenging to design reward functions that help an RL agent efficiently learn behavior that is considered good by the agent designer. A number of different formulations of the reward-design problem, or close variants thereof, have been proposed in the literature. In this…
This paper studies the correlations of the average winnings of agents and the volatilities of systems based on mix-game model which is an extension of minority game (MG). In mix-game, there are two groups of agents; group1 plays the majority game, but the group2 plays the minority game. The results show that the correl…
We consider a two-agent MDP framework where agents repeatedly solve a task in a collaborative setting. We study the problem of designing a learning algorithm for the first agent (A1) that facilitates a successful collaboration even in cases when the second agent (A2) is adapting its policy in an unknown way. The key ch…
Study uses LLMs for financial sentiment analysis without fine-tuning.
problem Lack of prescriptive knowledge to leverage generative models in FSA.
method Proposes a design framework with heterogeneous LLM agents based on Minsky's theory.
result Framework yields better accuracies, especially with substantial discussions.
AIDA designs personalized audio processing algorithms for hearing aids.
problem Improving hearing aid performance based on user feedback.
method Active inference-based agent with Bayesian trial design.
result AIDA proposes optimal alternative values for hearing aid parameters.
A study on how a principal can incentivize an agent to make better decisions in a repeated game.
problem Optimizing a principal's utility in a misaligned principal-agent bandit game.
method Developed nearly optimal learning algorithms for the principal's regret in multi-armed and linear contextual settings.
result The principal can iteratively learn an incentive policy to maximize her total utility.
Introduces bsuite for studying RL agent capabilities.
problem Investigate core capabilities of RL agents.
method Collects and automates evaluation of experiments.
result Facilitates reproducible research on RL.
FEN model learns fairness and efficiency in multi-agent systems.
problem Fairness in multi-agent systems for stability and productivity.
method Hierarchical reinforcement learning model with fair-efficient reward and sub-policies.
result FEN model easily learns both fairness and efficiency in multi-agent scenarios.
AI agents learn to cooperate with users of unknown type.
problem Designing AI agents that can cooperate with new users effectively.
method Modeling user behavior as parameters, observing user actions to infer type, and adapting policies.
result Adaptive AI agents perform significantly better than non-adaptive ones in real scenarios.
FinMem enhances LLM trading agents with layered memory and character design.
problem Developing purpose-driven LLM agents for financial decision-making.
method Integrates layered memory and character design modules into an LLM framework.
result Significantly enhanced trading performance in financial markets.
Video games improve vehicle routing performance.
problem Optimizing vehicle routes with unpredictable passenger requests.
method Replaced vehicle routing with a game, trained agents to play.
result General game-playing agents outperform traditional methods.
Model simulates Perpetual Futures market with agent behavior.
problem Reproduce Perpetual Futures market dynamics.
method Agent-based model with heterogeneous agents trading via a central limit order book.
result Simulation accurately reproduces Perpetual Futures price pegging to Spot price.
The paper addresses human-like decision-making in multi-agent systems using bounded risk-sensitive Markov Games.
problem Modeling human-like decision-making in multi-agent systems with risk-seeking and loss-aversion behaviors.
method Forward policy design and inverse reward learning with iterative reasoning and cumulative prospect theory.
result The proposed algorithms demonstrate both risk-averse and risk-seeking behaviors in multi-agent systems.
Study of repeated games with unobserved agent rewards using MAB framework.
problem Designing policies for principals in repeated principal-agent games with unobservable agent rewards.
method Developed a policy achieving low regret (square-root regret up to a log factor) for perfect-knowledge agents.
result Constructed an estimator for agent's expected reward and designed a policy achieving low regret.
Mechanism designs for unknown agent values in stochastic bandit settings.
problem Designing truthful mechanisms for maximizing social welfare in settings with unknown agent values and stochastic feedback.
method Developed a VCG-like mechanism with regret bounds for multi-round allocations, balancing agent and seller welfare.
result Achieved an $Ω(T^{ rac{2}{3}})$ lower bound for the maximum of welfare, agent utilities, and mechanism utility after T rounds. Framework improves self-play for cooperative multi-agent learning.
problem Evolutionary learning converges to bad local optima in multi-agent RL.
method Add imaginary rewards using peer prediction method to elicit truthful signals.
result State-of-the-art performance in predator prey, traffic junction and StarCraft tasks.
CausalGame benchmarks LLM agents' causal thinking in games.
problem Evaluating causal thinking in AI Scientists with LLMs.
method Interactive games with 14 scenarios incorporating selection bias, measurement error, and hidden confounders.
result None of the 30 LLM agents demonstrated reliable causal thinking, with the best model achieving only 68.0% survival.
Optimal contracts are found for agents with quadratic effort costs.
problem Finding optimal contracts in principal-agent problems with quadratic effort costs.
method Modeling the problem using Hamilton-Jacobi-Bellman (HJB) equations and proving the existence of classical solutions.
result Existence of optimal contracts for agents with quadratic effort costs is proven.
Study optimizes scoring rules for incentivizing agent's information gathering in online settings.
problem Optimizing incentives for agents to acquire information in online settings.
method Designing a sample-efficient algorithm that tailors the UCB algorithm to the strategic agent's model.
result Achieves sublinear T2/3-regret after T iterations, independent of the number of states. We introduce two tactics to attack agents trained by deep reinforcement learning algorithms using adversarial examples, namely the strategically-timed attack and the enchanting attack. In the strategically-timed attack, the adversary aims at minimizing the agent's reward by only attacking the agent at a small subset of…
KineticSim accelerates financial market simulations 3406x over CPU.
problem Simulating financial markets at scale with multi-agent models is bottlenecked by sequential processing and GPU kernel overhead.
method Formalized and implemented a reusable parallel design pattern for iterative multi-agent reductions in thread-block shared memory.
result Achieved a peak throughput of over 54.7 billion agent-events per second, delivering 3406x speedup over CPU.
TraderTalk uses LLMs to simulate human trading interactions in financial markets.
problem Simulating realistic human trading interactions in financial markets.
method Hybrid ABM with LLM-generated behaviors for detailed conversations.
result Successfully replicates trade-to-order volume ratios in financial markets.
This paper addresses reward estimation and incentive design for agents with hidden rewards.
problem Estimating and incentivizing agents with unknown rewards in a learning setting.
method Repeated adverse selection game with a self-interested learning agent and a learning principal. Introduces an estimator for consistent reward estimation and a data-driven incentive policy.
result Finite-sample consistency of the estimator and a rigorous regret bound for the principal.
People who design, use, and are affected by autonomous artificially intelligent agents want to be able to \emph{trust} such agents -- that is, to know that these agents will perform correctly, to understand the reasoning behind their actions, and to know how to use them appropriately. Many techniques have been devised …
This paper explores how LLMs can improve pipeline-based conversational agents.
problem Limitations of pipeline-based conversational agents in human-like conversations.
method Investigated LLMs' capabilities in two phases: design and development, and operations.
result LLMs can enhance pipeline-based agents in various tasks like data generation, intent classification, and auto-correction.
A test measures artificial agents' human-like behavior in video games.
problem Measuring the believability of artificial agents' human-like behavior.
method Developed a non-parametric two-sample hypothesis test.
result The p-value correlates with human judgment of human-like behavior. Deep learning model for AI co-creativity boosts design creativity.
problem Design creativity enhancement through AI collaboration.
method Novelty metric applied to deep learning-generated vector representations for conceptual shifts in a co-creative design system.
result Increasing novelty in AI contributions correlates with higher creative outcomes in design.
This paper introduces CENIE to quantify environment novelty for better UED.
problem Challenges in measuring environment novelty for effective UED.
method CENIE framework using state-action space coverage and Gaussian Mixture Models.
result CENIE improves UED performance across multiple benchmarks.
Study uses multi-agent reinforcement learning to control self-assembly with high-resolution external control.
problem Designing effective external control protocols for self-assembly with high-resolution control.
method Investigated a multi-agent reinforcement learning approach, comparing fully decentralized and partially decentralized strategies.
result Partially decentralized approach outperforms fully decentralized in controlling self-assembly towards target structures.
Dynamic sentiment analysis improves stock trading strategies.
problem Improving stock trading performance through dynamic agent switching.
method Combining sentiment analysis with deep reinforcement learning and dynamic agent switching.
result Dynamic sentiment-based trading strategy outperforms traditional methods.
New algorithm identifies near-optimal policies in adversarial distributed RL settings.
problem Adversarial agents in distributed RL settings that can collude and report arbitrary data.
method Weighted-Clique algorithm for robust mean estimation from batches, combined with novel distributed algorithms.
result Achieves superior robustness guarantees and near-optimal sample complexities in both offline and online settings.
Market making is a fundamental trading problem in which an agent provides liquidity by continually offering to buy and sell a security. The problem is challenging due to inventory risk, the risk of accumulating an unfavourable position and ultimately losing money. In this paper, we develop a high-fidelity simulation of…
This thesis analyzes MACL systems with low-regret learning algorithms for sequential decision making.
problem Designing efficient learning algorithms for multi-agent cooperative systems to minimize regret.
method Analyzes and develops algorithms for cooperative multi-agent multi-armed bandit problems and online convex optimization in distributed settings.
result Presented regret lower bounds and efficient algorithms for achieving these bounds, providing guidance on communication protocols.
New RL approach learns dynamic VCG mechanisms in unknown MDP environments.
problem Learning dynamic VCG mechanisms in unknown MDP environments.
method Reward-free online RL for exploration, combined with function approximation.
result Regret bound of O~(T2/3) for dynamic VCG mechanism learning. We introduce the simulation tool SABCEMM (Simulator for Agent-Based Computational Economic Market Models) for agent-based computational economic market (ABCEM) models. Our simulation tool is implemented in C++ and we can easily run ABCEM models with several million agents. The object-oriented software design enables th…
This paper optimizes driver repositioning using MARL and reward design for better service and traffic management.
problem Unserved passenger requests due to drivers' cruising behavior during passenger seeking.
method Mean field multi-agent reinforcement learning (MARL) with a reward design scheme and Bayesian optimization (BO) to solve bilevel optimization problems.
result Optimal toll charges and service charges can improve platform and city planner objectives by significant margins, leading to better traffic conditions.
fintech-kMC simulates financial platforms for AI/ML model validation.
problem Validation of AI/ML models in real-world financial applications.
method Agent-based model with kinetic Monte Carlo engine.
result Generates realistic synthetic data for testing AI/ML models.