We propose a method to efficiently learn diverse strategies in reinforcement learning for query reformulation in the tasks of document retrieval and question answering. In the proposed framework an agent consists of multiple specialized sub-agents and a meta-agent that learns to aggregate the answers from sub-agents to…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Diverse Agents”
DiCE uses diverse agents to explore and learn, avoiding local minima.
problem Local minima in RL due to limited exploration and correlated behavior.
method DiCE employs a group of heterogeneous agents to explore simultaneously and share experiences, with a diversity regularization mechanism.
result DiCE achieves substantial improvement over baselines in MuJoCo locomotion tasks.
Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images o…
InvestorBench benchmarks LLM agents in financial tasks.
problem Lack of a comprehensive benchmark for LLM-based financial agents.
method Developed a benchmark with diverse financial tasks and datasets.
result Evaluated LLM agents' performance across various financial products and market environments.
Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmascs.MA
Heterogeneous SVO leads to diverse policies in sequential social dilemmas.
problem Understanding how diverse social value orientations affect behavior in sequential social dilemmas.
method Extending prior reinforcement learning studies, we instantiated heterogeneous SVO in a sequential social dilemma setting and measured task-specific diversity metrics.
result Heterogeneous SVO leads to meaningfully diverse policies across various incentive structures.
This note will extend the research presented in Brown & Rogers (2009) to the case of CRRA agents. We consider the model outlined in that paper in which agents had diverse beliefs about the dividends produced by a risky asset. We now assume that the agents all have CRRA utility, with some integer coefficient of relative…
Agent learns diverse hierarchical structures in unknown environments.
problem Autonomous discovery and learning of diverse structures in unknown changing environments.
method Progressive construction of a Hierarchy of Observation Latent Models for Exploration Stratification (HOLMES).
result Agent can learn and reuse representations to progressively explore and discover diverse structures.
A novel framework uses goal-conditioned reinforcement learning to generate diverse samples.
problem Generating high-quality, diverse samples from generative models.
method Two agents: GC-agent learns to reconstruct the training set, S-agent learns to imitate GC-agent without knowing the goals.
result Empirically, the method generates diverse and high-quality samples in image synthesis.
Unified policy controls diverse agents through modular neural networks.
problem Learning control policies for various agent morphologies.
method Shared Modular Policies (SMP) with decentralized control and message passing.
result A single modular policy controls multiple agent morphologies.
DIVA generates diverse tasks for complex simulators, enabling adaptive agent training.
problem Lack of diverse training data for complex, open-ended simulators.
method Evolutionary approach using domain randomization and procedural generation.
result Successfully trains adaptive agent behavior in complex simulators.
In an adaptive population which models financial markets and distributed control, we consider how the dynamics depends on the diversity of the agents' initial preferences of strategies. When the diversity decreases, more agents tend to adapt their strategies together. This change in the environment results in dynamical…
The paper improves QD policy ensembles using distribution ratio estimators.
problem Training diverse and high-quality reinforcement learning agents.
method Using Stein variational gradient descent and distribution ratio estimators.
result The method generates diverse and high-quality reinforcement learning agents.
Adaptive populations such as those in financial markets and distributed control can be modeled by the Minority Game. We consider how their dynamics depends on the agents' initial preferences of strategies, when the agents use linear or quadratic payoff functions to evaluate their strategies. We find that the fluctuatio…
This paper proposes a new method to connect language and physical actions in reinforcement learning.
problem Connecting linguistic representations to the physical world in embodied agents.
method Language-conditioned goal generators to decouple sensorimotor learning from language acquisition.
result Agents can demonstrate a diversity of behaviors for any given instruction.
Cash Flow Entropyq-fin.GN
In this paper we aim to find a measure for the diversity of cash flows between agents in an economy. We argue that cash flows can be linked to probabilities of finding a currency unit in a given cash flow. We then use the information entropy as a natural measure of diversity. This leads to a hirarchical inequality meas…
PCL tackles collaborative learning for diverse agents, reducing sample complexity.
problem Balancing collaborative speedup with personalization for heterogeneous agents.
method AffPCL, with bias and importance correction mechanisms.
result AffPCL reduces sample complexity by a factor of , where is the number of agents and measures heterogeneity.
Improves exploration in reinforcement learning with diverse population.
problem Lack of diversity in reinforcement learning agents leads to limited exploration.
method Optimizes a population of agents using volume of behavioral manifold and online learning techniques.
result Improves exploration without sacrificing performance.
We derive a class of macroscopic differential equations that describe collective adaptation, starting from a discrete-time stochastic microscopic model. The behavior of each agent is a dynamic balance between adaptation that locally achieves the best action and memory loss that leads to randomized behavior. We show tha…
Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards. We augment the self-play setting by providing an external memory where the agent can store experience from the previous tasks. This enables the agent to come…
Framework improves agent's ability to learn from noisy images.
problem Agents tend to focus on distracting regions in unsupervised image-based goal exploration.
method Proposes a novel framework combining absolute Learning Progress with unsupervised image-based goal exploration.
result Agents successfully identify and ignore distracting regions, improving overall performance.
Algorithm improves learning by integrating diverse agents' behaviors.
problem Lack of social learning in reinforcement learning algorithms.
method Free energy approach for social bandit learning.
result Algorithm converges to optimal policy and enhances learning.
Social and individual learning in the Minority Gamephysics.soc-ph
Social learning can make financial markets inefficient, but individual learning can fix this.
problem Inefficiencies in financial markets due to social learning.
method Study of the Minority Game model with social and individual learning mechanisms.
result Individual learning can rescue a population from the inefficiencies caused by social learning.
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systemscs.MA
KVCOMM optimizes multi-agent LLM systems by reusing KV-caches, reducing redundant processing.
problem Substantial overhead from reprocessing overlapping contexts across multi-agent systems.
method KVCOMM reuses KV-caches and adjusts offsets for shared content using a pool of cached examples (anchors).
result Achieves over 70% reuse rate across diverse multi-agent tasks, up to 7.8x speedup.
Diverse Beliefsq-fin.GN
This paper presents a general framework for studying diverse beliefs in dynamic economies. Within this general framework, the characterization of a central-planner general equilbrium turns out to be very easy to derive, and leads to a range of interesting applications. We show how for an economy with log investors hold…
Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow. Recent work demonstrated that using a memory buffer of previous successful trajectories can result in more effective…
In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biase…
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
We consider a scenario where an agent has multiple available strategies to explore an unknown environment. For each new interaction with the environment, the agent must select which exploration strategy to use. We provide a new strategy-agnostic method that treat the situation as a Multi-Armed Bandits problem where the…
A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcs.LG
A method for a single policy to solve various tasks across diverse agent morphologies.
problem Generalizing a single policy to solve various tasks across diverse agent morphologies.
method Unified representation and behavior distillation using a morphology-task graph and Transformer architecture.
result Improves multi-task performances compared to baselines, suggesting a promising approach.
Multi-agent models have been used in many contexts to study generic collective behavior. Similarly, complex networks have become very popular because of the diversity of growth rules giving rise to scale-free behavior. Here we study adaptive networks where the agents trade ``wealth'' when they are linked together while…
Advanced simulation paradigm of human behaviour unveils complex financial systemic projectionq-fin.TR
New simulation model predicts financial market dynamics with high accuracy.
problem Extreme difficulty in financial market projections due to human behavioural complexity.
method Agent-based modeling with a hierarchical knowledge architecture to simulate diverse human groups.
result Simulator achieves 13.29% deviation in crisis scenarios and lower mean square error under normal conditions.
PEAR dynamically reconfigures agent roles to prevent persistent biases in multi-agent debates.
problem Persistent positional biases and sensitivity to role assignments in fixed topologies.
method Dynamic reconfiguration of agent roles and sparse topologies based on evolving agent states.
result Significantly improves average accuracy over debate baselines across multiple reasoning benchmarks.
We propose MAD-GAN, an intuitive generalization to the Generative Adversarial Networks (GANs) and its conditional variants to address the well known problem of mode collapse. First, MAD-GAN is a multi-agent GAN architecture incorporating multiple generators and one discriminator. Second, to enforce that different gener…
Study optimal stopping for group with diverse discount rates using an attitude function.
problem Optimal stopping for a group with diverse discount rates under an aggregation preference.
method Develop iterative approach using consistent planning for time-consistent equilibria.
result Characterize all time-consistent mild equilibria as fixed points of an operator.
Dynamic Process of Money Transfer Modelsphysics.soc-ph
We have studied numerically the statistical mechanics of the dynamic phenomena, including money circulation and economic mobility, in some transfer models. The models on which our investigations were performed are the basic model proposed by A. Dragulescu and V. Yakovenko [1], the model with uniform saving rate develop…
We use the Minority Game as a testing frame for the problem of the emergence of diversity in socio-economic systems. For the MG with heterogeneous impacts, we show that the direct generalization of the usual agents' profit does not fit some real-world situations. As a typical example we use the traffic formulation of t…
Regularize! Don't Mix: Multi-Agent Reinforcement Learning without Explicit Centralized Structurescs.LG
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
AutoStan improves Bayesian models via predictive feedback.
problem Improving Bayesian models written in Stan.
method Iterative improvement of Stan models using NLPD and sampler diagnostics feedback.
result AutoStan can autonomously improve diverse Bayesian models across various structures.
This work benchmarks MARL algorithms in cooperative tasks.
problem Lack of evaluation tasks and criteria for comparing MARL algorithms.
method Systematic evaluation of three MARL algorithm classes in diverse cooperative tasks.
result Insights into the effectiveness of different learning approaches.
We propose a unified mechanism for achieving coordination and communication in Multi-Agent Reinforcement Learning (MARL), through rewarding agents for having causal influence over other agents' actions. Causal influence is assessed using counterfactual reasoning. At each timestep, an agent simulates alternate actions t…
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
Quantitative finance has had a long tradition of a bottom-up approach to complex systems inference via multi-agent systems (MAS). These statistical tools are based on modelling agents trading via a centralised order book, in order to emulate complex and diverse market phenomena. These past financial models have all rel…
We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments. This targeting behavior is learnt solely from downstream task-specific reward withou…
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
Fairness is essential for human society, contributing to stability and productivity. Similarly, fairness is also the key for many multi-agent systems. Taking fairness into multi-agent learning could help multi-agent systems become both efficient and stable. However, learning efficiency and fairness simultaneously is a …
Study optimal investment decisions for diverse risk-tolerant agents.
problem Optimizing investment choices for agents with varying risk preferences.
method Characterizes optimal behavior using certainty equivalents and lognormal risks.
result Derives optimal decision menus under known and uncertain preference distributions.
AutoFS combines trainers to improve feature selection efficiency and effectiveness.
problem Balancing feature selection efficiency and effectiveness.
method Interactive Reinforced Feature Selection (IRFS) framework with diverse trainers.
result Improved feature selection efficiency and effectiveness compared to existing methods.
Framework analyzes RL agents' behavior to explain their strengths and weaknesses.
problem Understanding and explaining RL agents' capabilities and limitations.
method Data analysis and visualization of interaction history to extract interestingness elements.
result Visual summaries help humans correctly perceive agents' strengths and weaknesses.