Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,982 papers · 148 categories

Trend · papers per month

255075100 · May 202619922001200920172026
48 results for agent-environment boundary

Addressing RL's agent-environment boundary issues, a novel analysis ensures optimal value functions are invariant.

problem Fundamental RL concepts like value functions are not uniquely defined due to the agent-environment boundary.
method A boundary-invariant analysis of Fitted Q-Iteration, ensuring optimality guarantees are independent of the boundary choice.
result Theoretical analyses of RL algorithms, including Fitted Q-Iteration, are made invariant to the boundary choice.

Learning to cooperate is crucially important in multi-agent environments. The key is to understand the mutual interplay between agents. However, multi-agent environments are highly dynamic, where agents keep moving and their neighbors change quickly. This makes it hard to learn abstract representations of mutual interp…

2018-10-22abs ↗pdf ↗

AutoDIME automates design of multi-agent environments for RL.

problem Designing multi-agent environments for reinforcement learning is challenging.
method Developed intrinsic teacher rewards for multi-agent settings and evaluated them in various tasks.
result Value disagreement was found to be most consistent and effective across tasks.

Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…

2018-06-12abs ↗pdf ↗

PettingZoo library accelerates multi-agent reinforcement learning research.

problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.

Agents learn to give rewards to others in a shared learning environment.

problem How to encourage cooperation among RL agents in a shared environment.
method Each agent learns a reward function to influence others, optimizing for its own and others' extrinsic objectives.
result Agents significantly outperform standard RL in Markov games, often finding near-optimal division of labor.

This work uses reinforcement learning to optimize task scheduling and execution in a dynamic multi-agent warehouse environment.

problem Optimizing task scheduling and execution in a dynamic multi-agent warehouse environment with limited observability.
method Deep reinforcement learning to solve both high-level scheduling and low-level multi-agent execution problems.
result Demonstrates the effectiveness of reinforcement learning in optimizing task scheduling and execution in a dynamic multi-agent environment.

Reinforcement learning in multi-agent scenarios is important for real-world applications but presents challenges beyond those seen in single-agent settings. We present an actor-critic algorithm that trains decentralized policies in multi-agent settings, using centrally computed critics that share an attention mechanism…

2018-10-05abs ↗pdf ↗

In this paper, we study a multi-step interactive recommendation problem, where the item recommended at current step may affect the quality of future recommendations. To address the problem, we develop a novel and effective approach, named CFRL, which seamlessly integrates the ideas of both collaborative filtering (CF) …

2019-02-02abs ↗pdf ↗

This paper surveys methods to handle non-stationarity in multi-agent deep reinforcement learning.

problem Non-stationarity in multi-agent reinforcement learning environments.
method Modifications in training procedures, opponent policy representation learning, meta-learning, communication, and decentralized learning.
result A comprehensive review of recent works on addressing non-stationarity in multi-agent deep reinforcement learning.

Framework insures AI actions with reserve capital, preventing loss.

problem Ensuring safety and accountability for AI actions with varying side effects.
method Developed Actuarial Action Interface (AAI) and Authority Frontier to price and gate AI actions.
result Found common refusal and release patterns across domains, with varying required reserve capital.

This work improves autonomous racing by creating diverse opponents and adapting risk.

problem Balancing performance and safety in autonomous racing environments.
method Developed a self-play method using replica-exchange Markov chain Monte Carlo for diverse opponents and a distributionally robust bandit optimization for adaptive risk adjustment.
result Demonstrated real-time motion-planning methods achieving speeds comparable to Formula One racecars.

PSRL extension for continuing environments reduces regret.

problem Formalizing and analyzing resampling approach for reinforcement learning.
method Continuing PSRL maintains a model of the environment and replaces it with samples from the posterior distribution.
result Established an ildeO(τSAT) ilde{O}(τS \sqrt{A T}) bound on Bayesian regret.

The behavioral dynamics of multi-agent systems have a rich and orderly structure, which can be leveraged to understand these systems, and to improve how artificial agents learn to operate in them. Here we introduce Relational Forward Models (RFM) for multi-agent learning, networks that can learn to make accurate predic…

2018-09-28abs ↗pdf ↗

Sample efficiency is critical in solving real-world reinforcement learning problems, where agent-environment interactions can be costly. Imitation learning from expert advice has proved to be an effective strategy for reducing the number of interactions required to train a policy. Online imitation learning, which inter…

2018-06-12abs ↗pdf ↗

Improved RL algorithm stabilizes unknown linear systems with polynomial regret.

problem Learning and stabilizing unknown linear dynamical systems.
method Proposes an algorithm with an improved exploration strategy for fast stabilization.
result Achieves ildeO(T) ilde{\mathcal{O}}(\sqrt{T}) regret after TT time steps.

Method models other agents' behaviors without requiring direct observation.

problem Understanding and interacting effectively with other agents in reinforcement learning.
method Extracts representations from local observations of the controlled agent using encoder-decoder architectures.
result The method achieves higher returns than baseline methods in multi-agent environments.

This work improves MARL exploration efficiency using predictive guidance and prioritized experience replay.

problem Challenges in multi-agent reinforcement learning, especially in exploration efficiency and reward informativeness.
method A predictive network forecasts reward, guiding MARL to accelerate exploration. Improved prioritized experience replay utilizes TD error effectively.
result The proposed algorithm outperforms existing methods in cooperative multi-agent environments.

SARD improves adversarial robustness in two-stage L2D systems.

problem Adversarial attacks can manipulate query allocation in two-stage L2D systems.
method Introduces SARD, a convex learning algorithm with provable guarantees.
result SARD significantly improves robustness under adversarial attacks while maintaining strong clean performance.

AI simplifies trading strategies, potentially making markets more efficient.

problem Efficient market hypothesis (EMH) relies on traders optimising trading strategies based on information.
method Generalised notion of market efficiency, distinguishing model complexity through investor beliefs and trading strategies.
result Increased availability of low-cost AI systems may push towards more advanced trading strategies, potentially harder for inefficient traders.

MARL algorithm uses regularization to avoid explicit structures, improving performance.

problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.

This work explains RL policies using causal models, revealing important patterns and failures.

problem Understanding why RL policies succeed or fail in complex, high-dimensional systems.
method Developed a nonlinear Causal Model Reduction framework to learn simplified causal models from RL policy actions and rewards.
result The approach can uncover important behavioral patterns and failure modes in trained RL policies.

Advances in renewable energy generation and introduction of the government targets to improve energy efficiency gave rise to a concept of a Zero Energy Building (ZEB). A ZEB is a building whose net energy usage over a year is zero, i.e., its energy use is not larger than its overall renewables generation. A collection …

2018-10-08abs ↗pdf ↗

The paper investigates how supervised learning and self-play improve sample efficiency in teaching AI to communicate.

problem Improving sample efficiency in training AI to use natural language.
method Investigates the relationship between supervised learning and self-play, introduces supervised self-play (S2P).
result First training agents via supervised learning followed by self-play outperforms self-play followed by supervised learning.

DESTRESS optimizes decentralized nonconvex optimization with optimal IFO complexity and efficient communication.

problem Decentralized nonconvex finite-sum optimization in multi-agent systems.
method DESTRESS uses stochastic recursive gradient updates, gradient tracking, and careful hyper-parameter choices to achieve optimal IFO complexity with efficient communication.
result DESTRESS matches the optimal IFO complexity of centralized algorithms while maintaining communication efficiency.

Paper proposes a reinforcement learning method for trading using expert trajectories.

problem Inability of existing methods to handle long-term goals and delayed rewards in futures trading.
method Modeling futures trading as MDP, using reinforcement learning with expert trajectories and multiple short-term alpha factors.
result The proposed method outperforms traditional and deep learning methods in trading performance.

This paper uses deep reinforcement learning to optimize stock liquidation strategies.

problem Optimizing liquidation strategies in the stock market considering market impact and trader risk.
method Multi-agent deep reinforcement learning model to learn optimal selling decisions.
result Reinforcement learning models can effectively solve realistic liquidation problems.

ARPs improve exploration and sample efficiency in continuous control tasks.

problem Limited exploration in continuous control tasks leading to low sample efficiency.
method Introduce autoregressive policies (ARPs) with temporally coherent standard normal distributions.
result ARPs enhance exploration and sample efficiency in both simulated and real-world domains.

This work tackles robust RL in multi-agent settings, improving sample efficiency.

problem Overcoming environmental uncertainties in multi-agent reinforcement learning.
method Proposes DRNVI, a sample-efficient algorithm for learning robust equilibria in RMGs.
result Establishes near-optimal sample complexity for solving RMGs.

AdaptOn achieves logarithmic regret in adaptive control of unknown partially observable linear systems.

problem Adaptive control in partially observable linear dynamical systems.
method AdaptOn algorithm that estimates system dynamics through online learning and gradient descent.
result AdaptOn achieves a logarithmic regret bound of polylog(T) after T steps.

PFedRL-Rep learns shared and personalized policies for heterogeneous environments.

problem Poor performance of single policy in heterogeneous environments.
method Develops PFedRL-Rep framework with shared feature representation and personalized weights.
result Proves linear convergence speedup with respect to the number of agents.

New approach uses RS levels to improve stock trading RL model efficiency and stability.

problem Stability and adaptivity challenges in current financial RL applications.
method Leverages RS levels as regularization terms in MBRL for stock trading optimization.
result Enhanced performance and better profit gain with less riskiness compared to pure MBRL models.

Survey of RL methods for control systems with time delays.

problem Time delays in cyber-physical systems degrade RL performance and stability.
method Categorizes and analyzes five major families of RL methods for time delays.
result Identifies key trade-offs and practical guidelines for selecting RL methods.

ReOPD uses pre-collected teacher trajectories to distill knowledge from multi-turn interactions.

problem The cost of fully online on-policy distillation for multi-turn interactions.
method ReOPD, an off-environment alternative that reuses pre-collected teacher trajectories as replayed prefixes, addressing the prefix trap and distribution shift.
result ReOPD preserves or improves OPD-level accuracy, uses zero tool calls, and is at least 4imes imes faster per training step.

INTAGS uses interactive simulation to improve realism in multi-agent systems.

problem Challenges in developing realistic agent-based simulators for multi-agent systems.
method INTAGS introduces a novel metric to evaluate the difference between real and synthetic multi-agent systems, optimizing a stochastic policy in reinforcement learning to adapt to interactive sequential decision-making environments.
result INTAGS generates more realistic market data compared to state-of-the-art approaches.

Paper proposes DEMER to reconstruct hidden confounders for better reinforcement learning in recommendation.

problem Reinforcement learning in real-world applications is costly due to exploration in the environment.
method DEMER uses a multi-agent generative adversarial imitation learning framework to learn the environment and hidden confounder.
result DEMER effectively reconstructs hidden confounders and improves recommendation policy performance.

Study boundary value problems for elliptic operators on manifolds.

problem Characterize and analyze boundary conditions for first-order elliptic differential operators.
method Develops a new framework for elliptic boundary conditions, proving equivalence and regularity of solutions.
result Elliptic boundary conditions yield a Fredholm operator on compact manifolds.