Proposes ABIDES-Gym for financial markets simulation.
problem Training multi-agent systems in financial markets.
method Wrapping DEMAS into OpenAI Gym framework.
result Developed two financial markets OpenAI Gym environments.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Gym Environment”
Success stories of applied machine learning can be traced back to the datasets and environments that were put forward as challenges for the community. The challenge that the community sets as a benchmark is usually the challenge that the community eventually solves. The ultimate challenge of reinforcement learning rese…
In this paper, we propose Rogue-Gym, a simple and classic style roguelike game built for evaluating generalization in reinforcement learning (RL). Combined with the recent progress of deep neural networks, RL has successfully trained human-level agents without human knowledge in many games such as those for Atari 2600.…
Python module for RL trading in limit order books.
problem Training RL agents for algorithmic trading in limit order books.
method Model-based gym environments for reinforcement learning.
result Efficient RL training for trading problems.
Tool converts industrial systems to RL environments for optimization.
problem Lack of RL methods for optimizing industrial systems.
method Develops a tool to convert FSM models to OpenAI Gym wrappers.
result Traditional RL methods can be applied to industrial systems.
PettingZoo library accelerates multi-agent reinforcement learning research.
problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.
MARS-Gym framework for marketplaces to train and evaluate recommender systems.
problem Challenges in designing, training, and evaluating recommender systems in marketplaces.
method Open-source framework for Reinforcement Learning agents in marketplaces.
result Empowers researchers and engineers to quickly build and evaluate agents for recommendations.
The capability to learn and adapt to changes in the driving environment is crucial for developing autonomous driving systems that are scalable beyond geo-fenced operational design domains. Deep Reinforcement Learning (RL) provides a promising and scalable framework for developing adaptive learning based solutions. Deep…
We present MDP Playground, a testbed for Reinforcement Learning (RL) agents with dimensions of hardness that can be controlled independently to challenge agents in different ways and obtain varying degrees of hardness in toy and complex RL environments. We consider and allow control over a wide variety of dimensions, i…
Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Marketsq-fin.TR
Deep RL strategies outperform traditional methods in cryptocurrency trading.
problem Designing profitable trading strategies for cryptocurrency markets.
method Applied Proximal Policy Optimization, Soft Actor-Critic, and Generative Adversarial Imitation Learning to a Gym environment based on cryptocurrency markets.
result Highest gain of 4850 US dollars per 10000 US dollars investment on unseen data.
Online random forests improve Q-learning performance in specific gym environments.
problem Improving Q-learning performance in reinforcement learning tasks.
method Proposed online random forests as Q-function approximators and growing them as learning progresses.
result Improved performance over state-of-the-art Deep Q-Networks in specific gym environments.
Band-limited SAC improves learning efficiency and stability in simulated environments.
problem Improving sample efficiency and stability in SAC algorithms.
method Artificially bandlimiting the target critic's spatial resolution using a convolutional filter.
result Bandlimited SAC outperforms classic twin-critic SAC in various Gym environments and is more stable.
A simple DQN-based multi-agent RL system for binary actions.
problem Complexity and training issues in multi-agent reinforcement learning.
method Shared state and rewards, agent-specific actions, experience replay pool.
result Better performance and faster convergence compared to conventional methods.
Multi-agent Q-learning untangles braids, improving over training.
problem Untangling braids using reinforcement learning.
method Two competing players learn to tangle and untangle braids in the OpenAI Gym environment.
result The system improves at untangling braids with more training, producing good tangled examples.
Photonic quantum reinforcement learning for control problems.
problem Solving continuous control problems with noisy quantum computers.
method Proximal policy optimization for photonic variational quantum agents.
result Photonic policy learning achieves comparable performance to classical neural networks.
Paper proposes a new RL approach combining IL and RL methods to improve decision-making.
problem Challenges in RL with large state and action spaces, and difficulty in reward determination.
method Combines Imitation Learning and RL methods (SARSA and A3C) to learn sequential decision-making policies.
result Significantly decreases human effort and exploration time in learning decision-making policies.
Paper robustifies reinforcement learning with risk-averse methods.
problem Making predictions robust to changes in system dynamics or rewards.
method Approximates Robust Reinforcement Learning using -divergence and Risk-Averse formulation.
result Classical Reinforcement Learning can be robustified using standard deviation penalization.
GAN Q-learningstat.ML
Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we p…
FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learningq-fin.TR
FinRL-Meta offers market environments and benchmarks for financial reinforcement learning.
problem Challenges in creating high-quality market environments and benchmarks for financial reinforcement learning.
method DataOps paradigm, automatic pipeline, community-wise competitions, Jupyter/Python demos.
result Openly accessible FinRL-Meta library for data-driven financial reinforcement learning.
Parrot learns optimal cache replacement policies using imitation learning.
problem Improving cache hit rates in complex access patterns.
method Imitation learning approach using Belady's oracle policy.
result Parrot increases cache hit rates by 61% on a web search benchmark.
In this paper, we propose a dual memory structure for reinforcement learning algorithms with replay memory. The dual memory consists of a main memory that stores various data and a cache memory that manages the data and trains the reinforcement learning agent efficiently. Experimental results show that the dual memory …
Investigates Q value evolution in Stable Baselines for DQL in simple vs complex environments.
problem DQL in Stable Baselines struggles with simple non-game environments.
method Comparison of TrafficLight and FrozenLake environments; Q value decomposition analysis.
result Q values meander far from optimal in complex relationships between states.
This project combines recent advances in experience replay techniques, namely, Combined Experience Replay (CER), Prioritized Experience Replay (PER), and Hindsight Experience Replay (HER). We show the results of combinations of these techniques with DDPG and DQN methods. CER always adds the most recent experience to th…
Paper tackles efficient navigation in constrained environments using supervised and reinforcement learning.
problem Building learning agents that efficiently navigate in obstacle-cluttered environments.
method Synergistic use of supervised learning for path prediction and reinforcement learning for path following.
result Proposed method achieves good generalization and faster learning compared to existing work.
A new framework scales active search for large datasets.
problem Scaling active search for large, high-dimensional data sets.
method Hierarchical Batch Bandit Search (HBBS) framework.
result HBBS improves performance and scalability for batch search.
In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised…
Predicting movement of objects while the action of learning agent interacts with the dynamics of the scene still remains a key challenge in robotics. We propose a multi-layer Long Short Term Memory (LSTM) autoendocer network that predicts future frames for a robot navigating in a dynamic environment with moving obstacl…
OtoWorld helps agents learn to navigate by listening in interactive environments.
problem Training agents to navigate using auditory information.
method Interactive environment with agents learning to listen and navigate.
result Agents can win at OtoWorld, a navigation game with sound sources.
Randomly guessing weights helps analyze RL benchmarks objectively.
problem Understanding the complexity of reinforcement learning benchmarks.
method Generate policy networks by randomly guessing their parameters, evaluate on benchmarks, and analyze results.
result Small untrained networks can provide a robust baseline for various RL tasks.
We propose Deep Q-Networks (DQN) with model-based exploration, an algorithm combining both model-free and model-based approaches that explores better and learns environments with sparse rewards more efficiently. DQN is a general-purpose, model-free algorithm and has been proven to perform well in a variety of tasks inc…
This paper proposes an exploration method for deep reinforcement learning based on parameter space noise. Recent studies have experimentally shown that parameter space noise results in better exploration than the commonly used action space noise. Previous methods devised a way to update the diagonal covariance matrix o…
In this paper, the method UCB-RS, which resorts to recommendation system (RS) for enhancing the upper-confidence bound algorithm UCB, is presented. The proposed method is used for dealing with non-stationary and large-state spaces multi-armed bandit problems. The proposed method has been targeted to the problem of the …
The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward …
IDAC improves reinforcement learning efficiency by modeling implicit distributions.
problem Improving sample efficiency in reinforcement learning algorithms.
method IDAC uses two DGNs for a distributional critic and a semi-implicit actor to model implicit policy distributions.
result IDAC outperforms state-of-the-art algorithms on OpenAI Gym environments.
ReF-ER algorithm improved performance in multi-agent reinforcement learning.
problem Improving performance in multi-agent reinforcement learning environments.
method Extended ReF-ER algorithm to include dependencies between agents and modeled environment dynamics.
result ReF-ER MARL outperforms state-of-the-art algorithms in collaborative environments.
We propose a general formulation for addressing reinforcement learning (RL) problems in settings with observational data. That is, we consider the problem of learning good policies solely from historical data in which unobserved factors (confounders) affect both observed actions and rewards. Our formulation allows us t…
In order to satisfy safety conditions, an agent may be constrained from acting freely. A safe controller can be designed a priori if an environment is well understood, but not when learning is employed. In particular, reinforcement learned (RL) controllers require exploration, which can be hazardous in safety critical …
We introduce gym-city, a Reinforcement Learning environment that uses SimCity 1's game engine to simulate an urban environment, wherein agents might seek to optimize one or a combination of any number of city-wide metrics, on gameboards of various sizes. We focus on population, and analyze our agents' ability to genera…
AI agents beat previous best on NetHack, but symbolic bots still outperform.
problem Developing AI agents that can ascend in the NetHack game.
method Used a procedurally generated NetHack Learning Environment for reinforcement learning.
result Symbolic bots outperform neural approaches on NetHack.
In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the ag…
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
GNMC reduces XCSF population size while preserving function approximation and policy accuracy.
problem Population bloat in XCSF compaction.
method Introduced GNMC, a novel compaction algorithm.
result GNMC reduces population size significantly without compromising function approximation or policy accuracy.
In value-based reinforcement learning methods such as deep Q-learning, function approximation errors are known to lead to overestimated value estimates and suboptimal policies. We show that this problem persists in an actor-critic setting and propose novel mechanisms to minimize its effects on both the actor and the cr…
Model-free reinforcement learning based methods such as Proximal Policy Optimization, or Q-learning typically require thousands of interactions with the environment to approximate the optimum controller which may not always be feasible in robotics due to safety and time consumption. Model-based methods such as PILCO or…
RAD enhances RL algorithms with data augmentations.
problem Challenges in RL learning from visual observations.
method RAD is a simple plug-and-play module for RL algorithms.
result RAD improves data-efficiency and final performance.
Simulations of infectious disease spread have long been used to understand how epidemics evolve and how to effectively treat them. However, comparatively little attention has been paid to understanding the fairness implications of different treatment strategies -- that is, how might such strategies distribute the expec…
RL agent learns to place limit orders for trading signals in financial markets.
problem Training an RL agent to execute trading signals in limit order book markets.
method Deep Duelling Double Q-learning with APEX architecture, using synthetic alpha signals.
result RL agent outperforms heuristic trading strategies in inventory management and order placing.
Challenges of Context and Time in Reinforcement Learning: Introducing Space Fortress as a Benchmarkcs.LG
Research in deep reinforcement learning (RL) has coalesced around improving performance on benchmarks like the Arcade Learning Environment. However, these benchmarks conspicuously miss important characteristics like abrupt context-dependent shifts in strategy and temporal sensitivity that are often present in real-worl…