Paper introduces reinforcement learning for managing power grids.
problem Balancing power flows and maintaining grid stability in real-time.
method Reinforcement Learning applied to power network operations.
result Demonstrates feasibility of machine learning in power grid management.
Non-stationary reinforcement learning is challenging due to the complexity of updating value functions.
problem Challenges in non-stationary reinforcement learning, especially in updating value functions.
method Proved a worst-case complexity result for modifying reinforcement learning problems.
result Modifying reinforcement learning problems requires an amount of time almost as large as the number of states.
Deep RL agent secures 2nd place in CityLearn Challenge for district demand management.
problem Optimizing electrical demand of diverse buildings in a district.
method Centralised 'Soft Actor Critic' deep reinforcement learning agent.
result Achieved an averaged score of 0.967 on challenge dataset.
Survey of reinforcement learning for sustainable energy challenges.
problem Sequential decision-making challenges in sustainable energy.
method Reinforcement learning applied to energy production, storage, transmission, and consumption.
result Survey identifies various reinforcement learning themes and potential future directions.
EduGym simplifies reinforcement learning education for students.
problem Students struggle to translate between equations and code in reinforcement learning.
method EduGym provides educational environments and interactive notebooks.
result 86% of students and researchers find EduGym useful for education.
The paper reviews MARL and its causal challenges, advocating for a 'causality first' approach.
problem Challenges in multi-agent reinforcement learning (MARL) and lack of theoretical guarantees for emergent behavior.
method Discussion of how causal methods can improve safety, interpretability, and robustness in MARL.
result Causal methods can provide strong theoretical guarantees for emergent behavior in MARL.
Framework uses expert intervention to solve long-horizon reinforcement learning tasks.
problem Long horizon robot learning tasks with sparse rewards.
method Option templates and expert intervention to enable high-level task understanding.
result Framework outperforms state-of-the-art approaches by two orders of magnitude.
This tutorial reviews offline RL methods and challenges in deep learning.
problem Extracting optimal policies from large datasets without online data.
method Review of existing offline RL algorithms and potential solutions.
result Challenges and open problems in modern deep RL methods.
Survey explores how transfer learning improves deep reinforcement learning.
problem Challenges in reinforcement learning efficiency and effectiveness.
method Categorizes and analyzes transfer learning approaches.
result Transfer learning enhances reinforcement learning performance.
Reinforcement Learning improves insulin bolus decisions for type-I diabetes patients.
problem Optimal insulin bolus decisions for type-I diabetes patients are not well-established.
method Applied Reinforcement Learning to simulated T1DM data.
result Optimal bolus rule differs from standard advisors and can prevent hypoglycemia.
Proposes a pricing agent using reinforcement learning to balance renewable energy demand.
problem Intermittent renewable energy sources challenge carbon-free electricity generation.
method Reinforcement learning approach to balance customer demand with renewable energy generation.
result Demonstrates improved electricity pricing strategy for renewable energy integration.
We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning. We show that our proposed framework conceptually unifies multiple previous methods in exploration. We also derive a practical algorithm that achieves efficien…
RL applied to finance tasks, highlighting challenges and future directions.
problem Decision-making tasks in finance using RL.
method Meta-analysis of RL applications, identifying challenges and proposing future directions.
result Challenges in RL performance and future research directions.
Framework for multi-agent RL with human feedback in a Snake game.
problem Improving multi-agent reinforcement learning with human feedback.
method Developed a simulated game environment for offline model training and online competitions. Introduced HILL methods and reward manipulation heuristics.
result Agents with HILL methods outperform those without in online competitions.
In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks…
Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are often hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. We pres…
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
problem Memory models struggle in partially observable reinforcement learning environments.
method Introduces a novel memory model using the Hadamard product for efficient memory management and updates.
result Significantly outperforms state-of-the-art memory-based methods on challenging benchmarks.
Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and un…
This paper proposes GProp, a deep reinforcement learning algorithm for continuous policies with compatible function approximation. The algorithm is based on two innovations. Firstly, we present a temporal-difference based method for learning the gradient of the value-function. Secondly, we present the deviator-actor-cr…
Meta-algorithm for efficient reinforcement learning from human preferences.
problem Learning from human preference comparisons in Markov decision processes.
method Randomized exploration and experimental design for batch comparison queries.
result Meta-algorithm achieves both regret and last-iterate guarantees with minimal preference queries.
KL-regularized RL from expert demos can lead to slow, unstable learning.
problem Pathological training dynamics in KL-regularized RL from expert demonstrations.
method Empirical analysis and non-parametric behavioral reference policies.
result KL-regularized RL can be significantly improved by using non-parametric behavioral policies.
This work tackles real-world robotic reinforcement learning challenges.
problem Limited success of reinforcement learning in real-world robotics.
method Proposes a system for autonomous real-world learning without instrumentation.
result Demonstrates a complete system that learns without human intervention.
A common challenge in reinforcement learning is how to convert the agent's interactions with an environment into fast and robust learning. For instance, earlier work makes use of domain knowledge to improve existing reinforcement learning algorithms in complex tasks. While promising, previously acquired knowledge is of…
An app-based mHealth intervention uses reinforcement learning to send effective reminder notifications.
problem Designing an effective reinforcement learning algorithm for app-based mHealth interventions.
method Developed a reinforcement learning algorithm to send reminder notifications based on participant likelihood of app engagement.
result The algorithm improved app-based mHealth interventions by reducing participant burden and promoting behavior change.
This work simplifies IRL by using potential-based reward shaping.
problem Computational inefficiency in inverse reinforcement learning.
method Potential-based reward shaping to reduce RL sub-problems.
result Reduces computational burden of inverse reinforcement learning.
ADVISOR dynamically balances imitation and reinforcement learning to overcome the imitation gap.
problem The gap between imitation learning and reinforcement learning when teaching agents have privileged information.
method Adaptive Insubordination (ADVISOR) dynamically weights imitation and reward-based reinforcement learning losses.
result On-the-fly switching with ADVISOR outperforms pure imitation, pure reinforcement learning, and their combinations.
Fiber simplifies RL and population-based methods for distributed training.
problem Challenges in RL and population-based methods, including frequent interaction with simulations and dynamic scaling.
method Introducing Fiber, a scalable distributed computing framework.
result Significantly expands accessibility of large-scale parallel computation.
A green simulation-assisted reinforcement learning method for biomanufacturing.
problem Complexity, high variability, lead time, and limited historical data in biopharmaceutical manufacturing.
method Quantifies model risk, uses posterior distribution, and selectively reuses simulation data.
result Demonstrates promising performance in online learning and decision making.
This work simplifies choosing reinforcement-learning algorithms.
problem Challenging selection of reinforcement-learning algorithms.
method Structured overview and guidelines for choosing methods.
result Provides structured guidelines for algorithm selection.
The design of a reward function often poses a major practical challenge to real-world applications of reinforcement learning. Approaches such as inverse reinforcement learning attempt to overcome this challenge, but require expert demonstrations, which can be difficult or expensive to obtain in practice. We propose var…
On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central …
Paper tackles reinforcement learning generalization through invariant policy optimization.
problem Learning policies that generalize beyond training domains.
method Invariant policy optimization principle and novel learning algorithm IPO.
result Significant improvements in generalization performance on unseen domains.
This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compa…
This research develops an efficient reinforcement learning method for undercomplete POMDPs.
problem Learning undercomplete Partially Observable Markov Decision Processes (POMDPs) is computationally hard.
method OOM-UCB algorithm for episodic finite undercomplete POMDPs.
result Achieves optimal sample complexity of ildeO(1/ε2) for finding an ε-optimal policy. New RL method tackles dynamic MDPs with evolving rewards and states.
problem Dynamic MDPs with evolving rewards and states.
method Sliding Window Upper-Confidence bound for Reinforcement Learning (SWUCRL2-CW) and Bandit-over-Reinforcement Learning (BORL).
result Achieves dynamic regret bound for non-stationary MDPs.
Deep network learns Obstacle Tower challenge without human demonstrations.
problem Master procedurally generated levels that get progressively harder.
method Deep Reinforcement Learning with a simple feed-forward network.
result Performed competitively in a reinforcement learning competition.
Success stories of applied machine learning can be traced back to the datasets and environments that were put forward as challenges for the community. The challenge that the community sets as a benchmark is usually the challenge that the community eventually solves. The ultimate challenge of reinforcement learning rese…
New framework promotes reproducible, domain-agnostic reinforcement learning algorithms.
problem Domain-specific, compute-resource-maximizing, and non-reproducible participant solutions in reinforcement learning competitions.
method Submission retraining, domain randomization, desemantization through domain obfuscation, and compute/environment-sample budget limitation.
result Participant submissions are reproducible, non-specific to the competition environment, and sample/resource efficient.
IntelligentPooling improves treatment decisions in mHealth.
problem Optimizing treatment decisions in mobile health with limited data and non-stationary responses.
method Generalized Thompson-Sampling bandit algorithms to IntelligentPooling, addressing differential response, limited data, and non-stationary responses.
result IntelligentPooling achieves 26% lower regret compared to state-of-the-art methods.
Automatically finds effective security strategies through reinforcement learning and self-play.
problem Finding effective security strategies for intrusion prevention.
method Modeling interaction as a Markov game, evolving attack and defense strategies through reinforcement learning and self-play.
result Effective security strategies emerge from self-play, reflecting common-sense knowledge.
Study shows challenges in reinforcement learning math problems, proposing enhancements and a hardness measure.
problem Challenges in reinforcement learning finding rare high-reward instances.
method Combining combinatorial group theory, algorithmic enhancements, and topological hardness measure.
result Resolved mathematical questions and proposed enhancements for reinforcement learning.
This review analyzes RL in finance, highlighting its advantages and challenges.
problem Complex financial decision-making problems where traditional methods fail.
method Systematic review of 167 articles from 2017-2025, focusing on market making, portfolio optimization, and algorithmic trading.
result RL offers advantages over traditional methods, particularly in market making, but challenges remain.
Paper uses RL to optimize bid-ask spreads for diverse options.
problem Optimizing bid-ask spreads for options with various maturities and strikes.
method Combines stochastic policy with reinforcement learning.
result Proposes an effective approach for market making of options.
We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…
QTRAN++ improves MARL performance in complex environments.
problem Poor empirical performance of QTRAN in complex environments.
method Stabilizing training objective, removing role separation, and introducing a multi-head mixing network.
result QTRAN++ achieves state-of-the-art performance in the Starcraft Multi-Agent Challenge (SMAC).
Autonomous lane changing is a critical feature for advanced autonomous driving systems, that involves several challenges such as uncertainty in other driver's behaviors and the trade-off between safety and agility. In this work, we develop a novel simulation environment that emulates these challenges and train a deep r…
Plan2Explore learns new tasks efficiently through self-supervised planning.
problem Challenges in reinforcement learning, especially task-specific learning and sample efficiency.
method Self-supervised exploration and fast adaptation to new tasks through efficient planning.
result Plan2Explore outperforms prior methods in learning new tasks without supervision.
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…