RH-UCRL combines pessimism and optimism for robust RL.
problem Ensuring reliable performance in real-world RL tasks with worst-case scenarios.
method RH-UCRL is a model-based RL algorithm that optimizes between an agent and an adversary, distinguishing between epistemic and aleatoric uncertainty.
result RH-UCRL achieves near-optimal sample complexity guarantees and outperforms other robust RL algorithms in adversarial environments.
Proposes RFQI for robust RL using offline data.
problem Learning robust policies in the presence of model uncertainty.
method RFQI algorithm using offline data to learn optimal robust policy.
result RFQI learns near-optimal robust policy under standard assumptions.
Robust RL improves controller robustness to dynamics variations using adversarial populations.
problem Robustness issues in RL when dynamics are perturbed.
method Adversarial population augmentation to the Robust RL formulation.
result Population-based adversarial approach yields more robust and generalizable policies.
New RL algorithm tackles online robust MDPs with uncertainty.
problem Developing robust reinforcement learning models for real-world environments.
method Proposes a robust optimistic policy optimization algorithm for online robust MDPs.
result Establishes the first regret bound for online robust MDPs.
New RL method improves robustness across various conditions.
problem Training robust RL agents for diverse environments.
method Adversarial training with Langevin Dynamics.
result Consistently outperforms existing RL algorithms in generalization.
This thesis improves practical reinforcement learning methods with robustness, scalability, and efficiency.
problem Improving reinforcement learning methods for practical applications.
method Analyzes and develops robust, scalable, and efficient reinforcement learning algorithms.
result Proves the efficiency and robustness of new RL methods.
RADIAL-RL improves deep RL agents' robustness against adversarial attacks.
problem Vulnerability of deep reinforcement learning agents to small adversarial perturbations.
method RADIAL-RL, a principled framework for training robust reinforcement learning agents.
result RADIAL-RL-trained agents consistently outperform prior methods in robustness tests.
Paper tackles robust offline RL for non-Markovian processes, improving efficiency and applicability.
problem Learning robust policies for non-Markovian decision processes with limited offline data.
method Proposes a novel algorithm with dataset distillation and LCB design for robust values, derived new dual forms, and introduces concentrability coefficients.
result Proves polynomial sample efficiency for finding ε-optimal robust policies.
New algorithms tackle robust RL with linear models, revealing unique challenges.
problem Distributionally robust offline RL with uncertainty in dynamics.
method Proposes minimax optimal and computationally efficient algorithms using novel function approximation mechanisms.
result Function approximation in robust offline RL is distinct and harder than in standard offline RL.
AdaRL improves robust RL by adaptively adjusting policy complexity.
problem Handling epistemic uncertainty in environment dynamics.
method Bi-level optimization framework with adaptive rank adjustment.
result AdaRL outperforms existing methods on MuJoCo benchmarks.
We provide a framework for incorporating robustness -- to perturbations in the transition dynamics which we refer to as model misspecification -- into continuous control Reinforcement Learning (RL) algorithms. We specifically focus on incorporating robustness into a state-of-the-art continuous control RL algorithm call…
Develops a framework for robust RL with dynamic risk measures.
problem Optimal RL strategies depend on risk preferences and model dynamics.
method Dynamic robust distortion risk measures, Wasserstein ball, neural networks, strictly consistent scoring functions, policy gradient formulae, actor-critic algorithm.
result Demonstrates improved performance in portfolio allocation example.
RORL improves offline RL robustness with conservative smoothing.
problem Distribution shift and robustness issues in offline RL.
method RORL introduces regularization and conservative smoothing for robustness.
result RORL achieves state-of-the-art performance and robustness to adversarial perturbations.
This work bridges offline RL and DRL to address distributional shift.
problem Distributional shift in offline RL due to difference in state-action visitation distributions.
method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.
result Demonstrates superior performance of proposed algorithms through simulations.
New RL method tackles sim-to-real gap using interactive data collection.
problem Sim-to-real gap in reinforcement learning.
method Distributionally robust reinforcement learning with interactive data collection.
result Proves sample-efficient learning is impossible without additional assumptions.
New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
Robust RL with learned optimal adversary improves agent performance under adversarial state observations.
problem Ensuring reinforcement learning agents' robustness against adversarial perturbations of state observations.
method Proposed a framework of alternating training with learned adversaries (ATLA) to find optimal adversarial policies and enhance agent robustness.
result ATLA achieves state-of-the-art performance under strong adversaries in continuous control environments.
DR-RPO optimizes robust policies in RL with limited interaction, achieving sublinear regret.
problem Policy optimization in RL under distribution shift and adversarial dynamics.
method DR-RPO algorithm incorporating reference-policy regularization and upper confidence bonus for exploration.
result DR-RPO achieves sublinear regret and polynomial suboptimality bounds in robust RL.
This study tackles adversarial corruption in model-based reinforcement learning.
problem Adversarial corruption in model-based reinforcement learning.
method Maximum likelihood estimation (MLE) approach for learning transition model in both online and offline settings.
result Proves a regret of i l d e O ( T + C ) ilde{\mathcal{O}}(\sqrt{T} + C) i l d e O ( T + C ) for CR-OMLE and a suboptimality of O ( C / n ) \mathcal{O}(C/n) O ( C / n ) for CR-PMLE. New methods tackle robust reinforcement learning in sparse, corrupted data.
problem Tackles robust reinforcement learning in sparse, corrupted data.
method Proposes actor-critic methods with sparse robust estimator oracles.
result First non-vacuous guarantees in high-dimensional sparse MDPs with single-policy concentrability coverage.
Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such cont…
Paper shows RLHF can be solved similarly to standard RL.
problem Difficulty of RLHF compared to standard RL.
method Reduction to reward-based RL techniques.
result RLHF can be solved using existing algorithms for reward-based RL.
Paper introduces a new robust loss function for RL.
problem Heuristic selection of threshold parameters in quantile Huber loss.
method Derived from Wasserstein distance, captures noise in quantile values.
result Enhances robustness against outliers and enables parameter adjustment.
Paper studies S-rectangular DR-RL models for robust reinforcement learning with near-optimal sample complexity.
problem Addressing distributional discrepancies in reinforcement learning environments.
method Empirical value iteration algorithm for divergence-based S-rectangular DR-RL models.
result Near-optimal sample complexity bound of O ( ∣ S ∣ ∣ A ∣ ( 1 − γ ) − 4 ε − 2 ) O(|\mathcal{S}||\mathcal{A}|(1-γ)^{-4}\varepsilon^{-2}) O ( ∣ S ∣∣ A ∣ ( 1 − γ ) − 4 ε − 2 ) . New model-free DR-RL algorithm with finite sample complexity.
problem Limited model-free DR-RL methods with convergence guarantees or sample complexities.
method Integrates Multi-level Monte Carlo (MLMC) technique with threshold mechanism.
result First model-free DR-RL approach with finite sample complexity for total variation and Chi-square divergence.
Deep Reinforcement Learning (RL) demonstrates excellent performance on tasks that can be solved by trained policy. It plays a dominant role among cutting-edge machine learning approaches using multi-layer Neural networks (NNs). At the same time, Deep RL suffers from high sensitivity to noisy, incomplete, and misleading…
Paper tackles robust offline RL with heavy-tailed rewards.
problem Real-world applications often encounter heavy-tailed rewards, challenging offline RL.
method Proposes ROAM and ROOM algorithms using median-of-means method for robust off-policy evaluation and OPO.
result Demonstrates superior performance on heavy-tailed reward datasets compared to existing methods.
RRPI improves offline RL by optimizing policies against worst-case dynamics.
problem Offline RL's performance degrades under distribution shift and transition uncertainty.
method Formulates offline RL as robust policy optimization, treating transition kernel as decision variable.
result RRPI achieves strong average performance on D4RL benchmarks, outperforming recent baselines.
Develops ODRPO to improve RL algorithms with better performance and stability.
problem RL algorithms converge to sub-optimal solutions due to limited policy representation.
method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.
result Achieves globally optimal policy update and higher sample efficiency.
MOOSE improves offline RL robustness by using dynamics models.
problem Low robustness of model-free offline RL algorithms in industrial settings.
method MOOSE uses dynamics models to assess policy performance, keeping policies within data support.
result MOOSE outperforms state-of-the-art model-free offline RL algorithms in robust performance.
Paper tackles robust reinforcement learning with minimal data.
problem Learning robust policies from limited data in uncertain environments.
method Distributionally robust formulation, model-based algorithm combining value iteration and pessimism.
result Proves near-optimal sample complexity for robust offline RL.
Off-policy reinforcement learning (RL) using a fixed offline dataset of logged interactions is an important consideration in real world applications. This paper studies offline RL using the DQN replay dataset comprising the entire replay experience of a DQN agent on 60 Atari 2600 games. We demonstrate that recent off-p…
This paper tackles RL issues with robust policies using historical data.
problem Limited data and mismatch between training and testing environments.
method Distributionally robust offline RL with linear function approximation.
result Achieved error bounds for sample complexity in RL.
Improves reinforcement learning policies for robustness.
problem Lack of robustness in reinforcement learning policies.
method Risk-aware Distributional Reinforcement Learning (SDPG) with CVaR.
result Risk-averse policies achieve robustness against disturbances.
New framework improves sample efficiency and robustness in RL with smooth policies.
problem Sample inefficiency and lack of robustness in deep reinforcement learning.
method SR^2L framework, smoothness-inducing regularization.
result Improved sample efficiency and robustness in both on-policy and off-policy RL algorithms.
This work tackles robust RL in multi-agent settings, improving sample efficiency.
problem Overcoming environmental uncertainties in multi-agent reinforcement learning.
method Proposes DRNVI, a sample-efficient algorithm for learning robust equilibria in RMGs.
result Establishes near-optimal sample complexity for solving RMGs.
This paper analyzes risk-sensitive reinforcement learning with Conditional Value-at-Risk (CVaR) for robust Markov Decision Processes.
problem Risk-sensitive reinforcement learning for robust Markov Decision Processes (RMDPs) with state-action-dependent ambiguity sets.
method The paper establishes a connection between robustness and risk sensitivity, defining a new risk measure NCVaR and proposing value iteration algorithms.
result The proposed approach using NCVaR optimization and value iteration algorithms can solve problems with state-action-dependent ambiguity sets.
This review analyzes RL in finance, highlighting its advantages and challenges.
problem Complex financial decision-making problems where traditional methods fail.
method Systematic review of 167 articles from 2017-2025, focusing on market making, portfolio optimization, and algorithmic trading.
result RL offers advantages over traditional methods, particularly in market making, but challenges remain.
A new estimator improves off-policy evaluation in RL, outperforming existing methods.
problem Estimating performance of a new policy using historical data from a different policy.
method Doubly-robust estimator based on Targeted Maximum Likelihood Estimation, with variance reduction techniques.
result Our estimator uniformly outperforms existing methods across various RL environments and levels of model misspecification.
Enhances RL for jump processes using MSBVE algorithm.
problem Challenges in continuous-time RL with jumps and noise.
method Introduces MSBVE algorithm to minimize quadratic variation error.
result MSBVE algorithm outperforms MSTDE in jump processes.
This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DR…
RL applied to finance tasks, highlighting challenges and future directions.
problem Decision-making tasks in finance using RL.
method Meta-analysis of RL applications, identifying challenges and proposing future directions.
result Challenges in RL performance and future research directions.
Hybrid RL algorithm combines offline and online data for robust and efficient policy learning.
problem Combining robust on-policy methods with efficient offline data for hybrid RL.
method Integrates off-policy training on offline data into on-policy NPG framework.
result Achieves state-of-the-art theoretical guarantees and maintains on-policy NPG guarantees.
RePULSe improves language model alignment by reducing undesired outputs without sacrificing overall performance.
problem Aligning language models with human preferences while minimizing undesired outputs.
method Integrates probabilistic inference into RL training to reduce undesired outputs.
result RePULSe achieves a better balance between expected reward and undesired output probability.
Study uses RL to hedge financial derivatives, showing robust strategies outperform non-robust ones.
problem Risk mitigation and gain-seeking in hedging path-dependent financial derivatives.
method Robust risk-aware reinforcement learning (RL) with policy gradient approach.
result Robust hedging strategies outperform non-robust ones under varying data generating processes.
The dichotomous coordinate descent (DCD) algorithm has been successfully used for significant reduction in the complexity of recursive least squares (RLS) algorithms. In this work, we generalize the application of the DCD algorithm to RLS adaptive filtering in impulsive noise scenarios and derive a unified update formu…
RPI combines imitation and reinforcement learning to improve policies efficiently.
problem High sample complexity in reinforcement learning.
method Active interleaving between imitation and reinforcement learning, using oracle queries for exploration.
result RPI outperforms existing methods across various domains.
Experimentally, it has been observed that humans and animals often make decisions that do not maximize their expected utility, but rather choose outcomes randomly, with probability proportional to expected utility. Probability matching, as this strategy is called, is equivalent to maximum entropy reinforcement learning…