Paper robustifies reinforcement learning with risk-averse methods.
problem Making predictions robust to changes in system dynamics or rewards.
method Approximates Robust Reinforcement Learning using Φ Φ Φ -divergence and Risk-Averse formulation. result Classical Reinforcement Learning can be robustified using standard deviation penalization.
Deep RL for portfolio management shows poor robustness.
problem Robustness of Deep RL algorithms in online portfolio management.
method Proposed a training and evaluation process for assessing DRL algorithms.
result Most Deep RL algorithms are not robust, generalizing poorly and degrading quickly.
Safe reinforcement learning framework using optimal transport for robustness.
problem Robustness and safety in deep reinforcement learning with limited data assumptions.
method Optimal transport perturbations to construct worst-case virtual state transitions.
result Significantly improved safety at deployment time compared to standard methods.
A framework for robust exploration in reinforcement learning under ambiguity.
problem Optimal stopping under ambiguity in reinforcement learning.
method Continuous-time robust reinforcement learning framework using g g g -expectation and backward stochastic differential equations. result Constructs a robust exploratory stopping time approximating the optimal stopping time under ambiguity.
New approach connects reinforcement learning robustness and regularization.
problem Dealing with external uncertainty in reinforcement learning.
method Introducing Wasserstein distributionally robust MDPs and a new regularizer.
result Established a dual relation between robust MDPs and regularization.
Paper develops robust policy evaluation for reinforcement learning with outlier and heavy-tailed rewards.
problem Outlier contamination and heavy-tailed rewards in reinforcement learning.
method Develops a fully online robust policy evaluation procedure and efficient statistical inference.
result Establishes the Bahadur-type representation of the estimator and develops an online inference procedure.
Improves reinforcement learning policies for robustness.
problem Lack of robustness in reinforcement learning policies.
method Risk-aware Distributional Reinforcement Learning (SDPG) with CVaR.
result Risk-averse policies achieve robustness against disturbances.
Risk-averse model uncertainty framework for safe reinforcement learning.
problem Safe decision making in uncertain environments.
method Risk-averse perspective towards model uncertainty using coherent distortion risk measures; equivalent to distributionally robust safe reinforcement learning problems; efficient, model-free implementation.
result Demonstrates robust performance and safety across perturbed test environments.
RORL improves offline RL robustness with conservative smoothing.
problem Distribution shift and robustness issues in offline RL.
method RORL introduces regularization and conservative smoothing for robustness.
result RORL achieves state-of-the-art performance and robustness to adversarial perturbations.
Deep RL policies are vulnerable to adversarial perturbations, but vanilla training yields more robust policies.
problem Vulnerability of deep reinforcement learning policies to adversarial perturbations.
method Analysis of deep reinforcement learning policy landscape and comparison of vanilla vs. adversarial training.
result Vanilla training yields more robust policies compared to adversarial training.
WRAAC uses Wasserstein distance for robust reinforcement learning.
problem Lack of quantified robustness to system dynamics in existing reinforcement learning algorithms.
method Leverages Wasserstein distance to connect state disturbance to transition kernel disturbance, reducing infinite-dimensional optimization to a finite-dimensional problem.
result Designs a novel algorithm, WRAAC, that achieves robust reinforcement learning.
Reinforcement learning algorithms, though successful, tend to over-fit to training environments hampering their application to the real-world. This paper proposes W R 2 L \text{W}\text{R}^{2}\text{L} W R 2 L -- a robust reinforcement learning algorithm with significant robust performance on low and high-dimensional control tasks. Ou…
This work proposes robust reinforcement learning methods using both offline and online data.
problem Designing robust policies against parameter uncertainties in high-dimensional systems.
method Proposes RPQ for model-free learning with historical data and HyTQ for hybrid learning with both historical and online data.
result Unified analysis and theoretical guarantees for robust optimal policies in high-dimensional systems.
The paper tackles robust reinforcement learning with performance guarantees.
problem Finding a robust policy for RMDP with state space uncertainties.
method Proposes RLSPI algorithm for learning optimal robust policy with performance bounds.
result Demonstrates the performance of RLSPI on standard benchmark problems.
To improve efficiency and reduce failures in autonomous vehicles, research has focused on developing robust and safe learning methods that take into account disturbances in the environment. Existing literature in robust reinforcement learning poses the learning problem as a two player game between the autonomous system…
State entropy regularization improves robustness in reinforcement learning, especially under structured perturbations.
problem Structured and spatially correlated perturbations in reinforcement learning.
method State entropy regularization, compared to policy entropy.
result State entropy regularization provides better robustness to structured and spatially correlated perturbations.
This paper analyzes generalization issues in deep reinforcement learning.
problem Understanding and improving generalization capabilities of deep reinforcement learning policies.
method Formalizing and categorizing solutions to address overfitting in deep reinforcement learning.
result A comprehensive analysis of generalization challenges and solutions in deep reinforcement learning.
New algorithms optimize a soft-robust criterion in reinforcement learning, reducing conservatism.
problem Computing robust policies for high-stakes decisions with limited data.
method Soft-robust criterion using risk measures, two algorithms for optimization.
result Our algorithms produce less conservative solutions than existing methods.
A method for robust reinforcement learning in large state spaces.
problem Challenges in RL with large state spaces, costly data, and real-world dynamics deviation.
method Distributionally robust Markov decision processes with Gaussian Processes and maximum variance reduction.
result Efficient learning of multi-output nominal transition dynamics with statistical sample complexity bounds.
This thesis improves practical reinforcement learning methods with robustness, scalability, and efficiency.
problem Improving reinforcement learning methods for practical applications.
method Analyzes and develops robust, scalable, and efficient reinforcement learning algorithms.
result Proves the efficiency and robustness of new RL methods.
RH-UCRL combines pessimism and optimism for robust RL.
problem Ensuring reliable performance in real-world RL tasks with worst-case scenarios.
method RH-UCRL is a model-based RL algorithm that optimizes between an agent and an adversary, distinguishing between epistemic and aleatoric uncertainty.
result RH-UCRL achieves near-optimal sample complexity guarantees and outperforms other robust RL algorithms in adversarial environments.
New algorithms improve robust reinforcement learning under uncertainty.
problem Robust reinforcement learning in MDPs with contamination.
method Non-asymptotic convergence analysis of Q Q Q -learning and actor-critic methods. result Efficient algorithms learn robust policies with minimal samples.
A new reinforcement learning method reduces action complexity for robust control.
problem Deep reinforcement learning's susceptibility to spurious correlations.
method Minimizing trajectory entropy to encourage simple, predictable actions.
result Trajectory Entropy Reinforcement Learning achieves superior performance and robustness.
New method reduces sample complexity for robust reinforcement learning.
problem Finite sample analysis in robust reinforcement learning.
method Stochastic approximation framework with controlled bias, using MLMC techniques and geometric truncation.
result Order-optimal sample complexity of i l d e O ( ε − 2 ) ilde{\mathcal{O}}(ε^{-2}) i l d e O ( ε − 2 ) for robust policy evaluation. RADIAL-RL improves deep RL agents' robustness against adversarial attacks.
problem Vulnerability of deep reinforcement learning agents to small adversarial perturbations.
method RADIAL-RL, a principled framework for training robust reinforcement learning agents.
result RADIAL-RL-trained agents consistently outperform prior methods in robustness tests.
We study reinforcement learning under model misspecification, where we do not have access to the true environment but only to a reasonably close approximation to it. We address this problem by extending the framework of robust MDPs to the model-free Reinforcement Learning setting, where we do not have access to the mod…
New algorithm reduces performance loss in IRL with mismatched transition dynamics.
problem Performance degradation in inverse reinforcement learning due to mismatched transition dynamics.
method Proposed a robust Maximum Causal Entropy (MCE) IRL algorithm leveraging robust reinforcement learning insights.
result Empirically demonstrated stable performance improvement under transition dynamics mismatches.
New algorithm reduces sample size for robust reinforcement learning.
problem Creating robust policies for multi-agent reinforcement learning.
method Model-based algorithm RTZ-VI-LCB for tabular robust two-player zero-sum games.
result Establishes near-optimal sample complexity guarantees for offline robust reinforcement learning.
Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.
problem Lack of robustness and stability in adaptation for traffic signal control.
method Value-based Bayesian meta-reinforcement learning framework BM-DQN with fast-adaptation variation and DQN fast-update advantage.
result Framework adapts more quickly and robustly to new scenarios than previous methods.
DARL uses DDPMs to generate synthetic market crash scenarios for robust portfolio optimization.
problem Challenges in capturing complex market dynamics and aligning with diverse investor preferences.
method Synergistic integration of DDPMs and DRL for portfolio management.
result DARL outperforms traditional methods in delivering superior risk-adjusted returns and resilience against crises.
Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may …
Proposes RFQI for robust RL using offline data.
problem Learning robust policies in the presence of model uncertainty.
method RFQI algorithm using offline data to learn optimal robust policy.
result RFQI learns near-optimal robust policy under standard assumptions.
Reinforcement Learning optimizes low-thrust interplanetary trajectories under disturbances.
problem Designing robust interplanetary trajectories in the presence of disturbances.
method Reformulated as a Markov Decision Process, RL algorithm Proximal Policy Optimization trained on a deep neural network.
result Deep neural network provides robust nominal trajectory and guidance law.
Robust RL with learned optimal adversary improves agent performance under adversarial state observations.
problem Ensuring reinforcement learning agents' robustness against adversarial perturbations of state observations.
method Proposed a framework of alternating training with learned adversaries (ATLA) to find optimal adversarial policies and enhance agent robustness.
result ATLA achieves state-of-the-art performance under strong adversaries in continuous control environments.
This work extends HiP-MDPs to robust state abstractions for multi-task and meta-reinforcement learning.
problem Limited observability of state in HiP-MDPs for real-world scenarios with rich observation spaces.
method Inspired by Block MDPs, the work extends HiP-MDPs to enable robust state abstractions for multi-task and meta-reinforcement learning.
result Transfer and generalization bounds based on task and state similarity, and sample complexity bounds that depend on the aggregate number of samples across tasks.
In deep reinforcement learning (RL), adversarial attacks can trick an agent into unwanted states and disrupt training. We propose a system called Robust Student-DQN (RS-DQN), which permits online robustness training alongside Q networks, while preserving competitive performance. We show that RS-DQN can be combined with…
This paper analyzes risk-sensitive reinforcement learning with Conditional Value-at-Risk (CVaR) for robust Markov Decision Processes.
problem Risk-sensitive reinforcement learning for robust Markov Decision Processes (RMDPs) with state-action-dependent ambiguity sets.
method The paper establishes a connection between robustness and risk sensitivity, defining a new risk measure NCVaR and proposing value iteration algorithms.
result The proposed approach using NCVaR optimization and value iteration algorithms can solve problems with state-action-dependent ambiguity sets.
New framework for robust reinforcement learning policies in uncertain environments.
problem Robust reinforcement learning policies in environments with distributional shifts.
method Comprehensive modeling framework centered around robust Markov decision processes (RMDPs).
result Existence and conditions for the dynamic programming principle (DPP) in RMDPs.
Improved reinforcement learning for environments with distributional shifts.
problem Learning optimal policies in environments with distributional shifts.
method Distributionally robust Q-learning with multi-level Monte Carlo estimator.
result Proved upper bound on sample complexity for robust RL.
New algorithm improves RL performance across different environments.
problem Improving reinforcement learning performance across various environments.
method Designing a fully model-free DRRL algorithm that learns from a single trajectory.
result Demonstrates superior robustness and sample efficiency compared to existing methods.
Paper tackles robust reinforcement learning with minimal data.
problem Learning robust policies from limited data in uncertain environments.
method Distributionally robust formulation, model-based algorithm combining value iteration and pessimism.
result Proves near-optimal sample complexity for robust offline RL.
A new method learns robust policies from offline data with latent structures.
problem Conservative policies under unrealistic dynamics shifts.
method d-RRMDP framework with f f f -divergence regularization and R2PVI algorithm. result R2PVI learns robust policies with superior computational efficiency.
This paper achieves first-order regret bounds in reinforcement learning with large state spaces.
problem Achieving first-order regret bounds in reinforcement learning with large state spaces.
method Developed a novel robust self-normalized concentration bound based on the robust Catoni mean estimator.
result Obtained regret bounds scaling as O ~ ( d 3 H 3 ⋅ V 1 ⋆ ⋅ K + d 3.5 H 3 log K ) \widetilde{\mathcal{O}}(\sqrt{d^3 H^3 \cdot V_1^\star \cdot K} + d^{3.5}H^3\log K ) O ( d 3 H 3 ⋅ V 1 ⋆ ⋅ K + d 3.5 H 3 log K ) . Deep RL optimizes goal-based investing strategies.
problem Optimizing investment strategies for achieving financial goals.
method Novel deep reinforcement learning approach for goal-based investing.
result Superior performance compared to benchmarks.
Enhanced reinforcement learning using ensemble methods.
problem Improving reinforcement learning performance.
method Distributional reinforcement learning with ensemble group-aided training.
result Ensemble methods lead to more robust and efficient learning.
Develops a framework for robust RL with dynamic risk measures.
problem Optimal RL strategies depend on risk preferences and model dynamics.
method Dynamic robust distortion risk measures, Wasserstein ball, neural networks, strictly consistent scoring functions, policy gradient formulae, actor-critic algorithm.
result Demonstrates improved performance in portfolio allocation example.
Robust RL improves controller robustness to dynamics variations using adversarial populations.
problem Robustness issues in RL when dynamics are perturbed.
method Adversarial population augmentation to the Robust RL formulation.
result Population-based adversarial approach yields more robust and generalizable policies.
New algorithm identifies near-optimal policies in adversarial distributed RL settings.
problem Adversarial agents in distributed RL settings that can collude and report arbitrary data.
method Weighted-Clique algorithm for robust mean estimation from batches, combined with novel distributed algorithms.
result Achieves superior robustness guarantees and near-optimal sample complexities in both offline and online settings.