DisCoRL combines state learning and policy distillation for continual reinforcement learning.
problem Learning multiple policies with a single model and inferring the correct policy at test time.
method Combining state representation learning and policy distillation.
result The policy can solve all tasks and automatically infer the correct one to run.
This paper improves reinforcement learning policies in a scalable way.
problem Ensuring monotonic policy improvement in entropy-regularized RL.
method Derives an entropy-aware lower bound and proposes a novel RL algorithm.
result Demonstrates effectiveness in continuous-state tasks using a linear function approximator.
Researchers develop methods to protect reinforcement learning and control systems from policy poisoning attacks.
problem Attacks on reinforcement learning and control systems that manipulate learned policies.
method Unified framework for solving policy poisoning attacks, demonstrating global optimality and feasibility.
result Policy poisoning attacks are feasible and can be defended with a convex optimization approach.
We introduce Implicit Policy, a general class of expressive policies that can flexibly represent complex action distributions in reinforcement learning, with efficient algorithms to compute entropy regularized policy gradients. We empirically show that, despite its simplicity in implementation, entropy regularization c…
Paper develops a new multi-agent reinforcement learning algorithm.
problem Improving policies in a network of communicating agents.
method Develops a multi-agent off-policy actor-critic algorithm using emphatic temporal difference learning.
result Proves convergence of the algorithm under linear function approximation.
New reinforcement learning algorithms improve policy optimization with entropy regularization.
problem Improving policy optimization in reinforcement learning.
method Soft policy gradient theorem (SPGT) and new policy optimization algorithms.
result New algorithms outperform prior works on various benchmark tasks.
Deep RL policies are vulnerable to adversarial perturbations, but vanilla training yields more robust policies.
problem Vulnerability of deep reinforcement learning policies to adversarial perturbations.
method Analysis of deep reinforcement learning policy landscape and comparison of vanilla vs. adversarial training.
result Vanilla training yields more robust policies compared to adversarial training.
This paper analyzes generalization issues in deep reinforcement learning.
problem Understanding and improving generalization capabilities of deep reinforcement learning policies.
method Formalizing and categorizing solutions to address overfitting in deep reinforcement learning.
result A comprehensive analysis of generalization challenges and solutions in deep reinforcement learning.
Paper develops robust policy evaluation for reinforcement learning with outlier and heavy-tailed rewards.
problem Outlier contamination and heavy-tailed rewards in reinforcement learning.
method Develops a fully online robust policy evaluation procedure and efficient statistical inference.
result Establishes the Bahadur-type representation of the estimator and develops an online inference procedure.
The paper proposes an iterative approach to batch reinforcement learning for safer and more informative data collection.
problem Learning policies that are too rigid and do not adapt to new data.
method Safe diversified model-based policy search in an iterative batch reinforcement learning framework.
result Improved learned policies through continuous data collection and adaptation.
Combines soft greediness with Modified Policy Iteration for more efficient deep reinforcement learning.
problem Improving sample efficiency in deep reinforcement learning.
method Combines soft greediness with Modified Policy Iteration (MPI) for off-policy learning.
result The proposed algorithm is more sample efficient than original PPO and competitive with SAC.
The paper introduces a new framework for off-policy reinforcement learning.
problem Improving off-policy reinforcement learning algorithms.
method Conceptual framework based on conditional importance sampling.
result Theoretical analysis and concrete algorithm investigation.
Integrates multiple feedback channels into policy learning for reinforcement learning.
problem Combining multiple types of feedback into policy gradient algorithms.
method Lagrangian relaxation to satisfy constraints using gradient descent while maximizing rewards.
result Constraints are respected and can accelerate learning in reinforcement learning tasks.
Taylor expansions improve reinforcement learning policies.
problem Improving reinforcement learning policy optimization.
method Taylor expansion policy optimization.
result Taylor expansions enhance performance of distributed algorithms.
Improves imitation learning in RL by learning reward function efficiently.
problem Lack of effective reward function approximation in AIRL for imitation tasks.
method Proposes Off-Policy AIRL that combines adversarial learning with efficient reward function approximation.
result Shows superior imitation performance and efficiency compared to state-of-the-art AIL algorithms.
Paper proposes a framework for reliable off-policy evaluation in reinforcement learning.
problem Quantifying uncertainty in off-policy estimates for safe deployment of target policies.
method Distributionally robust optimization for creating confidence bounds.
result Non-asymptotic and asymptotic guarantees for robust cumulative reward estimates.
In some reinforcement learning problems an agent may be provided with a set of input policies, perhaps learned from prior experience or provided by advisors. We present a reinforcement learning with policy advice (RLPA) algorithm which leverages this input set and learns to use the best policy in the set for the reinfo…
Algorithm improves reinforcement learning policies using offline data.
problem Improving reinforcement learning policies with limited online data.
method Designs a single non-reactive policy using offline data with provable guarantees.
result Algorithm achieves better policy quality with less online data.
Entropy regularization improves policy optimization in reinforcement learning.
problem Improving policy optimization in reinforcement learning.
method Entropy regularization is introduced to soften the greedy policy towards a more diverse softmax policy, leading to a continuously parameterized algorithm that interpolates between policy gradient and Q-learning.
result An intermediate algorithm can improve performance in reinforcement learning.
Distributed learning method for multi-agent reinforcement learning with policy coordination.
problem Solving multi-agent reinforcement learning problems with coordination.
method Distributed off-policy actor critic with policy consensus.
result The proposed algorithm achieves asymptotic agreement on the global optimal policy function.
Improves reinforcement learning policies for robustness.
problem Lack of robustness in reinforcement learning policies.
method Risk-aware Distributional Reinforcement Learning (SDPG) with CVaR.
result Risk-averse policies achieve robustness against disturbances.
CODA resolves coordination issues in offline multi-agent reinforcement learning.
problem Coordination failure in offline multi-agent reinforcement learning.
method Diffusion-based multi-agent trajectory generator for data augmentation.
result CODA resolves coordination pathologies in continuous polynomial games and complex benchmarks.
A new framework estimates expert policy support to create a reward function for imitation learning.
problem Imitation learning from expert trajectories without reinforcement signals.
method Estimating the support of the expert policy to compute a fixed reward function.
result Comparable or better performance than state-of-the-art methods on discrete and continuous domains.
Paper tackles reinforcement learning generalization through invariant policy optimization.
problem Learning policies that generalize beyond training domains.
method Invariant policy optimization principle and novel learning algorithm IPO.
result Significant improvements in generalization performance on unseen domains.
Method solves long-horizon robotic tasks via imitation and reinforcement learning.
problem Long-horizon robotic tasks with complex sequences of actions.
method Two-phase approach: imitation learning followed by reinforcement learning.
result Method can scale to challenging long-horizon tasks.
Real-time policy distillation speeds up and improves reinforcement learning.
problem Slow and inefficient policy distillation in reinforcement learning.
method Simultaneous training and distillation of a teacher and student model.
result Significantly reduced distillation time and improved small model performance.
VPP learns joint policies for multi-agent RL through interactions.
problem Learning effective joint policies for multi-agent reinforcement learning.
method VPP integrates variational inference into policy layers for efficient sampling and differentiability.
result VPP outperforms previous methods on large-scale multi-agent tasks.
A model improves continual learning in reinforcement learning without forgetting.
problem Catastrophic forgetting in deep reinforcement learning.
method Policy consolidation model interacting with a cascade of hidden networks.
result Improves continual learning on various continuous control tasks.
The paper tackles robust reinforcement learning with performance guarantees.
problem Finding a robust policy for RMDP with state space uncertainties.
method Proposes RLSPI algorithm for learning optimal robust policy with performance bounds.
result Demonstrates the performance of RLSPI on standard benchmark problems.
This paper optimizes attacks on reinforcement learning policies, reducing their effectiveness.
problem Optimizing adversarial attacks on reinforcement learning policies to minimize rewards.
method Designing optimal attacks for both white-box and black-box scenarios using Markov Decision Processes and Reinforcement Learning.
result Optimal attacks can reduce the effectiveness of reinforcement learning policies, especially for smooth policies.
KL-regularized RL from expert demos can lead to slow, unstable learning.
problem Pathological training dynamics in KL-regularized RL from expert demonstrations.
method Empirical analysis and non-parametric behavioral reference policies.
result KL-regularized RL can be significantly improved by using non-parametric behavioral policies.
Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demonstrate that due to errors introduced by extrapolation, standard off-policy deep reinforcement learn…
Paper proposes Cycle-of-Learning framework for better reinforcement learning performance.
problem Efficiently updating policies trained with demonstrations using reinforcement learning.
method Cycle-of-Learning framework combining behavior cloning and 1-step Q-learning losses.
result Cycle-of-Learning framework improves reinforcement learning performance in dense and sparse reward scenarios.
Deep RL policies share adversarial features across different MDPs.
problem Understanding decision boundaries and loss landscapes in neural policies.
method Investigating similarities in high sensitivity directions across MDPs using Arcade Learning Environment.
result High sensitivity directions for neural policies are correlated across MDPs, suggesting shared non-robust features.
This paper investigates how policy conditioning affects reinforcement learning stability.
problem Improving stability and generalization of reinforcement learning agents.
method The authors study Jacobian conditioning behavior during policy optimization and propose a conditioning regularization algorithm.
result The proposed conditioning regularization algorithm enhances reinforcement learning agent generalization.
SafePILCO is a Python tool for safe reinforcement learning.
problem Safe and efficient policy synthesis in reinforcement learning.
method Extends PILCO algorithm with safety features, implemented in Python.
result Safe and data-efficient policy synthesis achieved.
Unified treatment of reinforcement learning via convex duality.
problem Applying Fenchel-Rockafellar duality to reinforcement learning.
method Unified derivation of various RL settings using convex duality.
result Ability to perform policy evaluation and on-policy policy gradient with offline data.
Paper clusters event sequences using a reinforcement learning approach with policy mixture model.
problem Clustering event sequences with varying temporal patterns.
method Reinforcement learning with a policy mixture model, decomposing sequences into states and actions.
result Effective clustering of event sequences into underlying policies, outperforming existing methods.
A lifelong learning architecture improves reinforcement learning policies using simulations and a DNC model.
problem Improving reinforcement learning policies in dynamic environments.
method Iterative training of a Reinforcement Learning agent and a DNC model in conjunction.
result DNC models can continually learn from pixels alone to simulate new tasks.
A new method learns policies from sub-optimal trajectories.
problem Reinforcement learning's brittleness and difficulty in tuning.
method Conditioning policies on the reward of sub-optimal trajectories.
result Effective policies can be learned without expert demonstrations.
New method improves reinforcement learning generalization.
problem Few environments lead to poor generalization in reinforcement learning.
method Integrates sequential structure into representation learning, using a policy similarity metric (PSM) and contrastive embeddings (PSEs).
result PSEs improve generalization across various benchmarks.
Quantum algorithms speed up reinforcement learning policies in large state-action spaces.
problem Limitations of quantum access in training reinforcement learning policies.
method Designing quantum algorithms to train reinforcement learning policies.
result Quantum algorithms offer full quadratic speed-ups in sample complexity for well-behaved policies.
Meta-reinforcement learning improves fault-adaptive control efficiency.
problem Adaptive control under abrupt system faults with strict time constraints.
method Model-agnostic meta learning (MAML) with a fault library of prior policies.
result Improved sample efficiency and quick adaptation to new faults.
Safe reinforcement learning with nonconvex constraints using convex approximations.
problem Safe reinforcement learning with nonlinear function approximation.
method Constructing surrogate convex constrained optimization problems by replacing nonconvex functions with convex quadratic functions.
result Solutions to surrogate problems converge to a stationary point of the original nonconvex problem.
New algorithms improve policy evaluation in reinforcement learning.
problem Off-policy stability and on-policy efficiency issues in policy evaluation.
method Introduced novel algorithms using oblique projection method.
result Demonstrated both off-policy stability and on-policy efficiency.
Introspection Learning asks policies introspective questions to improve learning.
problem Improving reinforcement learning efficiency and robustness.
method Synthesizes experience through direct policy introspection without interaction.
result Demonstrates improved training speed and robustness.
A new algorithm learns diverse policies in reinforcement learning.
problem Learning diverse behaviors in reinforcement learning.
method Proposes Maximum Entropy Diverse Exploration (MEDE) algorithm.
result The set of policies learned by MEDE capture the same modalities as the optimal maximum entropy policy.
A new imitation learning method uses random search for simple policies, outperforming complex models.
problem Complexity and reward dependency issues in imitation learning.
method Derivative-free optimization with simple linear policies and random search.
result The proposed method achieves competitive performance on MuJoCo locomotion tasks without a direct reward signal.