A new method approximates Laplacian eigenvectors for RL efficiently.
problem Efficiently learning state representations in RL.
method General and scalable approach to approximating Laplacian eigenvectors.
result Empirically shows improved performance in RL tasks.
Algorithm learns goals without rewards, controls environments.
problem Learning control without supervised rewards.
method Dual optimization of policy and reward function.
result Agent learns to achieve goals in diverse domains.
Improved cooperation between levels boosts reinforcement learning performance.
problem Training multi-level policies in hierarchical reinforcement learning.
method Modeling policy optimization as a multi-agent process and inducing cooperation between sub-policies.
result Inducing cooperation between sub-policies leads to stronger and more sample-efficient policies.
Paper presents efficient AL strategies using influence functions.
problem High computational costs in AL strategies.
method Influence functions for efficient model retraining approximation.
result Makes AL strategies applicable in practice.
Paper proposes Imitative Models combining IL and planning for flexible goal achievement.
problem Difficult to direct IL to arbitrary goals and specify reward functions for goal-directed planning.
method Imitative Models are probabilistic predictive models that plan interpretable expert-like trajectories.
result Imitative Models outperform IL and planning approaches in a dynamic autonomous driving task.
Hybrid RL algorithms improve offline and online RL in linear MDPs.
problem Improving RL performance without single-policy concentrability.
method Developed computationally efficient algorithms for PAC and regret-minimizing RL in linear MDPs.
result Achieved sharper error or regret bounds for linear MDPs.
Unified framework for planning under uncertainty using variational inference.
problem Planning under uncertainty with separate objectives for exploration and exploitation.
method Variational inference on a generative model augmented with priors.
result EFE-based planning emerges as variational inference, enabling scalable, resource-aware policies.
This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.
problem Investigating the representation complexity gap among model-based, policy-based, and value-based RL.
method Demonstrated through analysis of Markov decision processes (MDPs) and introduced new classes of MDPs.
result Representation complexity hierarchy: model-based RL > policy-based RL > value-based RL.
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
Paper shows RLHF can be solved similarly to standard RL.
problem Difficulty of RLHF compared to standard RL.
method Reduction to reward-based RL techniques.
result RLHF can be solved using existing algorithms for reward-based RL.
MaxEnt RL optimizes decision-making in uncertain environments.
problem MaxEnt RL optimizes decision-making in uncertain environments.
method Formal analysis of MaxEnt RL in POMDPs and adversarial reward games.
result MaxEnt RL optimally solves certain classes of control problems with uncertain reward functions.
RL tackles decision making in unknown environments, focusing on efficiency and efficacy.
problem Efficiency and efficacy in RL algorithms for sample-starved situations.
method Markov Decision Processes, model-based and value-based approaches, policy optimization.
result Enhanced understanding and improvements in sample and computational efficacies of RL algorithms.
Catalyst.RL accelerates RL research with efficient training.
problem Efficient reinforcement learning training in complex environments.
method Open-source PyTorch framework with distributed training and RL algorithms.
result Catalyst.RL achieved 2nd place in a computationally expensive RL challenge.
Study shows effectiveness of offline RL in online RL tasks.
problem Improving online RL efficiency using offline RL data.
method Formalized framework for incorporating offline RL as online RL subroutines, introducing techniques to enhance effectiveness.
result Effectiveness of the framework depends on task nature, techniques greatly enhance effectiveness, and existing methods are ineffective.
MOReL learns offline RL policies using pessimistic MDPs.
problem Offline RL's data efficiency and velocity.
method Two-step process: learn P-MDP and near-optimal policy in it.
result MOReL is minimax optimal and matches state-of-the-art results.
catalyst.RL simplifies RL research, enabling reproducibility and efficiency.
problem Difficulty in reproducing and comparing RL algorithms.
method Open-source framework with distributed training, flexible configurations, and efficient RL algorithms.
result Demonstrated effectiveness on AI for Prosthetics Challenge, achieving 3rd place.
This paper compares expected and distributional reinforcement learning methods.
problem Understanding why distributional reinforcement learning performs better than expected reinforcement learning.
method Analyzes differences in tabular, linear, and non-linear approximation settings.
result Distributional RL can hurt performance if it does not induce identical behavior.
RL research overhypes potential but lacks deployable solutions.
problem Current RL research overhypes potential and lacks deployable solutions.
method Identifies and critiques current RL research practices.
result Current RL research direction is unlikely to lead to practical, economically viable solutions.
New method makes deep RL agents more understandable.
problem Incomprehensible decision-making in NN-based RL agents limits their applications.
method Derives a secondary comprehensible agent from a NN-based RL agent.
result Empirical evaluation supports the possibility of building a comprehensible agent.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
RL applied to finance tasks, highlighting challenges and future directions.
problem Decision-making tasks in finance using RL.
method Meta-analysis of RL applications, identifying challenges and proposing future directions.
result Challenges in RL performance and future research directions.
A simple approach to offline RL without additional complexity.
problem Learning from a fixed dataset of actions with value estimation errors.
method Adding a behavior cloning term to the policy update of an online RL algorithm and normalizing the data.
result Matches the performance of state-of-the-art offline RL algorithms with minimal changes.
MVFST-RL tackles real-time network congestion control with RL.
problem Real-time network congestion control with RL in asynchronous environments.
method Asynchronous RL framework for delayed actions in QUIC.
result Scalable framework for congestion control in QUIC with improved state-of-the-art RL.
Unified framework for statistical inference of low-rank tensors.
problem Statistical inference for tensors in high-dimensional data.
method Unified framework using debiasing and tangent space projection.
result Achieves asymptotic normality and minimax-optimal confidence intervals.
MiniHack simplifies creation of complex RL environments.
problem Limited availability of challenging RL benchmarks.
method Develops a sandbox framework for easy RL environment design.
result MiniHack enables rapid creation of diverse RL testbeds.
Generative flow networks use RL to learn probabilistic models efficiently.
problem Training generative models with RL for compositional discrete objects.
method Reformulate GFlowNet training as entropy-regularized RL with specific reward and regularizer.
result Entropy-regularized RL can be competitive with established GFlowNet training methods.
RL agents outperform baselines in asset allocation.
problem Optimizing asset allocation using reinforcement learning.
method Model-free deep RL agents trained on real-world stock prices.
result RL agents significantly outperformed random and uniform allocation.
This paper shows RL with KL penalties is equivalent to Bayesian inference for fine-tuning LMs.
problem Fine-tuning large language models to avoid undesirable features.
method Analyzed KL-regularized RL and showed it's equivalent to variational inference.
result KL-regularized RL avoids distribution collapse and is more insightful as Bayesian inference.
HTMRL uses HTM for RL, adapting faster to changing environments.
problem Adapting to non-stationary environments in RL.
method Strictly HTM-based RL algorithm.
result HTMRL adapts faster to changing environments in a 10-armed bandit.
New benchmarks for offline RL from diverse datasets.
problem Measuring progress in offline RL due to lack of suitable benchmarks.
method Developed benchmarks tailored for offline RL, focusing on diverse dataset properties.
result Revealed deficiencies in existing offline RL algorithms.
M3PO improves model-based meta-RL with theoretical guarantees.
problem Improving sample efficiency in multi-task RL with theoretical guarantees.
method Extending Janner et al. (2019) theorems, proposing M3PO with performance guarantees.
result M3PO outperforms existing methods in continuous-control benchmarks.
Introduces a new RL formalism focusing on event desirability.
problem Limited consideration of performance distribution in RL.
method Introduces micro-objective reinforcement learning.
result New formalism allows for prior knowledge and event desirability.
evo-RL combines evolutionary computation with reinforcement learning for better adaptability.
problem Improving reinforcement learning algorithms' adaptability and performance in environments with rewardless states.
method Embedding reinforcement learning in an evolutionary cycle, distinguishing instinctive from learnable behavior.
result evo-RL leads to state-of-the-art performance on OpenAI Gym control problems with rewardless states.
This work bridges offline RL and DRL to address distributional shift.
problem Distributional shift in offline RL due to difference in state-action visitation distributions.
method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.
result Demonstrates superior performance of proposed algorithms through simulations.
RL Unplugged benchmarks offline RL methods across diverse domains.
problem Evaluate offline reinforcement learning methods without online data collection.
method Proposes a benchmark suite with diverse datasets and detailed evaluation protocols.
result Demonstrates the effectiveness of offline RL methods across various domains.
Optimistic RL algorithms are simplified for deep RL with competitive performance.
problem Achieving accurate optimism in model-based RL for large-scale problems.
method Interpreting scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.
result Competitive regret bound of i l d e O ( ∣ S ∣ H ∣ A ∣ T ) ilde{\mathcal{O}}( |\mathcal{S}|H\sqrt{|\mathcal{A}| T } ) i l d e O ( ∣ S ∣ H ∣ A ∣ T ) for Gaussian noise augmentation. Paper proposes a hybrid RL algorithm that combines offline and online data without needing reward info.
problem How to efficiently use online data to improve RL policies using only offline data.
method A three-stage hybrid RL algorithm that uses reward-agnostic exploration and model-based offline RL.
result The hybrid RL algorithm outperforms both pure offline and pure online RL in sample complexity.
Critic-regularized regression improves offline RL performance.
problem Poor performance of off-policy algorithms in offline RL.
method Critic-regularized regression (CRR) for policy learning from fixed datasets.
result CRR outperforms state-of-the-art offline RL algorithms significantly.
This paper surveys RL for combinatorial optimization, focusing on TSP.
problem Optimizing solutions for combinatorial optimization problems.
method Reinforcement learning applied to combinatorial optimization problems, specifically the TSP.
result Deep learning mechanisms enhance RL algorithms for near-optimal solutions.
MushroomRL simplifies RL experiments for researchers.
problem Complexity in implementing and testing RL experiments.
method Provides a comprehensive and flexible framework to minimize effort.
result Significantly benefits RL researchers in empirical analysis.
Hierarchical RL simplifies exploration in RL tasks.
problem Why does hierarchy work well in RL?
method Evaluated hierarchical RL on various tasks, focusing on exploration benefits.
result Most benefits of hierarchy can be attributed to improved exploration.
Mixed RL improves RL efficiency with dual representations.
problem Poor sampling efficiency in RL methods.
method Uses dual representations of environmental dynamics to search optimal policies.
result Proves convergence and recursive stability of the mixed RL.
Reduces RL to online learning, improving RL algorithms.
problem Designing efficient reinforcement learning algorithms.
method Reduction from RL to online learning, focusing on regret minimization and function approximation.
result Demonstrates a new RL algorithm with provable performance guarantees.
Survey of RL algorithms for dynamic environments.
problem Handling changing operating conditions in RL.
method Survey of algorithms developed for non-stationary environments.
result Methods to adapt to varying operating conditions.
This chapter reviews statistical tools for reinforcement learning.
problem Applying RL algorithms in healthcare and ride-sharing platforms.
method Statistical inference tools for RL, including hypothesis testing and confidence interval construction.
result Highlighting the value of statistical inference in RL for both communities.
Offline RL policies should adapt to unknown aspects of the environment.
problem Uncertainty in offline RL datasets leads to suboptimal policies.
method Adaptive policies that consider all transitions seen so far, solving an implicit POMDP.
result Optimal adaptive policies improve offline RL performance.
A new RL framework helps align RL solutions with existing practices.
problem Discrepancies between RL solutions and existing procedures.
method Student-teacher RL mechanism with a constraint on policy difference.
result Proves asymptotic optimality and effectiveness in multiple scenarios.