This work learns latent representations to speed up exploration in complex environments.
problem Challenging exploration in high-dimensional state and action spaces with sparse rewards.
method Representation learning using prior experience to learn effective latent representations.
result Learned latent representations reduce the dimensionality of the search space for effective exploration.
EBE enables efficient exploration in reinforcement learning.
problem Inefficient exploration in reinforcement learning.
method Entropy-based exploration (EBE) quantifies learning and adaptively explores unexplored regions.
result EBE enables faster learning without hyperparameter tuning.
Unified framework for efficient exploration in reinforcement learning.
problem Challenging control tasks in reinforcement learning.
method Combines Bayesian parameter updates with deep reinforcement learning.
result Practical algorithm achieves efficient exploration.
The paper proposes an algorithm to learn efficient and effective exploration policies in reinforcement learning.
problem Balancing exploration and exploitation in reinforcement learning.
method Formalized a counterfactual metric for exploration utility and used meta-learning to learn an end-to-end exploration policy.
result Demonstrated improved performance in high-dimensional control tasks in MuJoCo simulator compared to previous methods.
Q-learning with nonlinear Q-function can explore effectively without explicit exploration.
problem How to effectively explore in reinforcement learning.
method Q-learning with nonlinear Q-function and no explicit exploration.
result Q-learning with nonlinear Q-function can learn as well as ε-greedy exploration. Proposes a method to avoid excessive exploration in reinforcement learning.
problem Avoiding excessive exploration in reinforcement learning to deploy it in practice.
method Designs a novel algorithm using UCB reinforcement learning policy with adaptive exploration constraints.
result Proves that the approach remains conservative while minimizing regret in tabular settings and validates on real-world tasks.
New meta-RL method avoids exploration-exploitation trade-off.
problem Learning to explore and exploit simultaneously in meta-RL.
method Developed new objectives for exploration and exploitation.
result DREAM outperforms existing methods on complex tasks.
This work proposes sample complexity bounds for Q-learning with random exploration.
problem Understanding the sample efficiency of simple exploration strategies in reinforcement learning.
method Problem-specific sample complexity bounds for Q-learning with random walk exploration.
result Proposes bounds that relate to empirical performance in benchmark domains.
Study explores how to efficiently explore communities with limited budget.
problem Maximizing the number of members met with limited budget in community exploration.
method Systematic study from offline optimization to online learning, including greedy methods and upper confidence algorithms.
result Achieved logarithmic and constant regret bounds in online learning setting.
This work tackles the exploration-exploitation dilemma in RL by developing optimal policies that are inherently exploration-conscious.
problem The exploration-exploitation tradeoff in Reinforcement Learning, where policies need to balance new action exploration with past experience exploitation.
method Developed exploration-conscious criteria that result in optimal policies, solving these criteria by solving a surrogate Markov Decision Process.
result Demonstrated superior performance of exploration-conscious RL algorithms compared to non-exploration-conscious counterparts in both discrete and continuous action spaces.
MELEE learns good exploration strategies for contextual bandits.
problem Interactive contextual bandit exploration trade-off.
method Meta-learning from synthetic data to learn a good exploration policy.
result MELEE outperforms seven strong baseline algorithms on real-world datasets.
MAME models a separate exploration policy for faster adaptation.
problem Efficient exploration strategies for quick task adaptation in meta-reinforcement learning.
method Explicitly models a separate exploration policy for task distribution, using self-supervised or supervised learning objectives for adaptation.
result Superior performance compared to prior works in meta-reinforcement learning.
New findings on when to use action space exploration in reinforcement learning.
problem Understanding when to use action space exploration over traditional methods.
method Theoretical analysis and empirical testing of simple exploration methods.
result Exploration in action space is preferred when parametric complexity exceeds action space dimensionality and horizon length.
R3L uses planning algorithms to efficiently explore sparse reward environments.
problem Balancing exploration and exploitation in sparse reward reinforcement learning.
method Formulate exploration as a search problem using RRT, leverage demonstrations from initial solutions to refine RL policy.
result R3L outperforms classic and intrinsic exploration techniques, requiring fewer samples and achieving better asymptotic performance.
A new multi-objective RL framework improves intrinsic exploration performance.
problem Sub-optimal exploration performance due to ad-hoc handling of intrinsic exploration.
method A multi-objective RL framework where both exploration and exploitation are optimized as separate objectives.
result EMU-Q method outperforms classic and other intrinsic RL methods on benchmarks.
Proposes EVE for efficient exploration in reinforcement learning.
problem Efficient exploration in reinforcement learning.
method EVE: a recipe for posterior over parameters, facilitating efficient exploration.
result Competitive performance on benchmarks, efficient exploration confirmed.
IDS improves exploration in deep reinforcement learning.
problem Efficient exploration in reinforcement learning, especially with heteroscedastic returns.
method Information-Directed Sampling (IDS) for deep Q-learning.
result Significant improvement in Atari game performance over alternative approaches.
HyperX uses reward bonuses to enable efficient exploration in meta-learning.
problem Catastrophic failure of meta-learning with sparse rewards.
method HyperX uses novel reward bonuses to explore in approximate hyper-state space.
result HyperX meta-learns better task-exploration and adapts more successfully to new tasks.
New DRL algorithm improves sample efficiency and exploration performance.
problem High sample cost in deep reinforcement learning.
method Combines entropy and bootstrap techniques with Tsallis entropy regularization.
result Demonstrates more efficient and effective exploration on Atari games.
Curiosity-driven exploration learns disentangled goal spaces for efficient complex environments.
problem Efficient exploration in complex environments with high-dimensional continuous actions.
method Learned disentangled goal spaces to reflect the environment's structure and maximize learning progress.
result Disentangled goal spaces lead to better exploration performances than entangled goal spaces.
The paper explores using return distribution for exploration in RL, solving complex tasks.
problem Exploration in deterministic RL environments.
method Study of network losses and propagation mechanisms for different return distributions.
result Solves a randomized Chain task of length 100, previously unsolved.
New exploration bonuses improve reinforcement learning efficiency.
problem Efficient exploration in unknown environments with limited feedback.
method Improved exploration bonuses scaling with 1/n and improved stopping time analysis.
result Faster learning rates and improved sample complexity in pure-exploration settings.
The paper evaluates various bonus-based exploration methods in the ALE and finds limited improvement in performance.
problem Improving exploration in reinforcement learning algorithms, especially in challenging games.
method Empirical evaluation of different reward bonuses on the Arcade Learning Environment.
result Recently developed bonus-based exploration methods do not significantly improve performance in challenging games.
Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computationally and statistically efficient manner through use of randomized value functions. Unlike dithering strategies such as epsilon-greedy explorat…
A new exploration method for RL using parameter space noise.
problem Improving exploration in deep reinforcement learning.
method Switching isotropic and directional exploration in parameter space with parameter space noise.
result The proposed method achieves competitive results and better performance in sparse reward environments.
Directed exploration improves reinforcement learning efficiency and robustness.
problem Achieving good sample efficiency in reinforcement learning with efficient exploration.
method Directed exploration through goal-conditioned policies that are independent of uncertainty.
result Directed exploration is more efficient and robust to uncertainty than reward bonuses.
Achieving efficient and scalable exploration in complex domains poses a major challenge in reinforcement learning. While Bayesian and PAC-MDP approaches to the exploration problem offer strong formal guarantees, they are often impractical in higher dimensions due to their reliance on enumerating the state-action space.…
Method learns latent states from rich observations to improve RL exploration.
problem Improving RL performance with rich observations and latent states.
method Estimates latent states from observations through regression and clustering, providing finite-sample guarantees.
result Exponential improvement over Q-learning with naïve exploration. HOMER learns latent states to explore rich environments efficiently.
problem Exploration in rich observation environments with unknown latent states.
method Interleaves representation learning and strategic exploration to identify kinematic states.
result Provably efficient exploration with polynomial sample complexity in latent states and time horizon.
New L1-Coverage objective simplifies exploration in reinforcement learning.
problem Challenges in exploration for high-dimensional domains.
method Introduces L1-Coverage objective to enable efficient exploration and planning. result First computationally efficient algorithms for online reinforcement learning with low coverability.
Proposes goal-based exploration for better learning efficiency.
problem Difficult exploration in reinforcement learning.
method Random goal selection followed by several steps in their direction.
result Generates longer and safer trajectories than individual random actions.
This paper tackles hard exploration in the game Pommerman, improving RL learning.
problem Hard exploration in sparse, delayed, and deceptive reward domains.
method Developed a model-based automatic reasoning module to prune unsafe actions.
result Model-based approach significantly improves RL learning in Pommerman.
A new approach for deep exploration in sparse reward reinforcement learning.
problem Slow or no learning in reinforcement learning with rare rewards.
method Long-term visitation count planning and decoupling exploration and exploitation.
result Significantly outperforms existing methods in sparse reward environments.
ASE safely explores unknown MDPs with unknown dynamics, improving sample efficiency.
problem Balancing exploration and safety in unknown MDPs with stochastic dynamics.
method Exploits analogies between state-action pairs to safely learn near-optimal policies.
result Empirically improves sample efficiency compared to existing methods.
SIL learns to repeat past good decisions for better exploration.
problem Improving exploration in reinforcement learning environments.
method Self-Imitation Learning (SIL) is a simple off-policy actor-critic algorithm.
result SIL significantly improves A2C on Atari games and is competitive with state-of-the-art methods.
BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.
problem Exploration in visually complex environments.
method Optimizes a single prediction loss in latent space to learn world representation, dynamics, and exploration policy.
result Achieves superhuman performance on Atari games with simpler design.
This paper proposes a self-supervised exploration method using disagreement of dynamics models.
problem Efficient exploration in stochastic environments with real robots.
method Train an ensemble of dynamics models and incentivize exploration to maximize disagreement.
result Sample-efficient exploration achieved without external rewards or reinforcement learning.
Enhances POLITEX for exploration in reinforcement learning with no-reward learning.
problem Learning to explore in reinforcement learning problems with no-reward.
method Modifies POLITEX to incorporate a pre-existing exploration policy.
result Achieves sublinear regret guarantees similar to POLITEX but without requiring all policies to explore.
Meta-MDP approach optimizes exploration for lifelong reinforcement learning.
problem Improving reinforcement learning performance across multiple tasks.
method Formulates exploration strategy as a reinforcement learning problem, leveraging patterns from related tasks.
result Optimized exploration strategy improves performance on new tasks.
VASE uses Bayesian neural networks to improve exploration in sparse reward environments.
problem Exploration in environments with continuous control and sparse rewards.
method VASE uses a Bayesian neural network model of the environment dynamics and variational inference to alternately update the model's accuracy and policy.
result VASE outperforms other surprise-based exploration techniques in continuous control sparse reward environments.
New RL method finds near-optimal policies efficiently.
problem Efficient exploration in model-free RL.
method Modifies Delayed Q-learning with long-term exploration bonus.
result Polynomial time PAC-MDP solution for near-optimal policy finding.
VAN method optimizes learning tasks with unified methods.
problem Optimizing learning tasks in active and reinforcement learning.
method Variational Adaptive-Newton method that unifies optimization, inference, and evolution strategies.
result VAN performs well on various learning tasks.
Enhances exploration in reinforcement learning with diversity-driven approach.
problem Challenges in efficient exploration in reinforcement learning, especially in large state spaces.
method Diversity-driven exploration strategy combining off- and on-policy reinforcement learning algorithms.
result Significantly enhances exploratory behaviors, preventing local optima.
The paper explores how contingency-awareness improves exploration in reinforcement learning.
problem Improving exploration in reinforcement learning environments with sparse rewards.
method Developed an attentive dynamics model (ADM) to discover controllable elements of observations and used it for state representation in exploration.
result Combining actor-critic algorithms with count-based exploration using the ADM representation achieved impressive results on Atari games.
The paper explores how state abstraction affects pseudo-count-based exploration bonuses.
problem Understanding the interplay between exploration and approximation in reinforcement learning.
method Quantifying MBIE-EB performance, relating density models to abstractions, and deriving new pseudo-count bonuses.
result A new pseudo-count bonus alleviates the mismatch between implicit and explicit abstractions.
ISL algorithm tackles deep exploration efficiently.
problem Deep exploration in reinforcement learning.
method Derives ISL algorithm by augmenting RL objective with a novel regularization term.
result Empirically shows state-of-the-art performance on deep-exploration benchmarks.
Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.
problem Hyperparameter tuning is a significant challenge in RL, especially for curiosity-based exploration methods.
method Hyper robustly explores by effectively regularizing exploration visits and decoupling exploitation.
result Hyper is provably efficient and robust in various RL environments.
Go-Explore improves performance on hard-exploration problems in Atari games.
problem Challenges in reinforcement learning, especially with sparse or deceptive rewards.
method Exploits principles of remembering states, returning to promising states, and solving simulated environments.
result Scores significantly higher than previous state-of-the-art on Montezuma's Revenge and Pitfall.