Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrins…
Robots learn actions and language through curiosity-driven self-exploration.
problem Efficient development of actions and language in infants and robots.
method Curiosity-driven self-exploration using Q-learning to amortize active inference.
result Curiosity-driven exploration enables faster learning and compositional generalization.
This work enhances curiosity-driven exploration in reinforcement learning.
problem Sparse or insufficient extrinsic feedback in real-world scenarios.
method Probabilistic attention mechanism combined with Actor-Critic framework and ICM.
result New methods improve agent performance in reinforcement learning tasks.
This study proposes hidden state curiosity to enhance RL models' resilience against noise.
problem Curiosity traps in RL models distract agents from discovering novel experiences.
method Proposed hidden state curiosity based on the Free Energy Principle to reward agents for KL divergence between predictive priors and posteriors.
result Agents with hidden state curiosity are more resilient against curiosity traps compared to those with prediction error curiosity.
New method combines curiosity and hindsight for stacking blocks.
problem Sparse rewards in reinforcement learning.
method Curiosity-driven exploration combined with hindsight and curriculum learning.
result First to stack more than two blocks using only sparse reward.
New method uses hindsight to make exploration robust in stochastic environments.
problem Exploration in sparse-reward or reward-free environments, especially in stochastic settings.
method Learn representations of the future that capture unpredictable aspects, using them to predict and reward only the predictable parts of the world.
result Improves exploration in Atari games and Montezuma's Revenge, robust to stochasticity.
Curiosity-driven recommendations improve personalized learning efficiency and enjoyment.
problem Improving personalized learning through adaptive strategies.
method Curiosity-driven reinforcement learning with actor-critic neural networks.
result Demonstrated through numeric analyses, the proposed method enhances learning efficiency and enjoyment.
FICM uses motion features to encourage exploration in DRL environments.
problem Exploiting novelty in environments with moving objects.
method Flow-based intrinsic curiosity module (FICM) leveraging motion features between consecutive frames.
result FICM encourages exploration in environments featuring moving objects.
Universal AI seeks high-optionality states through empowerment and curiosity.
problem Understanding and optimizing AI behavior in uncertain environments.
method Unified framework combining AIXI and variational empowerment, showing how universal AI agents balance goal-directed behavior with uncertainty reduction curiosity.
result Self-AIXI asymptotically converges to AIXI performance and exhibits power-seeking behavior due to intrinsic motivations.
Capsule Networks improve autonomous navigation in sparse environments.
problem Challenges in reinforcement learning for sparse reward environments.
method Caps-EM pairs CapsNets with Advantage Actor Critic, using fewer parameters.
result Caps-EM achieves significant time improvements and fewer parameters compared to competing methods.
Curiosity enhanced by audio-visual associations improves learning efficiency.
problem Challenges in reinforcement learning, especially predicting the future.
method Exploits multiple modalities (audio and vision) to predict novel associations.
result Improves exploration and learning efficiency in various environments.
Agent learns goals and rewards through language and curiosity.
problem Autonomous agents lack intrinsic motivations and reward functions.
method LE2 algorithm using NL interactions and intrinsic motivations.
result Agent autonomously discovers and grounds goals in real behavior.
Curiosity-driven exploration using Bayesian surprise in latent space.
problem Enhance exploration capabilities in reinforcement learning.
method Apply Bayesian surprise in a latent space to favor exploration.
result Our method is computationally cheap and performs well on various tasks.
In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later in its life. We formulate curiosity as the error in an agen…
Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards. One promising strategy is inspired by human curiosity, requiring the agent to learn to predict the future. In this paper a curiosity-driven agent is extended to use these predictions di…
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to replicate some of these abilities with a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically …
Interprets how intrinsic motivation shapes behavior in RL agents.
problem Understanding how intrinsic motivation influences behavior in reinforcement learning agents.
method Analyzed five RL agents in procedurally generated environments using various interpretability techniques.
result Curiosity-driven agents exhibit broader and more dynamic attention than extrinsically motivated agents.
AutoOD automates outlier detection using curiosity-guided search and self-imitation learning.
problem Automated outlier detection for complex tasks with big data.
method Curiosity-guided search strategy and self-imitation learning.
result AutoOD identifies optimal neural network models with superior performance.
BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.
problem Exploration in visually complex environments.
method Optimizes a single prediction loss in latent space to learn world representation, dynamics, and exploration policy.
result Achieves superhuman performance on Atari games with simpler design.
Curious hierarchical reinforcement learning improves learning performance.
problem Combining hierarchical abstraction and curiosity-driven exploration in reinforcement learning.
method Developed a method that combines hierarchical reinforcement learning with curiosity.
result Curiosity can more than double learning performance and success rates.
BeBold improves exploration in sparse-reward tasks by regulating visitation counts.
problem Efficient exploration in deep reinforcement learning under sparse rewards.
method Regulated difference of inverse visitation counts.
result BeBold solves 12 challenging tasks in MiniGrid with fewer steps than previous state-of-the-art.
Intrinsically motivated goal exploration processes enable agents to autonomously sample goals to explore efficiently complex environments with high-dimensional continuous actions. They have been applied successfully to real world robots to discover repertoires of policies producing a wide diversity of effects. Often th…
Meta-learning curiosity algorithms improves exploration across various tasks.
problem Generating curious behavior in reinforcement learning.
method Meta-learning approach to adapt reward signals dynamically.
result Two novel curiosity algorithms outperform human-designed ones.
This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We envisaged actions as instrumental in agent's own epistemic disclosure. This motiva…
Compactness theorem for timed-metric spaces established.
problem Compactness of timed-metric spaces and causality.
method Timed-Gromov--Hausdorff distance and intrinsic timed-Hausdorff distance.
result Induces same notion of convergence as intrinsic timed-Hausdorff distance.
Quriosity analyzes curiosity-driven questions from diverse sources.
problem Understanding and analyzing human curiosity-driven questions.
method Collection of 13.5K questions from various sources, development of an iterative prompt improvement framework.
result 42% of questions are causal, revealing unique linguistic properties.
Bayesian method improves reinforcement learning efficiency by encouraging exploration.
problem High sample complexity due to inefficient exploration.
method Bayesian linear regression and latent space embedding for intrinsic reward.
result Significant improvement in sample complexity on various tasks.
This work designs an active world model learning system with progress-based curiosity.
problem Learning compact world models from visual exploration.
method Constructs a curious agent in a 3D environment, using γ-Progress as a curiosity signal. result The γ-Progress-driven controller outperforms state-of-the-art exploration strategies. Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, obs…
CCLF improves sample efficiency in RL by prioritizing informative samples.
problem Learning from high-dimensional observations is challenging and unstable.
method Contrastive-Curiosity-Driven Learning Framework (CCLF) that prioritizes informative samples.
result CCLF improves sample efficiency and learning performance in RL.
Unified framework for hybrid learning and optimization via active inference.
problem Sequential decisions in black-box evaluations requiring both task improvement and uncertainty reduction.
method Pragmatic Curiosity (PraC) framework that evaluates queries by balancing information gain and pragmatic value.
result Unified approach reduces decision risk and improves coverage of critical regions without task-specific rules.
CUDC collects diverse data for offline RL by predicting future states.
problem Challenges in collecting task-agnostic data for offline RL.
method Adaptive temporal distances for curiosity-driven data collection.
result CUDC outperforms existing unsupervised methods in offline RL tasks.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecolo…
Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.
problem Hyperparameter tuning is a significant challenge in RL, especially for curiosity-based exploration methods.
method Hyper robustly explores by effectively regularizing exploration visits and decoupling exploitation.
result Hyper is provably efficient and robust in various RL environments.
Efficient method for choosing data points in machine learning models.
problem Choosing optimal data points for semi-supervised learning.
method Computing a 'regularity tangent' vector to measure model complexity and influence.
result The method efficiently calculates a measure of model complexity and data point influence.
Study shows how to motivate AI agents like humans through learning from demonstrations.
problem Transfer human-like intrinsic motivation to AI agents in RL.
method Inverse RL approach to learn an exploration bonus from demonstrations.
result AI agents can learn and use complex exploration behaviors from demonstrations.
Exploration is a difficult challenge in reinforcement learning and even recent state-of-the art curiosity-based methods rely on the simple epsilon-greedy strategy to generate novelty. We argue that pure random walks do not succeed to properly expand the exploration area in most environments and propose to replace singl…
Signed compression progress on a sealed audit is goodhart-resistant.
problem Intrinsic motivation for agents to improve their world models by compressing experience.
method Rewarding agents for the signed decrease of a fixed sealed-audit loss.
result Cumulative reward telescopes exactly to endpoint audit improvement, preventing infinite reward push while true audit performance stagnates.
Curious Replay improves model-based reinforcement learning agents' adaptability.
problem Existing model-based reinforcement learning agents struggle to adapt quickly to changing environments.
method Curious Replay uses a curiosity-based priority signal for prioritized experience replay tailored to model-based agents.
result Agents using Curious Replay achieve improved performance in exploration and on benchmarks.
In Reinforcement Learning (RL), an agent explores the environment and collects trajectories into the memory buffer for later learning. However, the collected trajectories can easily be imbalanced with respect to the achieved goal states. The problem of learning from imbalanced data is a well-known problem in supervised…
New algorithm optimizes exploration in unknown MDPs.
problem Optimal exploration in unknown Markov decision processes.
method Trajectory-optimization-based approximate solution using Bayesian experimental design.
result Empirically superior to intrinsic-motivation-based algorithms in convergence and model fidelity.
We introduce a class of learning problems where the agent is presented with a series of tasks. Intuitively, if there is relation among those tasks, then the information gained during execution of one task has value for the execution of another task. Consequently, the agent is intrinsically motivated to explore its envi…
Expanding on a Heisenberg group case for a mathematical proposition.
problem Open case k=n in the Heisenberg group Hn. method Expanding on Proposition 3.6[1] in the Heisenberg group.
result Foster curiosity for its resolution.
Curious Meta-Controller alternates between model-based and model-free control to improve sample efficiency.
problem Combining the benefits of model-based and model-free control to enhance sample efficiency.
method Adaptive alternation between model-based and model-free control using curiosity feedback.
result Significantly improved sample efficiency and near-optimal performance on robotic tasks.
Project explores reinforcement learning solutions for sparse reward environments.
problem Difficulty in navigating environments with infrequent rewards.
method Contrast and investigate existing reinforcement learning solutions in various video games.
result Introduces a novel reinforcement learning solution combining curiosity and auxiliary tasks.
A graph abstraction speeds up reinforcement learning in complex environments.
problem Learning hierarchical reinforcement learning tasks in complex environments.
method Jointly trains a latent pivotal state model and a curiosity-driven policy. Uses a world graph to guide high-level and low-level agents.
result Significant performance and efficiency improvements over baseline methods.
ACE improves GFlowNet exploration efficiency by balancing complementary search strategies.
problem Efficient exploration of diverse high-probability regions in GFlowNets.
method Adaptive Complementary Exploration (ACE) trains a separate GFlowNet to search underexplored regions.
result Significantly improves approximation accuracy and diverse state discovery.