Paper shows intrinsic motivation boosts exploration efficiency in HRL.
problem Efficient exploration and subgoal discovery in model-free HRL.
method Unsupervised learning over agent's experiences for subgoal discovery.
result Intrinsic motivation learning improves exploration efficiency.
Paper reviews intrinsic motivations and their role in open-ended learning.
problem Understanding intrinsic motivations and their role in open-ended learning.
method Defining intrinsic motivations, presenting psychological/neuroscientific and computational models.
result Links between psychological/neuroscientific and computational models of intrinsic motivations.
AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
problem Learning in sparse reward environments.
method Adversarial intrinsic goals to generate a curriculum for a student policy.
result AMIGo enables agents to learn new skills without extrinsic rewards.
Study intrinsic motivation for synergistic tasks in reinforcement learning.
problem Sparse-reward synergistic tasks where multiple agents must work together.
method Propose incentivizing actions that affect the world in ways not achievable individually, using either true states or a dynamics model.
result Our approach yields more efficient learning than typical methods.
Graphical physics network learns intuitive physics using deep reinforcement learning with intrinsic motivation.
problem Teaching intuitive physics to AI agents.
method Integrates deep reinforcement learning with intrinsic reward normalization for efficient learning.
result Agent effectively learns object positions and velocities using intrinsic motivation.
Exploration in sparse reward reinforcement learning remains an open challenge. Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration. Commonly these signals are added as bonus rewards, which res…
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for …
A new reinforcement learning method uses mutual information to encourage agents to control their environment.
problem Learning from internal drives instead of external rewards.
method Formulate an intrinsic objective as mutual information between goal states and controllable states, derive a surrogate objective for efficient optimization.
result Demonstrated the efficacy of the approach in robotic tasks.
Study shows how to motivate AI agents like humans through learning from demonstrations.
problem Transfer human-like intrinsic motivation to AI agents in RL.
method Inverse RL approach to learn an exploration bonus from demonstrations.
result AI agents can learn and use complex exploration behaviors from demonstrations.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to replicate some of these abilities with a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically …
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
problem Improving exploration in reinforcement learning agents.
method Intrinsic rewards proportional to the entropy of future state-action features.
result The new objective leads to improved visitation of features within individual trajectories.
We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration in non-tabular reinforcement learning. Drawing inspiration from the intrinsic motivation literature, we use density models to measure uncert…
Agent learns goals and rewards through language and curiosity.
problem Autonomous agents lack intrinsic motivations and reward functions.
method LE2 algorithm using NL interactions and intrinsic motivations.
result Agent autonomously discovers and grounds goals in real behavior.
This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We envisaged actions as instrumental in agent's own epistemic disclosure. This motiva…
Automated discovery of diverse self-organized patterns in complex systems.
problem Automated identification of interesting spatially localized patterns in self-organizing systems.
method Intrinsically motivated machine learning algorithms (POP-IMGEPs) combined with deep auto-encoders and CPPN primitives.
result Efficiency and effectiveness of the proposed method in discovering diverse patterns compared to baselines.
Two new exploration methods for multi-agent systems improve team performance.
problem Exploration in transition-dependent multi-agent settings.
method EITI and EDTI, using mutual information and VoI to encourage coordinated exploration.
result Significant improvement in multi-agent performance through coordinated exploration.
Interprets how intrinsic motivation shapes behavior in RL agents.
problem Understanding how intrinsic motivation influences behavior in reinforcement learning agents.
method Analyzed five RL agents in procedurally generated environments using various interpretability techniques.
result Curiosity-driven agents exhibit broader and more dynamic attention than extrinsically motivated agents.
New RL approach uses future state and action visitation measures for better exploration.
problem Improving exploration in reinforcement learning.
method Intrinsic reward based on future state and action visitation measures, using contraction operators.
result Policies achieve good state-action space coverage and high performance.
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…
In this paper we present our scientific discovery that good representation can be learned via continuous attention during the interaction between Unsupervised Learning(UL) and Reinforcement Learning(RL) modules driven by intrinsic motivation. Specifically, we designed intrinsic rewards generated from UL modules for dri…
Enhances robotic grasping efficiency with learning-adaptive imagination.
problem Improving sample efficiency and performance in robotic grasping tasks.
method Learning-adaptive imagination approach using ensemble of local dynamics models in latent space.
result Significantly improves sample efficiency and achieves near-optimal performance.
A new multi-objective RL framework improves intrinsic exploration performance.
problem Sub-optimal exploration performance due to ad-hoc handling of intrinsic exploration.
method A multi-objective RL framework where both exploration and exploitation are optimized as separate objectives.
result EMU-Q method outperforms classic and other intrinsic RL methods on benchmarks.
Robots learn actions and language through curiosity-driven self-exploration.
problem Efficient development of actions and language in infants and robots.
method Curiosity-driven self-exploration using Q-learning to amortize active inference.
result Curiosity-driven exploration enables faster learning and compositional generalization.
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents…
Robotic arm learns to manipulate a ball by choosing goals from learned experience.
problem Efficient discovery of skills for long-living autonomous agents without supervision.
method Intrinsically motivated goal exploration using learned goal spaces from deep representation learning.
result Recent results show applicability of learned goal spaces on real-world robotic tasks.
We propose a new method for estimating the intrinsic dimension of a dataset by applying the principle of regularized maximum likelihood to the distances between close neighbors. We propose a regularization scheme which is motivated by divergence minimization principles. We derive the estimator by a Poisson process appr…
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
Agents learn sophisticated tool use and coordination in hide-and-seek.
problem Training agents to perform complex tasks in multi-agent environments.
method Multi-agent competition with hide-and-seek as a simple objective.
result Agents develop emergent strategies requiring sophisticated tool use and coordination.
Generates counterfactuals in target domain from source domain observations.
problem Cross-domain learning with domain shifts and lack of parallel datasets.
method Unsupervised, Neural Causal Models, Joint Causal Graphs, Effect-Intrinsic vs Domain-Intrinsic Variables.
result Framework generates counterfactuals that closely match ground truth.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecolo…
EXOC framework uses auxiliary variables for counterfactual fairness in machine learning.
problem Balancing fairness and predictive accuracy in models with sensitive attributes.
method EXOC framework uses auxiliary variables to define an auxiliary node and a control node for counterfactual fairness.
result EXOC framework outperforms state-of-the-art approaches in achieving counterfactual fairness.
Universal AI seeks high-optionality states through empowerment and curiosity.
problem Understanding and optimizing AI behavior in uncertain environments.
method Unified framework combining AIXI and variational empowerment, showing how universal AI agents balance goal-directed behavior with uncertainty reduction curiosity.
result Self-AIXI asymptotically converges to AIXI performance and exhibits power-seeking behavior due to intrinsic motivations.
Robot learns multiple tasks hierarchically by transferring knowledge.
problem Learning multiple complex tasks in open-ended environments.
method Task-oriented procedures, goal-babbling, imitation learning, active learning, intrinsic motivation.
result Robots can learn complex tasks more efficiently by transferring knowledge from simpler ones.
Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrins…
Q*BERT learns to navigate text-based games by building a knowledge graph.
problem Text-based games have bottlenecks that standard RL agents struggle to overcome.
method Q*BERT learns a knowledge graph and uses intrinsic motivation to detect and overcome bottlenecks.
result Q*BERT outperforms state-of-the-art agents in text games, including Zork.
A new method compares unaligned datasets using log-Euclidean signatures of SPD matrices.
problem Efficiently comparing datasets with unknown alignment.
method Diffusion operators, Riemannian geometry, log-Euclidean metric.
result LES distance recovers meaningful structural differences, outperforming existing methods.
Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees…
Research identifies four motivational groups for crypto-metaverse landowners.
problem Understanding motivations of retail investors in the crypto-metaverse.
method Detailed financial behavior survey and principal components analysis.
result Four distinct motivational groups identified: Aesthetics, Social, Speculation, Innovation.
We explain how to encode the essential data of a PDE on jet bundle into a more intrinsic object called Pfaffian fibration. We provide motivations to study this new notion and show how prolongations, integrability and linearisations of PDEs generalise to this setting.
Study semicontinuity of capacity in non-smooth spaces using intrinsic flat convergence.
problem Investigate semicontinuity of capacity in non-smooth spaces.
method Analyze sequences of local integral current spaces converging in the pointed Sormani-Wenger intrinsic flat sense.
result Prove upper semicontinuity of capacity for balls and Lipschitz sublevel sets under volume-preserving convergence.
We present a compensated compactness theorem in Banach spaces established recently, whose formulation is originally motivated by the weak rigidity problem for isometric immersions of manifolds with lower regularity. As a corollary, a geometrically intrinsic div-curl lemma for tensor fields on Riemannian manifolds is ob…
New method calculates empowerment from visual data, solving complex reinforcement learning problems.
problem Calculating empowerment in unknown dynamics from visual observation is challenging.
method Developed a novel approach using stochastic dynamic models in latent space and the Water-Filling algorithm.
result Efficiently computed empowerment in unknown dynamics from visual observation only.
Stability of positive mass theorem for hyperbolic manifolds studied.
problem Stability of the positive mass theorem for asymptotically hyperbolic manifolds.
method Adapted intrinsic flat distance approach to show stability for a class of manifolds.
result Stability of the positive mass theorem for a class of asymptotically hyperbolic graphical manifolds.
This paper identifies all topological symmetry groups for Heawood family graphs.
problem Understanding symmetries of spatial graphs in 3D space.
method Analyzing automorphisms of graphs embedded in S3. result All graphs in the Heawood family are intrinsically chiral.
New score matching method estimates local intrinsic dimension efficiently.
problem Quantifying the local intrinsic dimension of complex data.
method Denoising score matching loss and equivalent implicit score matching loss.
result Denoising score matching loss is a highly competitive and scalable LID estimator.
We present an active learning architecture that allows a robot to actively learn which data collection strategy is most efficient for acquiring motor skills to achieve multiple outcomes, and generalise over its experience to achieve new outcomes. The robot explores its environment both via interactive learning and goal…
Study of potential Carroll structures and special Carrollian manifolds for null hypersurfaces.
problem Understanding intrinsic geometry of null hypersurfaces.
method Initiate study of potential Carroll structures and explore their relationship to special Carrollian manifolds.
result Initiate the study of potential Carroll structures and their relationship to special Carrollian manifolds.
A method for self-supervised representation learning in partially observable environments.
problem Sparse rewards and stochasticity in partially observable environments.
method World model in latent space for estimating missing information.
result Significant improvement in exploration compared to prior work.