A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
AMIGo uses adversarial intrinsic goals to teach RL agents new skills.
There is a growing interest and literature on intrinsic motivations and open-ended learning in both cognitive robotics and machine learning on one side, and in psychology and neuroscience on the other. This paper aims to review some relevant contributions from the two literature threads and to draw links between them. …
Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method base…
Exploration in sparse reward reinforcement learning remains an open challenge. Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration. Commonly these signals are added as bonus rewards, which res…
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for …
We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks, which are tasks where multiple agents must work together to achieve a goal they could not individually. Our key idea is that a good guiding principle for intrinsic motivation in synergistic ta…
We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration in non-tabular reinforcement learning. Drawing inspiration from the intrinsic motivation literature, we use density models to measure uncert…
Study shows how to motivate AI agents like humans through learning from demonstrations.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to replicate some of these abilities with a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically …
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
We propose a new method for estimating the intrinsic dimension of a dataset by applying the principle of regularized maximum likelihood to the distances between close neighbors. We propose a regularization scheme which is motivated by divergence minimization principles. We derive the estimator by a Poisson process appr…
Autonomous reinforcement learning agents, like children, do not have access to predefined goals and reward functions. They must discover potential goals, learn their own reward functions and engage in their own learning trajectory. Children, however, benefit from exposure to language, helping to organize and mediate th…
This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We envisaged actions as instrumental in agent's own epistemic disclosure. This motiva…
At an early age, human infants are able to learn and build a model of the world very quickly by constantly observing and interacting with objects around them. One of the most fundamental intuitions human infants acquire is intuitive physics. Human infants learn and develop these models, which later serve as prior knowl…
In reinforcement learning, an agent learns to reach a set of goals by means of an external reward signal. In the natural world, intelligent organisms learn from internal drives, bypassing the need for external signals, which is beneficial for a wide range of tasks. Motivated by this observation, we propose to formulate…
Recent theory work has found that a special type of spatial partition tree - called a random projection tree - is adaptive to the intrinsic dimension of the data from which it is built. Here we examine this same question, with a combination of theory and experiments, for a broader class of trees that includes k-d trees…
Interprets how intrinsic motivation shapes behavior in RL agents.
Research identifies four motivational groups for crypto-metaverse landowners.
We explain how to encode the essential data of a PDE on jet bundle into a more intrinsic object called Pfaffian fibration. We provide motivations to study this new notion and show how prolongations, integrability and linearisations of PDEs generalise to this setting.
New RL approach uses future state and action visitation measures for better exploration.
We present a compensated compactness theorem in Banach spaces established recently, whose formulation is originally motivated by the weak rigidity problem for isometric immersions of manifolds with lower regularity. As a corollary, a geometrically intrinsic div-curl lemma for tensor fields on Riemannian manifolds is ob…
Study semicontinuity of capacity in non-smooth spaces using intrinsic flat convergence.
Stability of positive mass theorem for hyperbolic manifolds studied.
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…
Combining model-based and model-free deep reinforcement learning has shown great promise for improving sample efficiency on complex control tasks while still retaining high performance. Incorporating imagination is a recent effort in this direction inspired by human mental simulation of motor behavior. We propose a lea…
This paper identifies all topological symmetry groups for Heawood family graphs.
A new multi-objective RL framework improves intrinsic exploration performance.
Study of potential Carroll structures and special Carrollian manifolds for null hypersurfaces.
Universal AI seeks high-optionality states through empowerment and curiosity.
In many complex dynamical systems, artificial or natural, one can observe self-organization of patterns emerging from local rules. Cellular automata, like the Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence …
In this paper we present our scientific discovery that good representation can be learned via continuous attention during the interaction between Unsupervised Learning(UL) and Reinforcement Learning(RL) modules driven by intrinsic motivation. Specifically, we designed intrinsic rewards generated from UL modules for dri…
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents…
Robots learn actions and language through curiosity-driven self-exploration.
Through multi-agent competition, the simple objective of hide-and-seek, and standard reinforcement learning algorithms at scale, we find that agents create a self-supervised autocurriculum inducing multiple distinct rounds of emergent strategy, many of which require sophisticated tool use and coordination. We find clea…
Generates counterfactuals in target domain from source domain observations.
The paper explores geometric invariants of null hypersurfaces using Carrollian geometry.
New research shows DDPM can adapt to data's intrinsic low dimensionality efficiently.
Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…
Motivated by the important work of Brown adn York on quasilocal energy, we propose definitions of quasilocal energy and momentum surface energy of a spacelike 2-surface with positive intrinsic curvature in a spacetime. We show that the quasilocal energy of the boundary of a compact spacelike hypersurface which satisfie…
Study on predicting sequences with Gaussian constraints, linking to intrinsic volumes and metric complexity.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecolo…
EXOC framework uses auxiliary variables for counterfactual fairness in machine learning.
In-BO optimizes complex constrained domains using SIn-GP surrogate models.
Motivated by the study in Morse theory and Smale's work in dynamics, the following questions are studied and answered: (1) When does a 3-manifold admit an automorphism having a knotted Smale solenoid as an attractor? (2) When does a 3-manifold admit an automorphism whose non-wandering set consists of Smale solenoids? T…
Study of convergence of point-object configurations to a charged dust continuum.
New score matching method estimates local intrinsic dimension efficiently.
A new method compares unaligned datasets using log-Euclidean signatures of SPD matrices.
In this work we study the intrinsic geometry of the space of Kahler metrics under various Riemannian metrics. The first part is on the Dirichlet metric. We motivate its study, we compute its curvature, and we make links with the Calabi metric, the K-energy, the degenerate complex Hessian equation. The second part is on…