arXiv research

Survey on stated skein algebras and their representations.

This work improves understanding of reinforcement learning state representations.

Robots could learn their own state and world representation from perception and experience without supervision. This desirable goal is the main focus of our field of interest, state representation learning (SRL). Indeed, a compact representation of such a state is beneficial to help robots grasp onto their environment …

State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks. Learning such representations without supervision from rewards is a challenging open problem. We introduce a method that learns state…

Survey categorizes methods for learning state representations in reinforcement learning.

Intelligent agents can cope with sensory-rich environments by learning task-agnostic state abstractions. In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP). Our m…

New method learns high-quality Laplacian representations for reinforcement learning.

Paper learns meaningful state and action representations from MDP trajectories.

State representation learning aims at learning compact representations from raw observations in robotics and control applications. Approaches used for this objective are auto-encoders, learning forward models, inverse dynamics or learning using generic priors on the state characteristics. However, the diversity in appl…

Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore.…

New insights into state representations in RL help design better learning rules.

Representation learning algorithms are designed to learn abstract features that characterize data. State representation learning (SRL) focuses on a particular kind of representation learning where learned features are in low dimension, evolve through time, and are influenced by actions of an agent. The representation i…

Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, …

We consider the problem of building a state representation model in a continual fashion. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge. The learned features are then fed to a Reinforcement Learning algorithm to learn a policy. We propose to …

Graph-based state representation improves deep RL performance.

Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments with partial or noisy observations. In partially observable domains it is important for the representation to encode a belief state, a sufficie…

Recurrent neural networks (RNNs) are a vital modeling technique that rely on internal states learned indirectly by optimization of a supervised, unsupervised, or reinforcement training loss. RNNs are used to model dynamic processes that are characterized by underlying latent states whose form is often unknown, precludi…

Proves error bounds for state representation in RL using graph spectral features.

Study cost-driven state representation learning for control from partial observations.

Paper introduces SALE for better state-action learning in RL.

Study learns state representations from observations for control, proving guarantees.

We explore the benefits of augmenting state-of-the-art model-free deep reinforcement algorithms with simple object representations. Following the Frostbite challenge posited by Lake et al. (2017), we identify object representations as a critical cognitive capacity lacking from current reinforcement learning agents. We …

ReLEX algorithm improves RL efficiency by selecting optimal representations.

Extended quantum state result for gl_n weight systems.

We consider the problem of building a state representation model for control, in a continual learning setting. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge, and then use Reinforcement Learning on the resulting features for efficient policy …

We consider the problem of learning low-dimensional representations for large-scale Markov chains. We formulate the task of representation learning as that of mapping the state space of the model to a low-dimensional state space, called the kernel space. The kernel space contains a set of meta states which are desired …

Empirical mode modeling improves state-space analysis of noisy data.

We emphasize some properties of coherent state groups, i.e. groups whose quotient with the stationary groups, are manifolds which admit a holomorphic embedding in a projective Hilbert space. We determine the differential action of the generators of the representation of coherent state groups on the symmetric Fock space…

SPEDER extracts state-action abstraction from dynamics for reinforcement learning.

A method to train multi-agent reinforcement learning models without intrinsic rewards.

The paper introduces a method to learn Markov state abstractions for reinforcement learning.

POLAR learns efficient data acquisition policies using pretrained belief representations.

Reinforcement learning (RL) algorithms allow artificial agents to improve their selection of actions to increase rewarding experiences in their environments. Temporal Difference (TD) Learning -- a model-free RL method -- is a leading account of the midbrain dopamine system and the basal ganglia in reinforcement learnin…

We present state sums for quantum link invariants arising from the representation theory of $U_q(\mathfrak{gl}_{N|M})$. We investigate the case of the $N$-th exterior power of the standard representation of $U_q(\mathfrak{gl}_{N|1})$ and explicit the relation with Kashaev invariants.

The paper studies properties of stated SL(n)-skein algebras and their centers.

MuZero visualizes its internal representations to stabilize planning.

A new method for estimating uncertainty in deep neural networks.

This thesis explores DAHA representations using stated skein theory.

We introduce Recurrent Predictive State Policy (RPSP) networks, a recurrent architecture that brings insights from predictive state representations to reinforcement learning in partially observable environments. Predictive state policy networks consist of a recursive filter, which keeps track of a belief about the stat…

Can simple algorithms with a good representation solve challenging reinforcement learning problems? In this work, we answer this question in the affirmative, where we take "simple learning algorithm" to be tabular Q-Learning, the "good representations" to be a learned state abstraction, and "challenging problems" to be…

New method improves reinforcement learning generalization.

Adversarially trained generative models (GANs) have recently achieved compelling image synthesis results. But despite early successes in using GANs for unsupervised representation learning, they have since been superseded by approaches based on self-supervision. In this work we show that progress in image generation qu…

DAGR improves navigation by refining goal representations conditioned on the current state.

The smallest eigenvectors of the graph Laplacian are well-known to provide a succinct representation of the geometry of a weighted graph. In reinforcement learning (RL), where the weighted graph may be interpreted as the state transition process induced by a behavior policy acting on the environment, approximating the …

We extend some results of Bonahon, Bullock, Turaev and Wong concerning the skein algebras of closed surfaces to L^e's stated skein algebra associated to open surfaces. We prove that the stated skein algebra with deforming parameter +1 embeds canonically into the centers of the stated skein algebras whose deforming para…

Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems. RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. In …

In this paper we introduce a simple approach for exploration in reinforcement learning (RL) that allows us to develop theoretically justified algorithms in the tabular case but that is also extendable to settings where function approximation is required. Our approach is based on the successor representation (SR), which…

Studying general quantum many-body systems is one of the major challenges in modern physics because it requires an amount of computational resources that scales exponentially with the size of the system.Simulating the evolution of a state, or even storing its description, rapidly becomes intractable for exact classical…