arXiv research

RECODE uses clustering and embedding to track state visitation counts in RL.

The paper introduces return parity for fairness in MDPs, addressing delayed and adverse effects.

Study how untrained policies explore in RL environments.

We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…

Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurre…

Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches t…

We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …

We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon pr…

Suppose an agent is in a (possibly unknown) Markov Decision Process in the absence of a reward signal, what might we hope that an agent can efficiently learn to do? This work studies a broad class of objectives that are defined solely as functions of the state-visitation frequencies that are induced by how the agent be…

Survey of methods for learning from observation without requiring expert actions.

MobILE learns from expert demonstrations without access to actions, achieving strong performance guarantees.

New reinforcement learning algorithm achieves instance-optimal sample complexity.

New method improves policies by combining Markov and non-Markov strategies.

New method reduces state distribution mismatch in off-policy RL.

Self-guided ALPs improve MDP policies without domain knowledge.

MOPO optimizes offline RL by penalizing dynamics uncertainty.

New algorithms speed up inverse reinforcement learning by solving MDPs once.

Reward-free RL in linear MDPs is as hard as reward-aware RL.

Brazil proposes a new BRICS trade currency to dominate international trade.