arXiv research

Framework learns useful subgoals from demonstrations and instructions.

Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only self-supervision, reaching novel goals in cluttered scenes with unseen objects. However, due to the compounding uncertainty in long horizon video prediction and poor s…

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method base…

Hierarchical Reinforcement Learning (HRL) exploits temporally extended actions, or options, to make decisions from a higher-dimensional perspective to alleviate the sparse reward problem, one of the most challenging problems in reinforcement learning. The majority of existing HRL algorithms require either significant m…

The paper introduces an adjacency constraint to improve goal-conditioned HRL.

In this work we present ISA, a novel approach for learning and exploiting subgoals in reinforcement learning (RL). Our method relies on inducing an automaton whose transitions are subgoals expressed as propositional formulas over a set of observable events. A state-of-the-art inductive logic programming system is used …

Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight difference. However, when the task distribution becomes wider, it would be quite inefficient to directly…

Typical reinforcement learning (RL) agents learn to complete tasks specified by reward functions tailored to their domain. As such, the policies they learn do not generalize even to similar domains. To address this issue, we develop a framework through which a deep RL agent learns to generalize policies from smaller, s…

A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space …

Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of the agent's policy can affect its exploration capabilities. First, we design EscapeRoom environments, where the agent must figure out how to n…

Advances in the field of inverse reinforcement learning (IRL) have led to sophisticated inference frameworks that relax the original modeling assumption of observing an agent behavior that reflects only a single intention. Instead of learning a global behavioral model, recent IRL methods divide the demonstration data i…

Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive N…

Deep learning optimizes VWAP strategy for lower transaction costs.

This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options framework and demonstrate that they work well together and indeed it is only after one combines the two that the full benefit of each is re…

Prediction is arguably one of the most basic functions of an intelligent system. In general, the problem of predicting events in the future or between two waypoints is exceedingly difficult. However, most phenomena naturally pass through relatively predictable bottlenecks---while we cannot predict the precise trajector…

Learning robust value functions given raw observations and rewards is now possible with model-free and model-based deep reinforcement learning algorithms. There is a third alternative, called Successor Representations (SR), which decomposes the value function into two components -- a reward predictor and a successor ma…

We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to …

Building systems that autonomously create temporal abstractions from data is a key challenge in scaling learning and planning in reinforcement learning. One popular approach for addressing this challenge is the options framework (Sutton et al., 1999). However, only recently in (Bacon et al., 2017) was a policy gradient…

Learning to solve complex goal-oriented tasks with sparse terminal-only rewards often requires an enormous number of samples. In such cases, using a set of expert trajectories could help to learn faster. However, Imitation Learning (IL) via supervised pre-training with these trajectories may not perform as well and gen…

In many real-world scenarios, an autonomous agent often encounters various tasks within a single complex environment. We propose to build a graph abstraction over the environment structure to accelerate the learning of these tasks. Here, nodes are important points of interest (pivotal states) and edges represent feasib…

Many decision-making problems naturally exhibit pronounced structures inherited from the characteristics of the underlying environment. In a Markov decision process model, for example, two distinct states can have inherently related semantics or encode resembling physical state configurations. This often implies locall…

Goal-conditioned policies are used in order to break down complex reinforcement learning (RL) problems by using subgoals, which can be defined either in state space or in a latent feature space. This can increase the efficiency of learning by using a curriculum, and also enables simultaneous learning and generalization…

Director learns hierarchical behaviors from pixels, outperforming exploration methods.

Unified framework for multi-objective curriculum learning in robotics.

New model shows hierarchical proof structure helps theorem provers.

Extended Predictable Feature Analysis (PFAx) [Richthofer and Wiskott, 2017] is an extension of PFA [Richthofer and Wiskott, 2015] that allows generating a goal-directed control signal of an agent whose dynamics has previously been learned during a training phase in an unsupervised manner. PFAx hardly requires assumptio…

Paper defines generalized braids and proves their subgroup status.

Defines a new Poisson structure for generalized Sasakian spaces.

Improved image generation through iterative flow matching to reduce hallucinations.

Framework generates personalized insulin treatment strategies using deep models.

OptiGAN uses GAN and RL to optimize sequence generation for specific goals.

Survey on deep models for graph generation.

Improves deep generative models to generate images of any size.

Develops a unified theory of Yang-Mills and GR using generalized principal bundles.

Meta-CoTGAN improves adversarial text generation by preventing mode collapse.

Generative models can still learn from contaminated data, but with limitations.

Generative AI tasks analyzed for text, images, audio, video, code, and molecules.

Defines Kahler angle for a broader context.

Established a generalized Boothby-Wang theorem in contact geometry.

The twistor construction for Riemannian manifolds is extended to the case of manifolds endowed with generalized metrics (in the sense of generalized geometry à la Hitchin). The generalized twistor space associated to such a manifold is defined as the bundle of generalized complex structures on the tangent spaces of the…

We present a characterization, in terms of torsion-free generalized connections, for the integrability of various generalized structures (generalized almost complex structures, generalized almost hypercomplex structures, generalized almost Hermitian structures and generalized almost hyper-Hermitian structures) defined …

This research proves guarantees on sequence models' generalization to longer and novel sequences.

SHADOWCAST generates graphs with user-specified attributes.

In this paper we define the notion of a generalized coKähler structure and prove that the product $M_{1}\times M_{2}$ of generalized contact metric manifolds $(M_i, Φ_i,E_{\pm,i}, G_i)$, $i=1, 2$, where $M_{1}\times M_{2}$ is endowed with the product generalized complex structure induced from $Φ_1$ and $Φ_2$, is gener…

We define the generalized connected sum for generic closed plane curves, generalizing the strange sum defined by Arnold, and completely describe how the Arnold invariants $J^{\pm}$ and $\mathit{St}$ behave under the generalized connected sums.

In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of en…

A new method generates graphs with hierarchical structures.

The paper extends symplectic techniques to generalized complex geometry.