arXiv research

Director learns hierarchical behaviors from pixels, outperforming exploration methods.

The ability to use a 2D map to navigate a complex 3D environment is quite remarkable, and even difficult for many humans. Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community. In this paper we teach a reinforce…

A simple DQN-based multi-agent RL system for binary actions.

In many real-world scenarios, an autonomous agent often encounters various tasks within a single complex environment. We propose to build a graph abstraction over the environment structure to accelerate the learning of these tasks. Here, nodes are important points of interest (pivotal states) and edges represent feasib…

Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiability of the entire architecture. Despite their effectiveness, they suffer from several disadvantages including training instability, random …

Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions f…

Deep network solves maze path planning without training.

MAZE attacks a model without data, using synthetic inputs.

In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, invol…

Logical neural networks solve mazes by filling dead ends, but not all methods generalize well.

Entropy regularization is used to get improved optimization performance in reinforcement learning tasks. A common form of regularization is to maximize policy entropy to avoid premature convergence and lead to more stochastic policies for exploration through action space. However, this does not ensure exploration in th…

Unified approach to verify NN properties using ReLU's unique polytope structure.

We introduce a conceptually simple and scalable framework for continual learning domains where tasks are learned sequentially. Our method is constant in the number of parameters and is designed to preserve performance on previously encountered tasks while accelerating learning progress on subsequent problems. This is a…

We study smooth {\sf traversing} vector fields $v$ on compact manifolds $X$ with boundary. A traversing $v$ admits a Lyapunov function $f: X \to \Bbb R$ such that $df(v) > 0$. We show that the trajectory spaces $\mathcal T(v)$ of {\sf traversally generic} $v$-flows are {\sf Whitney stratified spaces}, and thus admit tr…

This paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult…

HRM-Agent learns to navigate dynamic mazes using reinforcement learning.

In this paper we approach two relevant deep learning topics: i) tackling of graph structured input data and ii) a better understanding and analysis of deep networks and related learning algorithms. With this in mind we focus on the topological classification of reachability in a particular subset of planar graphs (Maze…

PBCS combines RL and motion planning for better exploration.

Euler's theorem extended to complex structures.

Improves latent space structure for better data representation.

A novel multi-resolution Gaussian process model for efficient time traversal.

Deep Reinforcement Learning has shown tremendous success in solving several games and tasks in robotics. However, unlike humans, it generally requires a lot of training instances. Trajectories imitating to solve the task at hand can help to increase sample-efficiency of deep RL methods. In this paper, we present a simp…

Many tasks in computer vision can be cast as a "label changing" problem, where the goal is to make a semantic change to the appearance of an image or some subject in an image in order to alter the class membership. Although successful task-specific methods have been developed for some label changing applications, to da…

New RL approach builds short ancestral recombination graphs.

Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…

AutoDIME automates design of multi-agent environments for RL.

We combine Gromov's amenable localization technique with the Poincaré duality to study the traversally generic vector flows on smooth compact manifolds $X$ with boundary. Such flows generate well-understood stratifications of $X$ by the trajectories that are tangent to the boundary in a particular canonical fashion. Sp…

CSAC enables cooperative reinforcement learning for multi-stage tasks.

Any traversally generic vector flow on a compact manifold $X$ with boundary leaves some residual structure on its boundary $\d X$. A part of this structure is the flow-generated causality map $C_v$, which takes a region of $\d X$ to the complementary region. By the Holography Theorem from \cite{K4}, the map $C_v$ allow…

Unsupervised representation learning has succeeded with excellent results in many applications. It is an especially powerful tool to learn a good representation of environments with partial or noisy observations. In partially observable domains it is important for the representation to encode a belief state, a sufficie…

This paper introduces a new approach to active inference using constrained Bethe Free Energy.

Counterfactual regret minimization (CFR) is the most popular algorithm on solving two-player zero-sum extensive games with imperfect information and achieves state-of-the-art performance in practice. However, the performance of CFR is not fully understood, since empirical results on the regret are much better than the …

Predicts node sequences in graphs using multi-order network models.

Let $X$ be a compact smooth manifold with boundary. In this article, we study the spaces $\mathcal V^\dagger(X)$ and $\mathcal V^\ddagger(X)$ of so called boundary generic and traversally generic vector fields on $X$ and the place they occupy in the space $\mathcal V(X)$ of all fields (see Theorems \ref{th3.4} and Theo…

ERM uses energy-based selection to improve recursive reasoning.

Attention-based encoder decoder network uses a left-to-right beam search algorithm in the inference step. The current beam search expands hypotheses and traverses the expanded hypotheses at the next time step. This traversal is implemented using a for-loop program in general, and it leads to speed down of the recogniti…

A colored graph is a directed graph in which nodes or edges have been assigned colors that are not necessarily unique. Observability problems in such graphs consider whether an agent observing the colors of edges or nodes traversed on a path in the graph can determine which node they are at currently or which nodes wer…

Method generates counterfactual explanations for graph classifiers.

The Morris Water Maze is commonly used in behavioural neuroscience for the study of spatial learning with rodents. Over the years, various methods of analysing rodent data collected in this task have been proposed. These methods span from classical performance measurements (e.g. escape latency, rodent speed, quadrant p…

SURF steers scalarization weights to uniformly traverse the Pareto front.

In low-dimensional topology, many important decision algorithms are based on normal surface enumeration, which is a form of vertex enumeration over a high-dimensional and highly degenerate polytope. Because this enumeration is subject to extra combinatorial constraints, the only practical algorithms to date have been v…

Let $C_{k_1}^{k_2}$ denote the set of all closed curves of class $C^r$ on the sphere $S^2$ whose geodesic curvatures are restricted to lie in $(k_1,k_2)$, furnished with the $C^r$ topology (for some $r >= 2$ and possibly infinite $k_1 < k_2$). In 1970, J. Little proved that the space $C_0^{+\infty}$ of closed curves ha…

This paper is the third in a series that researches the Morse Theory, gradient flows, concavity and complexity on smooth compact manifolds with boundary. Employing the local analytic models from \cite{K2}, for \emph{traversally generic flows} on $(n+1)$-manifolds $X$, we embark on a detailed and somewhat tedious study …

We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime. Recent work of \citep{Gale2019, Liu2018} has shown that sparse ResNet-50 architectures trained on ImageNet-2012 dataset converge to solutions that a…

New algorithm minimizes FE objectives for synthetic AIF agents.

Graph algorithms are key tools in many fields of science and technology. Some of these algorithms depend on propagating information between distant nodes in a graph. Recently, there have been a number of deep learning architectures proposed to learn on undirected graphs. However, most of these architectures aggregate i…

MB-DQN uses different backup lengths for improved reinforcement learning.

We study knots in 3d Chern-Simons theory with complex gauge group $SL(N,\mathbb{C})$, in the context of its relation with 3d $\mathcal{N}=2$ theory (the so-called 3d-3d correspondence). The defect has either co-dimension 2 or co-dimension 4 inside the 6d $(2,0)$ theory, which is compactified on a 3-manifold $\hat{M}$. …