arXiv research

A new benchmark task for evaluating policy learning in complex, high-dimensional action spaces.

Hybridizes CEM and gradient descent for efficient model-predictive control.

In recent years, deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games. Many reinforcement learning problems, however, involve high-dimensional discrete action spaces as well as high-dimensional state spaces. This pa…

AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.

Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among ac…

Study cyclic group actions on specific high-dimensional manifolds.

Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent ba…

Study on group actions on spheres using multisymplectic geometry.

New algorithm learns from sparse data to make decisions in high dimensions.

The paper proves rigidity for mapping class group actions on metrics of positive scalar curvature.

The paper extends a learning heuristic to high-dimensional contexts, reducing the risk of unusual actions.

We explicitly classify all pairs $(M,G)$, where $M$ is a connected complex manifold of dimension $n\ge 2$ and $G$ is a connected Lie group acting properly and effectively on $M$ by holomorphic transformations and having dimension $d_G$ satisfying $n^2+2\le d_G<n^2+2n$. These results extend -- in the complex case -- the…

New method reduces bias in high-dimensional action spaces for efficient reinforcement learning.

Mutual-information regularization improves RL algorithms, especially in high-dimensional domains.

Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

Imitation from observation (IfO) is the problem of learning directly from state-only demonstrations without having access to the demonstrator's actions. The lack of action information both distinguishes IfO from most of the literature in imitation learning, and also sets it apart as a method that may enable agents to l…

Algorithm learns diffusion processes with high-dimensional state spaces.

Improved CEM for fast real-time planning in high-dimensional control tasks.

This research tackles intervention-centric causal reasoning in learning agents by using meta-learning.

The paper analyzes Q-learning convergence rates with asynchronous updates.

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…

The paper calculates the value of information in high-dimensional decision making.

Paper tackles action selection in deep RL, proposing a data-driven approach.

A new method for sparse linear bandits reduces exploration-exploitation tradeoff.

High-dimensional representations often have a lower dimensional underlying structure. This is particularly the case in many decision making settings. For example, when the representation of actions is generated from a deep neural network, it is reasonable to expect a low-rank structure whereas conventional structures l…

A planning approach learns skills from interactions, balancing exploration and exploitation.

Action-bisimulation learns long-horizon controllability for reinforcement learning.

Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.

Jointly tackles assortment and pricing in retail, using bandit models.

We introduce the notion of an EZ-structure on a group. Delta-hyperbolic groups and CAT(0)-groups have EZ-structures. We show torsion-free groups having an EZ-structure automatically have an action by homeomorphisms on a closed (high-dimensional) ball, which is well-behaved away from a "bad limit set" in the boundary of…

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …

Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use param…

HashReward improves imitation learning in high-dimensional environments by balancing reward generation and dimensionality reduction.

Data-efficient reinforcement learning (RL) in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. We consider a particularly important instance of this challenge, the pixels-to-torques problem, where an RL agent learns a closed-loop con…

New RL method learns from state transitions without actions.

In this paper we continue to study actions of high-dimensional Lie groups on complex manifolds. We give a complete explicit description of all pairs $(M,G)$, where $M$ is a connected complex manifold $M$ of dimension $n\ge 2$, and $G$ is a connected Lie group of dimension $n^2+1$ acting effectively and properly on $M$ …

Study dynamic batch learning in high-dimensional sparse linear bandits.

RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.

Random small feature subsets outperform FS in diverse datasets.

Data-efficient learning in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. In this paper, we consider one instance of this challenge, the pixels to torques problem, where an agent must learn a closed-loop control policy from pixel i…

Develops a statistical learning framework for personalized asset allocation.

Successor Options discovers reusable skills using landmark states.

A new reinforcement learning method reduces action complexity for robust control.

Approximate linear programming (ALP) represents one of the major algorithmic families to solve large-scale Markov decision processes (MDP). In this work, we study a primal-dual formulation of the ALP, and develop a scalable, model-free algorithm called bilinear $π$ learning for reinforcement learning when a sampling or…

Minimum attention improves reinforcement learning performance in high-dimensional dynamics.

New model for bandit problem with linear rewards and side information.

Due to the capability of deep learning to perform well in high dimensional problems, deep reinforcement learning agents perform well in challenging tasks such as Atari 2600 games. However, clearly explaining why a certain action is taken by the agent can be as important as the decision itself. Deep reinforcement learni…