arXiv research

One problem in the application of reinforcement learning to real-world problems is the curse of dimensionality on the action space. Macro actions, a sequence of primitive actions, have been studied to diminish the dimensionality of the action space with regard to the time axis. However, previous studies relied on human…

A new method learns action representations for reinforcement learning.

UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.

Learning how to act when there are many available actions in each state is a challenging task for Reinforcement Learning (RL) agents, especially when many of the actions are redundant or irrelevant. In such cases, it is sometimes easier to learn which actions not to take. In this work, we propose the Action-Elimination…

New reinforcement learning framework for adapting to new actions.

Intelligent agents can learn to represent the action spaces of other agents simply by observing them act. Such representations help agents quickly learn to predict the effects of their own actions on the environment and to plan complex action sequences. In this work, we address the problem of learning an agent's action…

Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions. Motivated by techniques from amortized inference, we replace the expensive maximization over all actions with a maximization over a small subset of possible actions sam…

In many practical problems, a learning agent may want to learn the best action in hindsight without ever taking a bad action, which is significantly worse than the default production action. In general, this is impossible because the agent has to explore unknown actions, some of which can be bad, to learn better action…

Most model-free reinforcement learning methods leverage state representations (embeddings) for generalization, but either ignore structure in the space of actions or assume the structure is provided a priori. We show how a policy can be decomposed into a component that acts in a low-dimensional space of action represen…

Solves action selection for large spaces in RL, achieving near-optimal performance.

Paper proposes Roweisposes for 3D action recognition using generalized eigenvalue problem.

Regularizes predictions to encourage beneficial user actions.

The paper investigates the effects of invalid action masking in policy gradient algorithms.

Imitation learning targets deriving a mapping from states to actions, a.k.a. policy, from expert demonstrations. Existing methods for imitation learning typically require any actions in the demonstrations to be fully available, which is hard to ensure in real applications. Though algorithms for learning with unobservab…

DFL framework improves action and outcome fairness in policy learning.

New algorithm optimizes online decision-making with dynamically generated actions.

Introduces Conditional Action Trees to simplify RL action spaces.

A new RL paradigm reduces state-action-value function approximation inefficiency.

New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.

From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Re…

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent …

Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hyb…

A new RL approach learns near-equivalent actions for healthcare decisions.

Unified reinforcement learning and stochastic processes with action-driven processes.

In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured action space into simpler action spaces along with a critic network to guide the training of all sub-acto…

New findings on when to use action space exploration in reinforcement learning.

Parameterised actions in reinforcement learning are composed of discrete actions with continuous action-parameters. This provides a framework for solving complex domains that require combining high-level actions with flexible control. The recent P-DQN algorithm extends deep Q-networks to learn over such action spaces. …

Machine Learning algorithms are typically regarded as appropriate optimization schemes for minimizing risk functions that are constructed on the training set, which conveys statistical flavor to the corresponding learning problem. When the focus is shifted on perception, which is inherently interwound with time, recent…

Stochastic Q-learning tackles large action spaces with reduced computation.

Paper learns meaningful state and action representations from MDP trajectories.

Simplifies large action space bandits by selecting representative actions.

MAGE optimizes policies using action gradients from model-based learning.

This work tackles large action spaces in RL by binarizing actions.

Unified Bayesian framework for efficient off-policy evaluation and learning in large action spaces.

Proposes a new algorithm for accurate tree-based models with guaranteed recourse actions.

A textbook on machine learning explaining patterns, predictions, and actions.

RODE learns roles to simplify multi-agent tasks.

New algorithm for context bandits with continuous actions.

Q-chunking improves RL for long tasks by chunking actions.

Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete acti…

Representation of human actions as a sequence of human body movements or action attributes enables the development of models for human activity recognition and summarization. We present an extension of the low-rank representation (LRR) model, termed the clustering-aware structure-constrained low-rank representation (CS…

Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among ac…

A new benchmark task for evaluating policy learning in complex, high-dimensional action spaces.

New method QMLE performs well in complex action spaces without policy gradients.

Action-bisimulation learns long-horizon controllability for reinforcement learning.

Improved Q-learning for multi-agent reinforcement learning by weighting joint action values.

New method learns robot actions from videos without explicit labels.

New RL method learns from state transitions without actions.