arXiv research

Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete acti…

Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and so here we derive an alternative version of the Soft Actor-Critic algorithm that is applicable to dis…

Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hyb…

In this article, we sketch an algorithm that extends the Q-learning algorithms to the continuous action space domain. Our method is based on the discretization of the action space. Despite the commonly used discretization methods, our method does not increase the discretized problem dimensionality exponentially. We wil…

We provide an action for gauge theories discretized on simplicial meshes, inspired by finite element methods. The action is discretely gauge invariant and we give a proof of consistency. A discrete Noether's theorem that can be applied to our setting, is also proved.

Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions. Motivated by techniques from amortized inference, we replace the expensive maximization over all actions with a maximization over a small subset of possible actions sam…

Groups can act on 3-manifolds if their Cayley complex can embed in specific types of 3-manifolds.

We use the combinatorial harmonic map theory to study the isometric actions of discrete groups on Hadamard spaces. Given a finitely generated group acting by automorphisms, properly discontinuously and cofinitely on a simplicial complex and its isometric action on a Hadamard space, we formulate criterions for the actio…

This paper improves MADDPG's performance in discrete grid-world scenarios.

Hybrid Policy Optimization tackles reinforcement learning in hybrid spaces, improving performance over PPO.

We propose here a new discretization method for a class continuum gauge theories which action functionnals are polynomials of the curvature. Based on the notion of holonomy, this discretization procedure appears gauge-invariant for discretized analogs of Yang-Mills theories, and hence gauge-fixing is fully rigorous for…

Let G be a group and let M be a CAT(0) proper metric space (e.g. a simply connected complete Riemannian manifold of non-positive sectional curvature or a locally finite tree). Isometric actions of G on M are (by definition) points in the space R := Hom(G, Isom(M)) with the compact open topology. Sample theorems: 1. The…

New spaces found without certain actions, using special subgroups.

We establish a necessary and sufficient condition for pairs of integers to arise as the weights at the fixed points of an effective circle action on a compact almost complex 4-manifold with a discrete fixed point set. As an application, we provide a necessary and sufficient condition for a pair of integers to arise as …

Study on 2-valued dynamics on complex plane, showing some dynamics can't be group actions.

New STDP rule for spiking neurons solves discrete action reinforcement learning tasks.

Study circle actions on unitary manifolds with discrete fixed points.

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinf…

Hybrid RL method optimizes trading by balancing continuous and discrete actions.

Local-to-global principle for Morse actions on symmetric spaces.

Introduces Conditional Action Trees to simplify RL action spaces.

Adaptive discretization improves model-based RL in large spaces.

New method constructs proper affine actions of groups in higher dimensions.

Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Recommender systems, industrial plants and language models are only some of the many real-world tasks involving large numbers of discrete actions for which curren…

We translate a classification scheme for periodic CMC surfaces developed by J. Dorfmeister and the author to discrete CMC surfaces in the sense of A. Bobenko and U. Pinkall. The scheme uses the dressing action on discrete CMC surfaces to arrive at a classification for periodic discrete CMC surfaces.

We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel $Q$-learning policy with adaptive data-driven discretization. The central idea is to maintain a finer partition of the state-action space in regions w…

For covering spaces and properly discontinuous actions with compatible diffusion processes, we discuss Lyons-Sullivan discretizations of the processes and the associated function theory.

In this technical note we give a purely geometric understanding of discrete torsion, as an analogue of orbifold Wilson lines for two-form tensor field potentials. In order to introduce discrete torsion in this context, we describe gerbes and the description of certain type II supergravity tensor field potentials as con…

Graphically discrete groups have strong rigidity properties.

For covering spaces and properly discontinuous actions with compatible diffusion operators, we discuss Lyons-Sullivan discretizations of the associated diffusions and harmonic functions of bounded growth.

We survey some results and questions about free actions of infinite groups on products of spheres and euclidean spaces, and give some new co-compact examples.

Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (…

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…

Study shows how certain spaces can be mapped to R^n with specific properties.

In this paper we propose a process of lagrangian reduction and reconstruction for nonholonomic discrete mechanical systems where the action of a continuous symmetry group makes the configuration space a principal bundle. The result of the reduction process is a discrete dynamical system that we call the discrete reduce…

Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in the discrete action ATARI domain, we demonstrate, for the first time, that low precision policy distillation from a high precision network pro…

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep $\mathcal{Q}$-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the …

Reinforcement learning (RL) has recently been introduced to interactive recommender systems (IRS) because of its nature of learning from dynamic interactions and planning for long-run performance. As IRS is always with thousands of items to recommend (i.e., thousands of actions), most existing RL-based methods, however…

Defines discrete symmetry of manifolds and proves bounds on its value.

Develops a statistical learning framework for personalized asset allocation.

This is the second of two papers in which we prove that a cell model of the moduli space of curves with marked points and tangent vectors at the marked points acts on the Hochschild co--chains of a Frobenius algebra. We also prove that a there is dg--PROP action of a version of Sullivan Chord diagrams which acts on the…

The action dimension of a discrete group $G$ is the minimum dimension of contractible manifold that admits a proper $G$-action. We compute the action dimension of the direct limit of a simple complex of groups for several classes of examples including: 1) Artin groups, 2) graph products of groups, and 3) fundamental gr…

We study two special cases of the equivariant index defined in part I of this series. We apply this index to deformations of Spin$^c$-Dirac operators, invariant under actions by possibly noncompact groups, with possibly noncompact orbit spaces. One special case is an index defined in terms of multiplicities of discrete…

We identify action representations from video data, proving their statistical benefits.

The paper extends invariant theory to non-compact and non-reductive actions, classifying four regimes.

Deep network predicts action sequences for complex tasks from a scene image.

A discrete conformal map (DCM) maps the square lattice to the Riemann sphere such that the image of every irreducible square has the same cross-ratio. This paper shows that every periodic DCM can be determined from spectral data (a hyperelliptic compact Riemann surface, called the spectral curve, equipped with some mar…

In this paper, we study a circle action on a compact oriented manifold with a discrete fixed point set. The fixed point data consists of the weights of the $S^1$-representations at the fixed points. We prove various results and properties of the action, in terms of the fixed point data. We show that the manifold can be…