arXiv research

Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hyb…

Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete acti…

Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions. Motivated by techniques from amortized inference, we replace the expensive maximization over all actions with a maximization over a small subset of possible actions sam…

We use the combinatorial harmonic map theory to study the isometric actions of discrete groups on Hadamard spaces. Given a finitely generated group acting by automorphisms, properly discontinuously and cofinitely on a simplicial complex and its isometric action on a Hadamard space, we formulate criterions for the actio…

In this article, we sketch an algorithm that extends the Q-learning algorithms to the continuous action space domain. Our method is based on the discretization of the action space. Despite the commonly used discretization methods, our method does not increase the discretized problem dimensionality exponentially. We wil…

New spaces found without certain actions, using special subgroups.

Introduces Conditional Action Trees to simplify RL action spaces.

This paper improves MADDPG's performance in discrete grid-world scenarios.

Hybrid Policy Optimization tackles reinforcement learning in hybrid spaces, improving performance over PPO.

Local-to-global principle for Morse actions on symmetric spaces.

Let G be a group and let M be a CAT(0) proper metric space (e.g. a simply connected complete Riemannian manifold of non-positive sectional curvature or a locally finite tree). Isometric actions of G on M are (by definition) points in the space R := Hom(G, Isom(M)) with the compact open topology. Sample theorems: 1. The…

Adaptive discretization improves model-based RL in large spaces.

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinf…

We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel $Q$-learning policy with adaptive data-driven discretization. The central idea is to maintain a finer partition of the state-action space in regions w…

Hybrid RL method optimizes trading by balancing continuous and discrete actions.

Study shows how certain spaces can be mapped to R^n with specific properties.

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…

Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (…

For covering spaces and properly discontinuous actions with compatible diffusion processes, we discuss Lyons-Sullivan discretizations of the processes and the associated function theory.

Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Recommender systems, industrial plants and language models are only some of the many real-world tasks involving large numbers of discrete actions for which curren…

We survey some results and questions about free actions of infinite groups on products of spheres and euclidean spaces, and give some new co-compact examples.

For covering spaces and properly discontinuous actions with compatible diffusion operators, we discuss Lyons-Sullivan discretizations of the associated diffusions and harmonic functions of bounded growth.

Investigates offline RL in factorisable action spaces, overcoming overestimation bias.

This is the second of two papers in which we prove that a cell model of the moduli space of curves with marked points and tangent vectors at the marked points acts on the Hochschild co--chains of a Frobenius algebra. We also prove that a there is dg--PROP action of a version of Sullivan Chord diagrams which acts on the…

We formalize an equivariant version of Bestvina-Brady discrete Morse theory, and apply it to Vietoris-Rips complexes in order to exhibit finite universal spaces for proper actions for all asymptotically CAT(0) groups.

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep $\mathcal{Q}$-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the …

The paper tackles multi-player information asymmetry bandits in metric spaces.

Let G/H be a strongly regular homogeneous space such that H is a Lie group of inner type. We show that G/H admits a proper action of a discrete non-virtually abelian subgroup of G if and only if G/H admits a proper action of a subgroup L of G locally isomorphic to SL(2,R). We classify all such spaces.

We extend the notion of Novikov-Shubin invariant for free G-CW-complexes of finite type to spaces with arbitrary G-actions and prove some statements about their positivity. In particular we apply this to classifying spaces of discrete groups.

We study the geometry and dynamics of discrete infinite covolume subgroups of higher rank semisimple Lie groups. We introduce and prove the equivalence of several conditions, capturing "rank one behavior'' of discrete subgroups of higher rank Lie groups. They are direct generalizations of rank one equivalents to convex…

Paper solves POMDPs in continuous time and discrete spaces.

This article gives an up-to-date account of the theory of discrete group actions on non-Riemannian homogeneous spaces. As an introduction of the motifs of this article, we begin by reviewing the current knowledge of possible global forms of pseudo-Riemannian manifolds with constant curvatures, and discuss what kind of …

Reinforcement learning (RL) has recently been introduced to interactive recommender systems (IRS) because of its nature of learning from dynamic interactions and planning for long-run performance. As IRS is always with thousands of items to recommend (i.e., thousands of actions), most existing RL-based methods, however…

Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and so here we derive an alternative version of the Soft Actor-Critic algorithm that is applicable to dis…

In this paper we propose a process of lagrangian reduction and reconstruction for nonholonomic discrete mechanical systems where the action of a continuous symmetry group makes the configuration space a principal bundle. The result of the reduction process is a discrete dynamical system that we call the discrete reduce…

Stochastic Q-learning tackles large action spaces with reduced computation.

Generalizing a classical theorem of Carlson and Toledo, we prove that any Zariski dense isometric action of a Kähler group on the real hyperbolic space of dimension at least 3 factors through a homomorphism onto a cocompact discrete subgroup of PSL(2,R). We also study actions of Kähler groups on infinite dimensional re…

Study proves duality in exotic option pricing under uncertain model and delayed information.

A 2-manifold's group structure is deduced from orbit configuration spaces.

A core novelty of Alpha Zero is the interleaving of tree search and deep learning, which has proven very successful in board games like Chess, Shogi and Go. These games have a discrete action space. However, many real-world reinforcement learning domains have continuous action spaces, for example in robotic control, na…

Groups can act on $L^p$ spaces with non-trivial cohomology.

The theme of this survey is that subgroups of the mapping class group of a finite type surface S can be studied via the geometric/dynamical properties of their action on the Thurston compactification of the Teichmuller space of S, just as discrete subgroups of the isometries of hyperbolic space can be studied via their…

We generalize Sunada's method to produce new examples of closed, locally non-isometric manifolds which are isospectral. In particular, we produce pairs of isospectral, simply-connected, locally non-isometric normal homogeneous spaces. These pairs also allow us to see that in general group actions with discrete spectra …

We study discrete, cocompact, isometric actions of groups on Hadamard spaces, and the induced actions on ideal boundaries. For a class of groups generalizing fundamental groups of three-dimensional graph manifolds, we find a set of invariants for the action which determine the boundary action up to equivariant homeomor…

We begin by showing that commensurators of Zariski dense subgroups of isometry groups of symmetric spaces of non-compact type are discrete provided that the limit set on the Furstenberg boundary is not invariant under the action of a (virtual) simple factor. In particular for rank one or simple Lie groups, Zariski dens…

Resolves singular fibers of a 5-manifold with $S^1$ action.

In this short note, we prove that the space of all admissible piecewise linear metrics parameterized by length square on a triangulated manifolds is a convex cone. We further study Regge's Einstein-Hilbert action and give a much more reasonable definition of discrete Einstein metric than our former version in \cite{G}.…

ZoomRL learns efficient strategies for large state-action spaces using a metric.