arXiv research

Hybrid actor-critic learns in complex action spaces.

We explore Deep Reinforcement Learning in a parameterized action space. Specifically, we investigate how to achieve sample-efficient end-to-end training in these tasks. We propose a new compact architecture for the tasks where the parameter policy is conditioned on the output of the discrete action policy. We also prop…

New algorithms tackle multi-agent problems with hybrid action spaces.

Machine learning uses invariant theory to restrict function classes.

We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards. Consequently, observing a particular state transition might yield useful information about other, unobserved, parts of the MDP. We present a…

Hybrid SAC improves RL for video games with discrete, continuous actions.

We propose a new sample-efficient methodology, called Supervised Policy Update (SPU), for deep reinforcement learning. Starting with data generated by the current policy, SPU formulates and solves a constrained optimization problem in the non-parameterized proximal policy space. Using supervised regression, it then con…

In recent years, deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games. Many reinforcement learning problems, however, involve high-dimensional discrete action spaces as well as high-dimensional state spaces. This pa…

For a Riemannian submersion from a simple compact Lie group with a bi-invariant metric, we prove the action of its holonomy group on the fibers is transitive. As a step towards classifying Riemannian submersions with totally geodesic fibers, we consider the parameterized surface induced by lifting a base geodesic to po…

ARM policy gradient reduces variance for binary actions.

New method learns policies without limiting to Gaussian distributions.

Paper presents an efficient exploration method for reinforcement learning.

Generically learns movement control policies from exploration data.

The recently proposed option-critic architecture Bacon et al. provide a stochastic policy gradient approach to hierarchical reinforcement learning. Specifically, they provide a way to estimate the gradient of the expected discounted return with respect to parameters that define a finite number of temporally extended ac…

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms, which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recent…

Math verifies Aganagic's proposal for Khovanov homology.

In this short note, we prove that the space of all admissible piecewise linear metrics parameterized by length square on a triangulated manifolds is a convex cone. We further study Regge's Einstein-Hilbert action and give a much more reasonable definition of discrete Einstein metric than our former version in \cite{G}.…

hyperSBINN improves drug cardiosafety assessment by efficiently modeling cardiac action potentials.

In this paper we describe the space of maximal components of the character variety of surface group representations into PSp(4,R) and Sp(4,R). For every rank 2 real Lie group of Hermitian type, we construct a mapping class group invariant complex structure on the maximal components. For the groups PSp(4,R) and Sp(4,R),…

Policy gradient converges linearly with Hadamard parameterization in tabular settings.

This is a survey of the theory of complex projective (CP^1) structures on compact surfaces. After some preliminary discussion and definitions, we concentrate on three main topics: (1) Using the Schwarzian derivative to parameterize the moduli space (2) Thurston's parameterization of the moduli space using grafting (3) …

Continuous control imitation learning fails if expert actions are smooth.

This paper improves reinforcement learning efficiency for large-scale MDPs.

The paper explores universal circles for Anosov foliations and their uniqueness.

Develops a new reinforcement learning framework for complex control problems.

The paper explores three methods to assign a metric to shape spaces.

Local PCA detects intrinsic parameterization of complex thermo-chemical state-spaces.

This paper identifies drift Lipschitz budget K as key to diffusion policy expressivity and statistical trade-offs.

We consider Markov Decision Problems defined over continuous state and action spaces, where an autonomous agent seeks to learn a map from its states to actions so as to maximize its long-term discounted accumulation of rewards. We address this problem by considering Bellman's optimality equation defined over action-val…

Study on rotational surfaces in de Sitter space with specific curvature conditions.

New algorithm for Q-learning in MDPs with infinite states and actions.

We prove generic regularity and Uhlenbeck-type compactification theorems for the moduli spaces of PU(2)-monopoles. Generic regularity is NOT obtained in the usual way (by applying Sard theorem to a smooth parameterized moduli space), since the parameterized moduli space can be a priori singular. We explain why, using t…

We study the pull-back of the 2-parameter family of quotient elastic metrics introduced in Mio-Srivastava-Joshi on the space of arc-length parameterized loops. This point of view has the advantage of concentrating on the manifold of arc-length parameterized curves, which is a very natural manifold when the analysis of …

We show that the real-valued function $S_α$ on the moduli space $\mathcal{M}_{0,n}$ of pointed rational curves, defined as the critical value of the Liouville action functional on a hyperbolic 2-sphere with $n\geq 3$ conical singularities of arbitrary orders $α=\{α_1,...,α_n\}$, generates accessory parameters of the as…

Geometric Occam's Razor shapes deep learning solutions.

We study a set $\mathcal{M}_{K,N}$ parameterizing filtered $SL(K)$-Higgs bundles over $\mathbb{CP}^1$ with an irregular singularity at $z = \infty$, such that the eigenvalues of the Higgs field grow like $\lvert λ\rvert \sim \lvert z ^{N/K} \mathrm{d} z \rvert$, where $K$ and $N$ are coprime. $\mathcal{M}_{K,N}$ carrie…

Method converts neural networks to function space for better uncertainty quantification.

New geometric interpretation explains over-parameterized models and adversarial perturbations.

Visualizes movement control optimization landscapes to understand why it's hard and how to make it easier.

Translation surfaces can be defined in an elementary way via polygons, and arise naturally in in the study of various basic dynamical systems. They can also be defined as Abelian differentials on Riemann surfaces, and have moduli spaces called strata that are related to the moduli space of Riemann surfaces. There is a …

The paper studies holomorphic curves in a pseudo-Riemannian space and their moduli space.

Paper proposes an efficient RL algorithm for discounted MDPs using feature mapping.

Kirchhoff energy is a classical functional on the space of arclength-parameterized framed curves whose critical points approximate configurations of springy elastic rods. We introduce a generalized functional on the space of framed curves of arbitrary parameterization, which model rods with axial stretch or cross-secti…

We establish the existence of an integer degree for the natural projection map from the space of parameterizations of asymptotically conical self-expanders to the space of parameterizations of the asymptotic cones when this map is proper. As an application we show that there is an open set in the space of cones in the …

HOPE improves SSMs for long-memory tasks with robust initialization and training.

We present a classification of compact Kaehler manifolds admitting a hamiltonian 2-form (which were classified locally in part I of this work). This involves two components of independent interest. The first is the notion of a rigid hamiltonian torus action. This natural condition, for torus actions on a Kaehler manifo…

Generalizes G-opers for arbitrary parabolics, parameterizing by Hitchin base.