A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithm for context bandits with continuous actions.
Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hyb…
Study on 2-valued dynamics on complex plane, showing some dynamics can't be group actions.
Any continuous action of SL(n,Z), where n > 2, on a r-dimensional mod 2 homology sphere factors through a finite group action if r < n - 1. In particular, any continuous action of SL(n+2,Z) on the n-dimensional sphere factors through a finite group action.
A core novelty of Alpha Zero is the interleaving of tree search and deep learning, which has proven very successful in board games like Chess, Shogi and Go. These games have a discrete action space. However, many real-world reinforcement learning domains have continuous action spaces, for example in robotic control, na…
Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions. Motivated by techniques from amortized inference, we replace the expensive maximization over all actions with a maximization over a small subset of possible actions sam…
Smooth approximations for continuous functions on orbit spaces.
The paper tackles counterfactual learning for stochastic policies with continuous actions.
Novel framework proves fast RL convergence in continuous spaces.
Let and be two -dimensional smooth manifolds with boundary. Suppose we glue and along some boundary components (which are, therefore, diffeomorphic). Call the result If we have a group acting continuously on and also acting continuously on such that the actions are comp…
Continuous reinforcement learning such as DDPG and A3C are widely used in robot control and autonomous driving. However, both methods have theoretical weaknesses. While DDPG cannot control noises in the control process, A3C does not satisfy the continuity conditions under the Gaussian policy. To address these concerns,…
New method extends low-rank MDPs to continuous action spaces.
Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algor…
Efficient algorithms for contextual bandits with smooth regret in continuous action spaces.
Temporal-difference (TD) networks are a class of predictive state representations that use well-established TD methods to learn models of partially observable dynamical systems. Previous research with TD networks has dealt only with dynamical systems with finite sets of observations and actions. We present an algorithm…
Proper actions on bornological spaces are characterized with compatible coarse structures.
Hybrid RL method optimizes trading by balancing continuous and discrete actions.
Hybrid Policy Optimization tackles reinforcement learning in hybrid spaces, improving performance over PPO.
Paper tackles RL with continuous actions and unmeasured confounders.
We show that every continuous action of a finite group on a smooth three-manifold is a uniform limit of smooth actions.
Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinf…
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…
The paper explores properties of continuous actions on manifolds, proving bounds on subgroup size and fixed points.
New proof shows path-connectedness of actions on intervals and circles.
Action chunking and data exploration improve behavior cloning in robotics.
The paper proposes a method to learn the structure of continuous-action games with non-parametric utilities using a limited number of samples.
Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among ac…
RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.
The paper extends Thompson Sampling to infinite action spaces using information theory.
New framework discovers non-affine continuous symmetries in neural networks.
In this paper we study perturbations of constant cocycles for actions of higher rank semi-simple algebraic groups and their lattices. Roughly speaking, for ergodic actions, Zimmer's cocycle superrigidity theorems implies that the perturbed cocycle is measurably conjugate to a constant cocycle modulo a compact valued co…
Models analyze strategic risk-taking in continuous action games.
Develops a statistical learning framework for personalized asset allocation.
Proposes a value-based method for continuous control without an actor.
Reinforcement Learning algorithms have recently been proposed to learn time-sequential control policies in the field of autonomous driving. Direct applications of Reinforcement Learning algorithms with discrete action space will yield unsatisfactory results at the operational level of driving where continuous control a…
We study contextual bandit learning with an abstract policy class and continuous action space. We obtain two qualitatively different regret bounds: one competes with a smoothed version of the policy class under no continuity assumptions, while the other requires standard Lipschitz assumptions. Both bounds exhibit data-…
Develops a new RL algorithm for medical treatment regimes.
Safe imitation learning with a safety layer for flexible training.
This paper proposes a novel deep reinforcement learning architecture that was inspired by previous tree structured architectures which were only useable in discrete action spaces. Policy Prediction Network offers a way to improve sample complexity and performance on continuous control problems in exchange for extra com…
Practical algorithm for contextual bandits with large action spaces.
In this paper, we study the problem of learning the set of pure strategy Nash equilibria and the exact structure of a continuous-action graphical game with quadratic payoffs by observing a small set of perturbed equilibria. A continuous-action graphical game can possibly have an uncountable set of Nash euqilibria. We p…
Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (…
In this article, we sketch an algorithm that extends the Q-learning algorithms to the continuous action space domain. Our method is based on the discretization of the action space. Despite the commonly used discretization methods, our method does not increase the discretized problem dimensionality exponentially. We wil…
Study a continuous-time PA problem with private effort and consumption decisions.
It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequ…
We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models. It is efficient because (1) local models are estimated only when the planner requires them; (2) the planner focuses on the most relevant states to the current planni…
Continuous control imitation learning fails if expert actions are smooth.
A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action , and (i) with probability , an alt…