A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New findings on when to use action space exploration in reinforcement learning.
Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space explorat…
Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language. They are ideal environments for studying how to extend reinforcement learning agents to meet the challenges of natural language understanding, partial observability, and action generation in …
Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.
Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as -greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well …
A new method combines online and offline learning to tackle contextual bandits with missing action support.
Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …
The paper explores proper actions and their relation to representation theory, with new quantitative methods.
Generically learns movement control policies from exploration data.
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent …
New algorithm identifies optimal actions in large reward spaces efficiently.
Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristi…
Q-chunking improves RL for long tasks by chunking actions.
We address reinforcement learning problems with finite state and action spaces where the underlying MDP has some known structure that could be potentially exploited to minimize the exploration rates of suboptimal (state, action) pairs. For any arbitrary structure, we derive problem-specific regret lower bounds satisfie…
Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use param…
Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or …
The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as -greedy ex…
New RL approach uses future state and action visitation measures for better exploration.
Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore.…
Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension, making it challenging to apply existing on-policy gradient based deep RL algorithms efficiently. To effectively operate in multidimensional discrete acti…
The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.
SPAQL improves RL by adaptively partitioning state-action space and learning a time-invariant policy.
Research explores the space-like embeddings in pseudo-hyperbolic space, finding geometric frames and actions.
New algorithms improve contextual bandits with neural networks and energy models.
We explore Deep Reinforcement Learning in a parameterized action space. Specifically, we investigate how to achieve sample-efficient end-to-end training in these tasks. We propose a new compact architecture for the tasks where the parameter policy is conditioned on the output of the discrete action policy. We also prop…
We study a formulation of the standard Poisson sigma model in which the target space Poisson manifold carries the Hamilton action of some finite dimensional Lie algebra. We show that the structure of the action and the properties of the gauge invariant observables can be understood in terms of the associated target spa…
Lower bounds on eigenspectrum show rich action spaces force polynomial regret in linear bandits.
Optimal control in latent factor models uses Tsallis entropy for exploration.
In recent years, deep reinforcement learning has been shown to be adept at solving sequential decision processes with high-dimensional state spaces such as in the Atari games. Many reinforcement learning problems, however, involve high-dimensional discrete action spaces as well as high-dimensional state spaces. This pa…
BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.
New method uses hindsight to make exploration robust in stochastic environments.
This paper proposes an exploration method for deep reinforcement learning based on parameter space noise. Recent studies have experimentally shown that parameter space noise results in better exploration than the commonly used action space noise. Previous methods devised a way to update the diagonal covariance matrix o…
Any nonpositively curved symmetric space admits a topological compactification, namely the Hadamard compactification. For rank one spaces, this topological compactification can be endowed with a differentiable structure such that the action of the isometry group is differentiable. Moreover, the restriction of the actio…
Intrinsically motivated goal exploration processes enable agents to autonomously sample goals to explore efficiently complex environments with high-dimensional continuous actions. They have been applied successfully to real world robots to discover repertoires of policies producing a wide diversity of effects. Often th…
Agent uses message passing to optimize robot navigation, balancing exploration and exploitation.
In multi-agent reinforcement learning, discovering successful collective behaviors is challenging as it requires exploring a joint action space that grows exponentially with the number of agents. While the tractability of independent agent-wise exploration is appealing, this approach fails on tasks that require elabora…
Maximizes Rényi entropy for efficient exploration in reward-free RL.
New findings show optimization is crucial for OPL in large action spaces.
Explores new perspectives in transverse index theory for Lie group actions.
DE is a new exploration method that limits resource usage based on expected improvement and surprise.
In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…
A new method improves actor-critic RL by integrating HMC, enhancing policy distribution and exploration.
Configuration spaces for computer systems can be challenging for traditional and automatic tuning strategies. Injecting task-specific knowledge into the tuner for a task may allow for more efficient exploration of candidate configurations. We apply this idea to the task of index set selection to accelerate database wor…
UTE improves reinforcement learning by measuring action uncertainty, enhancing policy learning efficiency.
Robots learn actions and language through curiosity-driven self-exploration.
Randomized exploration in linear bandits achieves optimal regret bounds.
Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performa…
A new method for sparse linear bandits reduces exploration-exploitation tradeoff.