arXiv research

New findings on when to use action space exploration in reinforcement learning.

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space explorat…

Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $ε$-greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well …

Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore.…

Novelty search in low-dimensional space improves sample efficiency in exploration tasks.

This paper proposes an exploration method for deep reinforcement learning based on parameter space noise. Recent studies have experimentally shown that parameter space noise results in better exploration than the commonly used action space noise. Previous methods devised a way to update the diagonal covariance matrix o…

Contextual bandit learning is an increasingly popular approach to optimizing recommender systems via user feedback, but can be slow to converge in practice due to the need for exploring a large feature space. In this paper, we propose a coarse-to-fine hierarchical approach for encoding prior knowledge that drastically …

BYOL-Explore learns to explore visually-rich environments by predicting world dynamics.

Intrinsically motivated goal exploration processes enable agents to autonomously sample goals to explore efficiently complex environments with high-dimensional continuous actions. They have been applied successfully to real world robots to discover repertoires of policies producing a wide diversity of effects. Often th…

HyperX uses reward bonuses to enable efficient exploration in meta-learning.

Active sampling improves design space exploration for analog circuits.

A new active learning method for Gaussian process models.

Proposes a method to avoid excessive exploration in reinforcement learning.

Generative Kernel PCA explores latent spaces for data interpretation and novelty detection.

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

LES reduces over-exploration in LSO, improving solution quality.

Achieving efficient and scalable exploration in complex domains poses a major challenge in reinforcement learning. While Bayesian and PAC-MDP approaches to the exploration problem offer strong formal guarantees, they are often impractical in higher dimensions due to their reliance on enumerating the state-action space.…

This paper investigates the effectiveness of adversarial training in enhancing the robustness of Deep Q-Network (DQN) policies to state-space perturbations. We first present a formal analysis of adversarial training in DQN agents and its performance with respect to the proportion of adversarial perturbations to nominal…

We describe our initial explorations in simulating non-euclidean geometries in virtual reality. Our simulation of the product of two-dimensional hyperbolic space with one-dimensional euclidean space is available at http://h2xe.hypernom.com.

Bayesian search optimizes exploration of feasible solutions under expensive constraints.

Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristi…

Study pure exploration in high-dimensional feature spaces using adaptive embeddings.

ManifoldMind uses adaptive-curvature probabilistic spheres for trustworthy recommendations in semantic hierarchies.

Balancing exploration and exploitation is a fundamental part of reinforcement learning, yet most state-of-the-art algorithms use a naive exploration protocol like $ε$-greedy. This contributes to the problem of high sample complexity, as the algorithm wastes effort by repeatedly visiting parts of the state space that ha…

Random exploration optimizes Bayesian optimization with optimal error rates and computational efficiency.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

Curiosity-driven exploration using Bayesian surprise in latent space.

Study generalizations of chainability and compactness in metric spaces.

We describe our initial explorations in simulating non-euclidean geometries in virtual reality. Our simulations of three-dimensional hyperbolic space are available at http://h3.hypernom.com.

Revel tackles safe exploration in RL with verified symbolic policies.

Hill-ADAM optimizes loss landscapes by exploring state space deterministically.

We present an algorithm, HOMER, for exploration and reinforcement learning in rich observation environments that are summarizable by an unknown latent state space. The algorithm interleaves representation learning to identify a new notion of kinematic state abstraction with strategic exploration to reach new states usi…

All reinforcement learning algorithms must handle the trade-off between exploration and exploitation. Many state-of-the-art deep reinforcement learning methods use noise in the action selection, such as Gaussian noise in policy gradient methods or $ε$-greedy in Q-learning. While these methods are appealing due to their…

We present a new model-based algorithm for reinforcement learning (RL) which consists of explicit exploration and exploitation phases, and is applicable in large or infinite state spaces. The algorithm maintains a set of dynamics models consistent with current experience and explores by finding policies which induce hi…

A new multi-objective RL framework improves intrinsic exploration performance.

The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as $ε$-greedy ex…

RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.

Entropy regularization is used to get improved optimization performance in reinforcement learning tasks. A common form of regularization is to maximize policy entropy to avoid premature convergence and lead to more stochastic policies for exploration through action space. However, this does not ensure exploration in th…

The paper explores stable surfaces in Einstein-Maxwell theory, proving mass bounds and nonexistence results.

A method for self-supervised representation learning in partially observable environments.

We propose an algorithm for exploring the entire regularization path of asymmetric-cost linear support vector machines. Empirical evidence suggests the predictive power of support vector machines depends on the regularization parameters of the training algorithms. The algorithms exploring the entire regularization path…

Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or …

In complex simulation environments, certain parameter space regions may result in non-convergent or unphysical outcomes. All parameters can therefore be labeled with a binary class describing whether or not they lead to valid results. In general, it can be very difficult to determine feasible parameter regions, especia…

The article explores causal structures in symmetric spaces and their relation to AQFT.

Context. Generative models open up the possibility to interrogate scientific data in a more data-driven way. Aims: We propose a method that uses generative models to explore hypotheses in astrophysics and other areas. We use a neural network to show how we can independently manipulate physical attributes by encoding ob…

Aims to optimize complex multivariate systems with constraints.

Maximizes Rényi entropy for efficient exploration in reward-free RL.

Improves neural network search in combinatorial spaces of mathematical symbols.