arXiv research

Random exploration optimizes Bayesian optimization with optimal error rates and computational efficiency.

The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as $ε$-greedy ex…

A single policy suffices for near-optimal parallel exploration in RL.

The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.

New $L_1$-Coverage objective simplifies exploration in reinforcement learning.

Optimal control in latent factor models uses Tsallis entropy for exploration.

We introduce the community exploration problem that has many real-world applications such as online advertising. In the problem, an explorer allocates limited budget to explore communities so as to maximize the number of members he could meet. We provide a systematic study of the community exploration problem, from off…

Optimizes pure exploration in linear bandits with a new algorithm.

An agent explores indefinitely in an environment with unlimited rewards.

The contextual bandit literature has traditionally focused on algorithms that address the exploration-exploitation tradeoff. In particular, greedy algorithms that exploit current estimates without any exploration may be sub-optimal in general. However, exploration-free greedy algorithms are desirable in practical setti…

Study uses RL to optimize investment with financial constraints, showing exploration benefits.

Paper optimizes industrial refrigeration using adaptive exploration.

Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…

LES reduces over-exploration in LSO, improving solution quality.

Study task-guided exploration in linear dynamical systems, improving sample complexity.

New findings on when to use action space exploration in reinforcement learning.

New algorithm explores optimally using same primitives as Thompson Sampling.

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

New meta-RL method avoids exploration-exploitation trade-off.

EQO uses a simple bonus term for efficient exploration in tabular RL.

In this article we explore an alternative approach to address deep exploration and we introduce the ISL algorithm, which is efficient at performing deep exploration. Similarly to maximum entropy RL, we derive the algorithm by augmenting the traditional RL objective with a novel regularization term. A distinctive featur…

Optimizes exploration for nonlinear systems to learn controllers efficiently.

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space explorat…

Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent from the literature. We aim to take a step towards solving this problem. We present two exploration m…

Paper presents an algorithm for optimal regret in communicating Markov decision processes.

Develops first optimal algorithm for logistic bandits.

We study the multi-armed bandit problem with subgaussian rewards. The explore-then-commit (ETC) strategy, which consists of an exploration phase followed by an exploitation phase, is one of the most widely used algorithms in a variety of online decision applications. Nevertheless, it has been shown in Garivier et al. (…

Optimally explores dynamical systems with varying properties using context inference.

Bayesian optimization (BO) and its batch extensions are successful for optimizing expensive black-box functions. However, these traditional BO approaches are not yet ideal for optimizing less expensive functions when the computational cost of BO can dominate the cost of evaluating the blackbox function. Examples of the…

Randomized exploration in linear bandits achieves optimal regret bounds.

Paper proposes Adaptive Pareto Exploration for identifying Pareto optimal arms in multi-objective scenarios.

A new multi-objective RL framework improves intrinsic exploration performance.

New methods for efficient exploration under unknown linear constraints in bandits.

This paper analyzes MORL and proposes efficient algorithms to learn Pareto optimal policies.

Recent years have witnessed a tremendous improvement of deep reinforcement learning. However, a challenging problem is that an agent may suffer from inefficient exploration, particularly for on-policy methods. Previous exploration methods either rely on complex structure to estimate the novelty of states, or incur sens…

This work introduces CAET, an algorithm for cost-aware pairwise pure exploration.

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

BOKE optimizes expensive functions with reduced computational costs.

USAC balances pessimism and optimism in actor-critic training for better exploration and performance.

To maximize its success, an AGI typically needs to explore its initially unknown world. Is there an optimal way of doing so? Here we derive an affirmative answer for a broad class of environments.

Sample-efficient exploration is crucial not only for discovering rewarding experiences but also for adapting to environment changes in a task-agnostic fashion. A principled treatment of the problem of optimal input synthesis for system identification is provided within the framework of sequential Bayesian experimental …

New AIM algorithm optimizes exploration-exploitation in bandits.

New algorithm learns optimal exploration parameters for contextual bandits.

New algorithm finds optimal policies without knowing reward functions.

ENIAC method optimizes and explores complex RL problems with non-linear policies.

We present the Variational Adaptive Newton (VAN) method which is a black-box optimization method especially suitable for explorative-learning tasks such as active learning and reinforcement learning. Similar to Bayesian methods, VAN estimates a distribution that can be used for exploration, but requires computations th…

Enhanced Gaussian process models accelerate optimization and posterior approximation.

The paper tackles fair sharing of exploration costs across groups in online learning.