arXiv research

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…

Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.

Improved Bayesian optimisation method using randomised Gaussian process UCB.

Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…

Multi-armed bandit problems are the most basic examples of sequential decision problems with an exploration-exploitation trade-off. This is the balance between staying with the option that gave highest payoffs in the past and exploring new options that might give higher payoffs in the future. Although the study of band…

Proposes a new acquisition function for batched Bayesian optimization.

ICEE learns new RL tasks in less time with a Transformer model.

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

New algorithms improve linear bandit performance with low computation.

New framework for RL with linear-convex models reduces performance gap.

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

New algorithm learns optimal exploration parameters for contextual bandits.

Survey on risk-aware multi-armed bandits for better decision-making.

Improved control approach for correlated bandits with better performance.

Paper proposes efficient sample collection strategy for RL.

Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…

This research tackles balancing exploration and exploitation in deep RL for partially observable systems.

Robo-advisor learns investor's risk preference through portfolio choices.

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

New policy tackles evolving externalities in contextual bandits.

A reinforcement learning agent tries to maximize its cumulative payoff by interacting in an unknown environment. It is important for the agent to explore suboptimal actions as well as to pick actions with highest known rewards. Yet, in sensitive domains, collecting more data with exploration is not always possible, but…

EDU method finds diverse optimal solutions for expensive simulators.

The exploration-exploitation trade-off is among the central challenges of reinforcement learning. The optimal Bayesian solution is intractable in general. This paper studies to what extent analytic statements about optimal learning are possible if all beliefs are Gaussian processes. A first order approximation of learn…

Meta-SAC automatically tunes SAC's entropy temperature for better exploration.

We consider an agent who is involved in a Markov decision process and receives a vector of outcomes every round. Her objective is to maximize a global concave reward function on the average vectorial outcome. The problem models applications such as multi-objective optimization, maximum entropy exploration, and constrai…

This paper analyzes the multi-armed bandit problem using frequency-domain methods.

We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the bandits literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AI…

We describe MELEE, a meta-learning algorithm for learning a good exploration policy in the interactive contextual bandit setting. Here, an algorithm must take actions based on contexts, and learn based only on a reward signal from the action taken, thereby generating an exploration/exploitation trade-off. MELEE address…

We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the ma…

New algorithms improve contextual bandits with neural networks and energy models.

BOiLS optimizes circuit quality using Bayesian optimization.

A simple uncertainty measure improves deep bandit performance.

New meta-RL method avoids exploration-exploitation trade-off.

Study examines impact of missing data on multi-armed bandit algorithms.

We consider a finite-horizon multi-armed bandit (MAB) problem in a Bayesian setting, for which we propose an information relaxation sampling framework. With this framework, we define an intuitive family of control policies that include Thompson sampling (TS) and the Bayesian optimal policy as endpoints. Analogous to TS…

The paper explores sampling problems and shows minimal exploration is needed.

New algorithms for batch decision-making with high-dimensional user data.

A heuristic method refines search space for Bayesian optimization with low budget.

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

Many efficient algorithms with strong theoretical guarantees have been proposed for the contextual multi-armed bandit problem. However, applying these algorithms in practice can be difficult because they require domain expertise to build appropriate features and to tune their parameters. We propose a new method for the…

Stable matching, a classical model for two-sided markets, has long been studied with little consideration for how each side's preferences are learned. With the advent of massive online markets powered by data-driven matching platforms, it has become necessary to better understand the interplay between learning and mark…

New AIM algorithm optimizes exploration-exploitation in bandits.

Quantum RL algorithm achieves logarithmic regret for exploration.

Improved algorithm for logistic bandits with better regret bounds.

We stabilize the Kumaraswamy distribution for efficient sampling and differentiation.