arXiv research

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

New AIM algorithm optimizes exploration-exploitation in bandits.

Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

Paper finds wide minima are better for generalization and proposes a new learning rate schedule.

We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…

ICEE learns new RL tasks in less time with a Transformer model.

New strategies improve multi-agent decision-making on irregular networks.

Interactive user interfaces need to continuously evolve based on the interactions that a user has (or does not have) with the system. This may require constant exploration of various options that the system may have for the user and obtaining signals of user preferences on those. However, such an exploration, especiall…

Study explores strategies for randomized allocation in delayed rewards bandits.

Lecture notes on reinforcement learning using statistical methods.

Improved Bayesian optimisation method using randomised Gaussian process UCB.

New algorithms improve linear bandit performance with low computation.

This paper applies Thompson Sampling to asymmetric $\alpha$-stable bandits for financial and wireless data.

New algorithm learns optimal exploration parameters for contextual bandits.

Improved control approach for correlated bandits with better performance.

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

New framework for RL with linear-convex models reduces performance gap.

We propose a meta path planning algorithm named \emph{Neural Exploration-Exploitation Trees~(NEXT)} for learning from prior experience for solving new path planning problems in high dimensional continuous state and action spaces. Compared to more classical sampling-based methods like RRT, our approach achieves much bet…

Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…

Multi-armed bandit problems are the most basic examples of sequential decision problems with an exploration-exploitation trade-off. This is the balance between staying with the option that gave highest payoffs in the past and exploring new options that might give higher payoffs in the future. Although the study of band…

Proposes a new acquisition function for batched Bayesian optimization.

ASD algorithm maximizes model estimates by adaptively labeling points.

Paper proposes efficient sample collection strategy for RL.

Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…

Research adapts Multi-Armed Bandits for Minesweeper, improving player strategy.

Can one parallelize complex exploration exploitation tradeoffs? As an example, consider the problem of optimal high-throughput experimental design, where we wish to sequentially design batches of experiments in order to simultaneously learn a surrogate function mapping stimulus to response and identify the maximum of t…

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

Efficiently solves exploration-exploitation in LQR using Lagrangian relaxation.

Meta-SAC automatically tunes SAC's entropy temperature for better exploration.

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptim…

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

New algorithm reduces RL complexity with low switching costs.

We introduce SCAL, an algorithm designed to perform efficient exploration-exploitation in any unknown weakly-communicating Markov decision process (MDP) for which an upper bound $c$ on the span of the optimal bias function is known. For an MDP with $S$ states, $A$ actions and $Γ\leq S$ possible next states, we prove a …

Survey on risk-aware multi-armed bandits for better decision-making.

We introduce a novel algorithmic approach to content recommendation based on adaptive clustering of exploration-exploitation ("bandit") strategies. We provide a sharp regret analysis of this algorithm in a standard stochastic noise setting, demonstrate its scalability properties, and prove its effectiveness on a number…

New method learns adaptive exploration strategies for dynamic tasks.

Monte Carlo Tree Search (MCTS) algorithms have achieved great success on many challenging benchmarks (e.g., Computer Go). However, they generally require a large number of rollouts, making their applications costly. Furthermore, it is also extremely challenging to parallelize MCTS due to its inherent sequential nature:…

New algorithms improve contextual bandits with neural networks and energy models.

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL$^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…

Efficient algorithm for CMDPs reduces to offline density estimation.

Contextual bandits have the same exploration-exploitation trade-off as standard multi-armed bandits. On adding positive externalities that decay with time, this problem becomes much more difficult as wrong decisions at the start are hard to recover from. We explore existing policies in this setting and highlight their …

Active learning method improves AI performance by balancing exploration and exploitation.

New meta-RL method avoids exploration-exploitation trade-off.

We consider a finite-horizon multi-armed bandit (MAB) problem in a Bayesian setting, for which we propose an information relaxation sampling framework. With this framework, we define an intuitive family of control policies that include Thompson sampling (TS) and the Bayesian optimal policy as endpoints. Analogous to TS…