Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.
problem Balancing exploration and exploitation in active learning for unknown functions.
method Develops BHEEM, a Bayesian hierarchical approach with approximate Bayesian computation for sampling trade-off parameters.
result BHEEM achieves at least 21% and 11% improvement over pure exploration and exploitation strategies respectively.
Proposes a heuristic to dynamically control exploration vs exploitation in Bayesian optimization.
problem The trade-off between exploration and exploitation in Bayesian optimization methods.
method Contextual Improvement heuristic to dynamically control the trade-off.
result Improves the speed and robustness of discovering optimal solutions.
We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…
Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.
problem Balancing exploration and exploitation in online decision making.
method Introducing a batch Thompson Sampling framework for stochastic multi-arm bandit and linear contextual bandit problems.
result Achieves asymptotic regret bound with O ( log T ) O(\log T) O ( log T ) batch queries, significantly reducing interactions. This research tackles balancing exploration and exploitation in deep RL for partially observable systems.
problem Balancing exploration and exploitation in deep RL for partially observable systems.
method Deployed and tested several techniques including adaptive and deterministic exploration strategies, and a modified quadratic loss function.
result Adaptive methods better approximate the trade-off between exploration and exploitation.
Algorithm achieves optimal pricing with minimal exploration for dynamic markets.
problem Optimal pricing in dynamic markets with contextual information.
method Localized exploration-then-commit (LetC) algorithm with pure exploration, refinement, and exploitation stages.
result Achieves minimax optimal, dimension-free regret bound.
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
Survey on methods for balancing exploration and exploitation in reinforcement learning.
problem Balancing exploration and exploitation in reinforcement learning, especially in domains with limited data.
method Survey of methods for computing robust solutions from fixed samples.
result Presentation of methods for balancing exploration-exploitation trade-off.
New meta-RL method avoids exploration-exploitation trade-off.
problem Learning to explore and exploit simultaneously in meta-RL.
method Developed new objectives for exploration and exploitation.
result DREAM outperforms existing methods on complex tasks.
Multi-armed bandit problems are the most basic examples of sequential decision problems with an exploration-exploitation trade-off. This is the balance between staying with the option that gave highest payoffs in the past and exploring new options that might give higher payoffs in the future. Although the study of band…
Paper proposes efficient sample collection strategy for RL.
problem Balancing exploration and exploitation in reinforcement learning.
method Decoupled approach with objective-specific and objective-agnostic strategies.
result Improved or novel sample complexity guarantees for various RL settings.
AdaLinUCB optimizes exploration-exploitation for contextually varying costs.
problem Optimizing decision-making in environments with varying exploration costs.
method Adaptive Upper-Confidence-Bound (AdaLinUCB) algorithm for opportunistic learning.
result AdaLinUCB achieves O((log T)^2) regret bound, significantly outperforming other algorithms.
Free lunch from noise reveals linear spectral features for RL.
problem Trade-off between expressiveness and tractability in RL.
method Noise assumption and Spectral Dynamics Embedding (SPEDE).
result SPEDE breaks the trade-off and completes optimistic exploration.
New algorithm learns optimal exploration parameters for contextual bandits.
problem Learning optimal exploration in contextual bandits.
method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.
result Demonstrates improved performance in learning optimal exploration compared to traditional methods.
AIS algorithm balances exploration and exploitation for efficient sampling.
problem Balancing exploration and exploitation in adaptive importance sampling.
method Daisee algorithm, partition-based approach, pseudo-regret analysis.
result Daisee achieves O ( T ( log T ) 3 4 ) \mathcal{O}(\sqrt{T}(\log T)^{\frac{3}{4}}) O ( T ( log T ) 4 3 ) cumulative pseudo-regret. Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…
BOiLS optimizes circuit quality using Bayesian optimization.
problem Optimizing circuits with complex search spaces.
method Adapting Bayesian optimization to logic synthesis, using Gaussian process kernels and trust-region constrained acquisitions.
result Demonstrated superior performance in sample efficiency and QoR values.
Improved Thompson Sampling for Bayesian Optimization.
problem Handling the exploitation-exploration dilemma in Bayesian optimization.
method Incorporating epsilon-greedy policy into Thompson Sampling.
result Epsilon-greedy Thompson Sampling outperforms standard TS extremes.
MULEX separates exploration and exploitation in reinforcement learning.
problem Balancing discovery of new rewards with past behavior in reinforcement learning.
method Disentangles exploration and exploitation by optimizing multiple losses in parallel.
result MULEX achieves sample-efficiency and robustness in a hard-exploration environment.
Proposes a new acquisition function for batched Bayesian optimization.
problem Intractability of acquisition functions for batched Bayesian optimization.
method Statistical physics inspired acquisition function for Gaussian processes.
result Demonstrates competitive performance on various problems.
NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.
problem Stochastic contextual bandit problem with general bounded reward functions.
method Reward-biased maximum likelihood estimation with neural networks to enforce exploration.
result Both NeuralRBMLE variants achieve O ~ ( T ) \widetilde{\mathcal{O}}(\sqrt{T}) O ( T ) regret. New framework for RL with linear-convex models reduces performance gap.
problem Continuous-time episodic reinforcement learning with unknown coefficients and convex objectives.
method Probabilistic framework and phase-based learning algorithm for optimal exploration-exploitation trade-off.
result Sublinear regrets achieved, matching best possible results in literature.
MELEE learns good exploration strategies for contextual bandits.
problem Interactive contextual bandit exploration trade-off.
method Meta-learning from synthetic data to learn a good exploration policy.
result MELEE outperforms seven strong baseline algorithms on real-world datasets.
Survey on risk-aware multi-armed bandits for better decision-making.
problem Risk measures in multi-armed bandits for better decision-making.
method Review of existing research, definition of risk-aware bandit problems, and algorithms for minimizing regret and identifying best arms.
result Consolidation and summarization of existing research on risk measures in multi-armed bandits.
ICEE learns new RL tasks in less time with a Transformer model.
problem Efficient in-context policy learning for reinforcement learning.
method In-context Exploration-Exploitation (ICEE) algorithm that optimizes efficiency without explicit Bayesian inference.
result ICEE solves Bayesian optimization problems as efficiently as Gaussian process biased methods but in significantly less time.
BelMan uses Bayesian methods to optimize decisions in multi-armed bandit problems.
problem Optimizing decisions in multi-armed bandit problems with varying rewards and beliefs.
method BelMan uses a geometric approach with information projection and reverse projection to balance exploration and exploitation.
result BelMan outperforms other algorithms in specific scenarios involving many arms and continuous rewards.
The paper explores the trade-off between recommendation system performance and bandwidth usage.
problem Balancing recommendation system performance with wireless bandwidth constraints.
method Analyzes two scenarios: multi-armed bandit with context and latent structure exploitation.
result Demonstrates a tradeoff between regret and bandwidth usage, with tight bounds for some instances.
Optimal regret achieved in stochastic, discrete multi-armed bandits using information-theoretic exploration.
problem Optimal exploration vs. exploitation in stochastic, discrete multi-armed bandits.
method Proposes an information-theoretic strategy based on the value of information criterion, using simulated-annealing-like updates of a parameter.
result Achieves logarithmic optimal regret with respect to the number of episodes.
New algorithms improve linear bandit performance with low computation.
problem Optimizing reward in linear stochastic bandits.
method Reward-biased maximum likelihood method modified for linear and generalized linear bandits.
result New policies achieve order-optimality and competitive empirical performance.
This paper analyzes the multi-armed bandit problem using frequency-domain methods.
problem The exploration-exploitation trade-off in sequential decision-making.
method Proposes a frequency-domain analysis framework, reformulating the bandit process as a signal processing problem.
result Confidence bound term in UCB algorithm is equivalent to a time-varying gain in frequency domain.
The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.
problem Balancing exploration and exploitation in CMDPs to satisfy constraints.
method Two approaches: optimistic planning and incremental updates of primal and dual variables.
result Both approaches achieve sublinear regret on utility and constraint violations, with stronger guarantees for the linear programming approach.
Meta-SAC automatically tunes SAC's entropy temperature for better exploration.
problem Exploration-exploitation dilemma in reinforcement learning.
method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.
result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.
New algorithms for risk-averse bandits minimize regret in finite time.
problem Minimizing regret in finite time for bandit problems.
method Proposes two algorithms for selecting the most probable arm with a good risk-return trade-off.
result Upper bound for the minimum number of experiments before commitment to guarantee a bound on regret.
A simple uncertainty measure improves deep bandit performance.
problem Efficient exploration in complex environments with deep neural networks.
method Sample Average Uncertainty (SAU) for estimating outcome uncertainty directly.
result SAU matches the uncertainty of Thompson Sampling and its regret bounds.
Improved control approach for correlated bandits with better performance.
problem General multi-armed bandit problem with correlated elements.
method Introducing entropy regularisation to obtain a smooth asymptotic approximation of the value function, leading to a semi-index approximation of the optimal decision process.
result Performance of Asymptotic Randomised Control (ARC) algorithm compares favorably with other approaches.
New policy tackles evolving externalities in contextual bandits.
problem Difficulty in recovering from wrong decisions over time.
method Rejection-based policy to achieve low regret.
result Low regret achieved regardless of reward matrix structure.
Bayesian approach improves ε \varepsilon ε -greedy exploration in RL.
problem Improving ε \varepsilon ε -greedy exploration in model-free RL. method Introducing a Bayesian model update for ε \varepsilon ε based on BMC. result Proposed ε \varepsilon ε - exttt{BMC} algorithm efficiently balances exploration and exploitation. Deep learning tackles contextual multi-armed bandits with principled exploration.
problem Contextual multi-armed bandits in industrial applications.
method Bayesian neural network with dropout for non-linear modeling and Thompson sampling for principled exploration.
result Substantially reduces regret compared to existing methods.
PBCS combines RL and motion planning for better exploration.
problem RL algorithms struggle with versatile exploration in complex environments.
method PBCS uses motion planning to find a good trajectory, then trains RL on a curriculum derived from it.
result PBCS outperforms state-of-the-art RL algorithms in 2D maze environments.
Bayesian model-based reinforcement learning is a formally elegant approach to learning optimal behaviour under model uncertainty, trading off exploration and exploitation in an ideal way. Unfortunately, finding the resulting Bayes-optimal policies is notoriously taxing, since the search space becomes enormous. In this …
New greedy algorithms improve Bayesian optimisation performance.
problem Optimizing continuous functions with exploration vs exploitation trade-offs.
method Introduced two novel ε-greedy acquisition functions and compared them with conventional methods.
result ε-greedy algorithms generally outperform conventional methods, especially in higher dimensions.
Study examines impact of missing data on multi-armed bandit algorithms.
problem Impact of missing data on performance of multi-armed bandit algorithms.
method Extensive simulation study of two-armed bandit algorithms with binary outcomes, considering different probabilities of missingness.
result Impact on performance varies depending on the balance between exploration and exploitation.
A framework for auto-tuning hyper-parameters in contextual bandit algorithms.
problem Auto-tuning hyper-parameters in real-time for contextual bandit algorithms.
method Proposes a Syndicated Bandits framework to learn multiple hyper-parameters dynamically.
result Achieves optimal regret bounds under certain scenarios and handles multiple contextual bandit algorithms.
Knowledge graph construction consists of two tasks: extracting information from external resources (knowledge population) and inferring missing information through a statistical analysis on the extracted information (knowledge completion). In many cases, insufficient external resources in the knowledge population hinde…
Agent learns directed exploration policies to improve performance in hard games.
problem Improving exploration in complex games.
method Episodic memory-based intrinsic reward, self-supervised inverse dynamics, UVFA framework.
result Doubles performance in hard exploration games, achieves non-zero rewards in Pitfall!.
We present a generic framework for trading off fidelity and cost in computing stochastic gradients when the costs of acquiring stochastic gradients of different quality are not known a priori. We consider a mini-batch oracle that distributes a limited query budget over a number of stochastic gradients and aggregates th…
EDU method finds diverse optimal solutions for expensive simulators.
problem Optimizing expensive black-box simulators for diverse solutions.
method EDU method searches for diverse locally-optimal solutions within a tolerance level.
result EDU yields a closed-form acquisition function facilitating efficient sequential queries.
SPEDER extracts state-action abstraction from dynamics for reinforcement learning.
problem Curse of dimensionality and limited applicability of spectral methods.
method Spectral Decomposition Representation (SPEDER) that extracts state-action abstraction from dynamics without policy dependence.
result Theoretical analysis establishes sample efficiency in online and offline settings.