On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “exploration exploitation”

Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescs.LG

New meta-RL method avoids exploration-exploitation trade-off.

problem Learning to explore and exploit simultaneously in meta-RL.

method Developed new objectives for exploration and exploitation.

result DREAM outperforms existing methods on complex tasks.

meta-reinforcement learning exploration-exploitation end-to-end training local optima task-relevant information

2020-08-06abs ↗pdf ↗

Dynamic Exploration-Exploitation Trade-Off in Active Learning Regression with Bayesian Hierarchical Modelingcs.LG

Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.

problem Balancing exploration and exploitation in active learning for unknown functions.

method Develops BHEEM, a Bayesian hierarchical approach with approximate Bayesian computation for sampling trade-off parameters.

result BHEEM achieves at least 21% and 11% improvement over pure exploration and exploitation strategies respectively.

active learning Bayesian hierarchical modeling exploration-exploitation approximate Bayesian computation black-box function

2023-04-16abs ↗pdf ↗

Approximate information for efficient exploration-exploitation strategiesstat.ML

New AIM algorithm optimizes exploration-exploitation in bandits.

problem Balancing exploration and exploitation in decision-making.

method Approximate Information Maximization (AIM) algorithm.

result AIM outperforms Infomax and Thompson sampling with enhanced speed and tractability.

bandit problems exploration-exploitation AIM algorithm entropy gradient asymptotic bound

2023-07-04abs ↗pdf ↗

Localized exploration in contextual dynamic pricing achieves dimension-free regretstat.ML

Algorithm achieves optimal pricing with minimal exploration for dynamic markets.

problem Optimal pricing in dynamic markets with contextual information.

method Localized exploration-then-commit (LetC) algorithm with pure exploration, refinement, and exploitation stages.

result Achieves minimax optimal, dimension-free regret bound.

dynamic pricing contextual learning regret minimization exploration-exploitation

2024-12-26abs ↗pdf ↗

Dealing with uncertainty: balancing exploration and exploitation in deep recurrent reinforcement learningstat.ML

This research tackles balancing exploration and exploitation in deep RL for partially observable systems.

problem Balancing exploration and exploitation in deep RL for partially observable systems.

method Deployed and tested several techniques including adaptive and deterministic exploration strategies, and a modified quadratic loss function.

result Adaptive methods better approximate the trade-off between exploration and exploitation.

Reinforcement Learning Deep Learning Exploration-Exploitation Autonomous Driving Deep Recurrent Q-Network

2023-10-12abs ↗pdf ↗

Epsilon-Greedy Thompson Sampling to Bayesian Optimizationcs.LG

Improved Thompson Sampling for Bayesian Optimization.

problem Handling the exploitation-exploration dilemma in Bayesian optimization.

method Incorporating epsilon-greedy policy into Thompson Sampling.

result Epsilon-greedy Thompson Sampling outperforms standard TS extremes.

Thompson Sampling Bayesian Optimization Epsilon-Greedy Reinforcement Learning Exploitation-Exploration

2024-03-01abs ↗pdf ↗

Generative Exploration and Exploitationcs.LG

Sparse reward is one of the biggest challenges in reinforcement learning (RL). In this paper, we propose a novel method called Generative Exploration and Exploitation (GENE) to overcome sparse reward. GENE automatically generates start states to encourage the agent to explore the environment and to exploit received rew…

2019-04-21abs ↗pdf ↗

Exploration Unboundcs.LG

An agent explores indefinitely in an environment with unlimited rewards.

problem Balancing exploration and exploitation in environments with unlimited rewards.

method Simple example of an environment with unbounded rewards and optimal agent behavior.

result An optimal agent always explores to maximize rewards, regardless of accumulated knowledge.

sequential decision-making exploration-exploitation unbounded rewards optimal agent behavior

2024-07-16abs ↗pdf ↗

Improved Analysis of UCRL2 with Empirical Bernstein Inequalitycs.LG

Improved analysis of UCRL2 with empirical Bernstein inequality reduces exploration-exploitation regret.

problem Exploration-exploitation in communicating Markov Decision Processes.

method Analysis of UCRL2 with Empirical Bernstein inequalities (UCRL2B).

result Regret bound of

\widetilde{O}(\sqrt{DΓS A T})

for UCRL2B.

UCRL2 Empirical Bernstein Inequality Exploration-Exploitation Markov Decision Processes Regret Analysis

2020-07-10abs ↗pdf ↗

Graph-Based Active Learning: A New Look at Expected Error Minimizationstat.ML

In graph-based active learning, algorithms based on expected error minimization (EEM) have been popular and yield good empirical performance. The exact computation of EEM optimally balances exploration and exploitation. In practice, however, EEM-based algorithms employ various approximations due to the computational ha…

2016-09-03abs ↗pdf ↗

EE-Net: Exploitation-Exploration Neural Networks in Contextual Banditscs.LG

Proposes EE-Net for neural exploration in contextual bandits.

problem Exploitation-Exploration tradeoff in contextual bandits.

method Uses two neural networks: Exploitation and Exploration, to learn reward function and adaptively explore.

result Achieves

\mathcal{O}(\sqrt{T\log T})

regret and outperforms existing methods.

contextual bandits neural networks exploration exploitation regret

2021-10-07abs ↗pdf ↗

Thompson Sampling on Asymmetric

α

-Stable Banditsstat.ML

This paper applies Thompson Sampling to asymmetric $\alpha$ -stable bandits for financial and wireless data.

problem Optimizing exploration-exploitation in multi-armed bandits with asymmetric

\alpha

-stable distributions.

method Thompson Sampling applied to unknown asymmetric

\alpha

-stable reward distributions.

result Demonstrates effectiveness of Thompson Sampling for asymmetric

\alpha

-stable bandits.

Thompson Sampling Bandits Asymmetric $\alpha$-stable Exploration-Exploitation Financial Data

2022-03-19abs ↗pdf ↗

Exploration-Exploitation Motivated Variational Auto-Encoder for Recommender Systemsstat.ML

XploVAE improves recommendation by balancing known and novel items.

problem Balancing known and novel items for better recommendations.

method Constructs user-specific subgraphs for exploitation and exploration, learns personalized item embeddings.

result Demonstrates effectiveness on various real-world datasets.

recommender systems variational auto-encoder exploitation exploration personalized recommendations

2020-06-05abs ↗pdf ↗

Dynamic Control of Explore/Exploit Trade-Off In Bayesian Optimizationstat.ML

Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…

2018-07-03abs ↗pdf ↗

Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learningcs.LG

Hyper addresses the hyperparameter tuning challenge in RL, improving exploration efficiency and robustness.

problem Hyperparameter tuning is a significant challenge in RL, especially for curiosity-based exploration methods.

method Hyper robustly explores by effectively regularizing exploration visits and decoupling exploitation.

result Hyper is provably efficient and robust in various RL environments.

reinforcement learning hyperparameter tuning curiosity exploration exploitation

2024-12-04abs ↗pdf ↗

Reinforcement Learning through Active Inferencecs.LG

Active inference enhances RL by balancing exploration and exploitation.

problem Traditional RL's balance between exploration and exploitation is often suboptimal.

method Developed a new decision-making objective based on active inference.

result The new algorithm successfully balances exploration and exploitation on various RL benchmarks.

reinforcement learning active inference exploration-exploitation free energy decision making

2020-02-28abs ↗pdf ↗

Parallelizing Thompson Samplingcs.LG

Batch Thompson Sampling reduces exploration-exploitation trade-off in online decision making.

problem Balancing exploration and exploitation in online decision making.

method Introducing a batch Thompson Sampling framework for stochastic multi-arm bandit and linear contextual bandit problems.

result Achieves asymptotic regret bound with

O(\log T)

batch queries, significantly reducing interactions.

Thompson Sampling Online Decision Making Bandit Problems Exploration-Exploitation Exponential Reduction

2021-06-02abs ↗pdf ↗

Explicit Explore-Exploit Algorithms in Continuous State Spacescs.LG

We present a new model-based algorithm for reinforcement learning (RL) which consists of explicit exploration and exploitation phases, and is applicable in large or infinite state spaces. The algorithm maintains a set of dynamics models consistent with current experience and explores by finding policies which induce hi…

2019-11-01abs ↗pdf ↗

Explore or exploit? A generic model and an exactly solvable casecond-mat.dis-nn

Finding a good compromise between the exploitation of known resources and the exploration of unknown, but potentially more profitable choices, is a general problem, which arises in many different scientific disciplines. We propose a stylized model for these exploration-exploitation situations, including population or e…

2013-10-18abs ↗pdf ↗

Kernel-Based Reinforcement Learning: A Finite-Time Analysiscs.LG

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

problem Exploration-exploitation dilemma in finite-horizon reinforcement learning with metric state-action spaces.

method Kernel-UCBVI, leveraging smoothness and kernel estimators of rewards and transitions.

result First regret bound for kernel-based RL using smoothing kernels,

O(H^3 K^{2d/(2d+1)})

Kernel-based reinforcement learning Metric state-action spaces Exploration-exploitation Regret bound Finite-horizon problems

2020-04-12abs ↗pdf ↗

Learning to Plan in High Dimensions via Neural Exploration-Exploitation Treescs.LG

We propose a meta path planning algorithm named \emph{Neural Exploration-Exploitation Trees~(NEXT)} for learning from prior experience for solving new path planning problems in high dimensional continuous state and action spaces. Compared to more classical sampling-based methods like RRT, our approach achieves much bet…

2019-02-28abs ↗pdf ↗

Exploration-Exploitation in Constrained MDPscs.LG

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

problem Balancing exploration and exploitation in CMDPs to satisfy constraints.

method Two approaches: optimistic planning and incremental updates of primal and dual variables.

result Both approaches achieve sublinear regret on utility and constraint violations, with stronger guarantees for the linear programming approach.

CMDPs exploration-exploitation constraint satisfaction optimistic planning dual formulation

2020-03-04abs ↗pdf ↗

Better safe than sorry: Risky function exploitation through safe optimizationstat.AP

Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…

2016-02-02abs ↗pdf ↗

Explore-Exploit: A Framework for Interactive and Online Learningcs.LG

Interactive user interfaces need to continuously evolve based on the interactions that a user has (or does not have) with the system. This may require constant exploration of various options that the system may have for the user and obtaining signals of user preferences on those. However, such an exploration, especiall…

2018-12-01abs ↗pdf ↗

To update or not to update? Delayed Nonparametric Bandits with Randomized Allocationstat.ML

Study explores strategies for randomized allocation in delayed rewards bandits.

problem Understanding the exploration-exploitation tradeoff in randomized strategies with delayed rewards.

method Examines two strategies: updating exploration sequence at every time point vs. updating only when a new reward is observed.

result The strategy updating only when a new reward is observed leads to strong consistency in allocation for a wider scope of situations.

bandits contextual exploration-exploitation randomized allocation delayed rewards

2020-05-26abs ↗pdf ↗

Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedulecs.LG

Paper finds wide minima are better for generalization and proposes a new learning rate schedule.

problem The challenge of finding optimal learning rates for model training.

method The paper introduces a new hypothesis about the density of wide minima and designs an explore-exploit learning rate schedule.

result The explore-exploit learning rate schedule improves model performance and reduces training time.

wide minima generalization learning rate explore-exploit accuracy

2020-03-09abs ↗pdf ↗

PAC-Bayesian Analysis of the Exploration-Exploitation Trade-offcs.LG

We develop a coherent framework for integrative simultaneous analysis of the exploration-exploitation and model order selection trade-offs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combinatio…

2011-05-23abs ↗pdf ↗

Hyper-parameter Tuning for the Contextual Banditcs.LG

New algorithm learns optimal exploration parameters for contextual bandits.

problem Learning optimal exploration in contextual bandits.

method Proposes two algorithms that learn optimal exploration parameters online based on context and reward.

result Demonstrates improved performance in learning optimal exploration compared to traditional methods.

contextual bandits exploration exploitation online learning multi-armed bandits

2020-05-04abs ↗pdf ↗

In-context Exploration-Exploitation for Reinforcement Learningcs.LG

ICEE learns new RL tasks in less time with a Transformer model.

problem Efficient in-context policy learning for reinforcement learning.

method In-context Exploration-Exploitation (ICEE) algorithm that optimizes efficiency without explicit Bayesian inference.

result ICEE solves Bayesian optimization problems as efficiently as Gaussian process biased methods but in significantly less time.

reinforcement learning in-context learning exploration-exploitation Transformer model Bayesian inference

2024-03-11abs ↗pdf ↗

A Provably Efficient Sample Collection Strategy for Reinforcement Learningcs.LG

Paper proposes efficient sample collection strategy for RL.

problem Balancing exploration and exploitation in reinforcement learning.

method Decoupled approach with objective-specific and objective-agnostic strategies.

result Improved or novel sample complexity guarantees for various RL settings.

reinforcement learning exploration-exploitation sample collection stochastic shortest path communicating MDPs

2020-07-13abs ↗pdf ↗

Greedy Algorithm almost Dominates in Smoothed Contextual Banditscs.LG

Greedy algorithm nearly outperforms exploration in contextual bandits.

problem Balancing exploration and exploitation in online learning.

method Smoothed analysis of the greedy algorithm in linear contextual bandits.

result Greedy algorithm nearly matches Bayesian regret rate under diversity conditions, with regret at most

O(T^{1/3})

contextual bandits greedy algorithm smoothed analysis Bayesian regret exploration vs exploitation

2020-05-19abs ↗pdf ↗

Exploration Conscious Reinforcement Learning Revisitedcs.LG

The Exploration-Exploitation tradeoff arises in Reinforcement Learning when one cannot tell if a policy is optimal. Then, there is a constant need to explore new actions instead of exploiting past experience. In practice, it is common to resolve the tradeoff by using a fixed exploration mechanism, such as $ε$ -greedy ex…

2018-12-13abs ↗pdf ↗

A Lipschitz Exploration-Exploitation Scheme for Bayesian Optimizationcs.LG

The problem of optimizing unknown costly-to-evaluate functions has been studied for a long time in the context of Bayesian Optimization. Algorithms in this field aim to find the optimizer of the function by asking only a few function evaluations at locations carefully selected based on a posterior model. In this paper,…

2012-03-30abs ↗pdf ↗

Risk-Averse Explore-Then-Commit Algorithms for Finite-Time Banditscs.LG

In this paper, we study multi-armed bandit problems in explore-then-commit setting. In our proposed explore-then-commit setting, the goal is to identify the best arm after a pure experimentation (exploration) phase and exploit it once or for a given finite number of times. We identify that although the arm with the hig…

2019-04-30abs ↗pdf ↗

Heterogeneous Explore-Exploit Strategies on Multi-Star Networksmath.OC

New strategies improve multi-agent decision-making on irregular networks.

problem Maximizing group reward in multi-agent settings with heterogeneous strategies.

method Design and analysis of heterogeneous explore-exploit strategies for multi-star networks.

result Group performance improves under heterogeneous strategies compared to homogeneous strategies.

multi-agent systems heterogeneous strategies explore-exploit multi-star networks stochastic bandits

2020-09-02abs ↗pdf ↗

Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Explorationcs.LG

Maximize to Explore integrates RL components for efficient policy discovery.

problem Balancing exploration and exploitation in online RL with general function approximators.

method Integrates estimation, planning, and exploration into a single objective function.

result Achieves sublinear regret for MDPs and MGs with general function approximations.

online reinforcement learning reinforcement learning sample efficiency exploration exploitation

2023-05-29abs ↗pdf ↗

Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processescs.LG

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL $^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bo…

2018-12-11abs ↗pdf ↗

Bayesian bandits: balancing the exploration-exploitation tradeoff via double samplingstat.ML

Reinforcement learning studies how to balance exploration and exploitation in real-world systems, optimizing interactions with the world while simultaneously learning how the world operates. One general class of algorithms for such learning is the multi-armed bandit setting. Randomized probability matching, based upon …

2017-09-10abs ↗pdf ↗

Intrinsic Exploration as Multi-Objective RLcs.LG

A new multi-objective RL framework improves intrinsic exploration performance.

problem Sub-optimal exploration performance due to ad-hoc handling of intrinsic exploration.

method A multi-objective RL framework where both exploration and exploitation are optimized as separate objectives.

result EMU-Q method outperforms classic and other intrinsic RL methods on benchmarks.

reinforcement learning exploration multi-objective control uncertainty

2020-04-06abs ↗pdf ↗

AdaLinUCB: Opportunistic Learning for Contextual Banditscs.LG

In this paper, we propose and study opportunistic contextual bandits - a special case of contextual bandits where the exploration cost varies under different environmental conditions, such as network load or return variation in recommendations. When the exploration cost is low, so is the actual regret of pulling a sub-…

2019-02-20abs ↗pdf ↗

Ensemble Active Learning by Contextual Bandits for AI Incubation in Manufacturingcs.LG

Active learning method improves AI performance by balancing exploration and exploitation.

problem Efficiently acquiring samples for supervised learning in streaming data.

method Ensemble active learning by contextual bandits.

result Improved AI modeling performance through better sample acquisition.

active learning contextual bandits supervised learning AI modeling streaming data

2023-10-10abs ↗pdf ↗

Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradientcs.LG

Meta-SAC automatically tunes SAC's entropy temperature for better exploration.

problem Exploration-exploitation dilemma in reinforcement learning.

method Meta-SAC uses metagradient and a novel meta objective to automatically adjust SAC's entropy temperature.

result Meta-SAC outperforms SAC-v2 by 10% on the humanoid-v2 task.

reinforcement learning soft actor-critic entropy temperature exploration-exploitation metagradient

2020-07-03abs ↗pdf ↗

Adaptive Exploration-Exploitation Tradeoff for Opportunistic Banditscs.LG

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptim…

2017-09-12abs ↗pdf ↗

Poisson Reweighted Laplacian Uncertainty Sampling for Graph-based Active Learningstat.ML

New algorithm improves graph-based active learning by identifying unexplored regions.

problem Improving graph-based active learning by identifying unexplored regions.

method Poisson Reweighted Laplacian Uncertainty Sampling (PWLL) with a diagonal perturbation.

result PWLL effectively identifies unexplored regions in graph-based data.

active learning graph-based uncertainty sampling diagonal perturbation exploration vs exploitation

2022-10-27abs ↗pdf ↗

Tuning Confidence Bound for Stochastic Bandits with Bandit Distancestat.ML

Improved UCB method for stochastic bandits using distance tuning.

problem Improving performance in stochastic bandit problems.

method Tuning confidence bounds based on bandit distance.

result Empirically shows increased performance compared to existing methods.

stochastic bandits upper confidence bound distance tuning exploration exploitation tradeoff regret minimization

2021-10-06abs ↗pdf ↗

Infomax strategies for an optimal balance between exploration and exploitationcs.LG

Proper balance between exploitation and exploration is what makes good decisions, which achieve high rewards like payoff or evolutionary fitness. The Infomax principle postulates that maximization of information directs the function of diverse systems, from living systems to artificial neural networks. While specific a…

2016-01-12abs ↗pdf ↗

Never Give Up: Learning Directed Exploration Strategiescs.LG

We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memory-based intrinsic reward using k-nearest neighbors over the agent's recent experience to train the directed exploratory policies, thereby encouraging the agent to…

2020-02-14abs ↗pdf ↗

Soft policy optimization using dual-track advantage estimatorcs.LG

Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.

problem Balancing exploration and exploitation in reinforcement learning.

method Soft policy optimization with entropy and dual-track advantage estimator (DTAE).

result DTAE accelerates RL algorithm convergence and improves performance.

reinforcement learning policy optimization entropy temporal-difference general advantage estimator

2020-09-15abs ↗pdf ↗