New policy combines Thompson sampling with best challenger rule for best arm identification.
problem Best arm identification in bandit framework with fixed confidence.
method Combines Thompson sampling with best challenger rule.
result Asymptotically optimal for any two-armed bandit problems, near optimal for general K-armed bandit problems.
Improved knowledge gradient (iKG) outperforms the original KG algorithm in best arm identification problems.
problem Best arm identification (BAI) problem with limitations of the original KG algorithm.
method Follows the one-step look ahead of KG but chooses the measurement that maximizes the probability of selecting the best arm.
result Improved knowledge gradient (iKG) is asymptotically optimal and easier to extend to variant BAI problems.
Efficiently identifies best policies in tabular MDPs with reduced computational cost.
problem Identifying the best policy in tabular MDPs with high computational cost.
method Combines posterior sampling with online learning to achieve asymptotic optimality.
result Achieves optimal sample complexity and posterior contraction rate with O(S2AH) per episode. New algorithm reduces contextual bandit identification to argmax calls.
problem Best-arm identification in stochastic contextual bandits.
method Instance-optimal PAC algorithm using argmax oracle calls.
result First instance-dependent PAC sample complexity for contextual bandits.
New strategies for identifying the best arm in bandits with decreasing variances.
problem Best arm identification in bandits with time-varying variances.
method Two policies: initial wait followed by continuous sampling, and periodic sampling with weighted average.
result Analytical guarantees and simulations show improved performance over existing methods.
In this paper, we consider a best action identification problem in the stochastic linear bandit setup with a fixed confident constraint. In the considered best action identification problem, instead of minimizing the accumulative regret as done in existing works, the learner aims to obtain an accurate estimate of the u…
Improved sample complexity for identifying best policies in risk-sensitive reinforcement learning.
problem Identifying approximately optimal policies in risk-sensitive reinforcement learning with exponential horizon dependence.
method Forward-model based algorithm with KL-based exploration bonuses adapted for entropic criterion, leveraging smoothness properties of exponential utility and a new stopping rule.
result Achieved sample complexity matching the lower bound, closing the gap between upper and lower bounds.
Algorithm identifies best policy in MDPs with adaptive sampling.
problem Best policy identification in discounted MDPs with limited samples.
method Derive lower and upper bounds on sample complexity, design KLB-TS algorithm.
result KLB-TS algorithm achieves nearly-optimal sample allocation.
Study optimizes experimental design for best treatment arm identification.
problem Identifying the best treatment arm given contextual information.
method Adaptive Sampling-Policy Learning (PLAS) strategy for minimax rate optimality.
result PLAS strategy achieves minimax rate optimality in expected simple regret.
New algorithms identify best policies in discounted linear MDPs efficiently.
problem Identifying the best policy in discounted linear MDPs with limited samples.
method Derive lower bounds and devise simple yet near-optimal algorithms.
result Upper bound on sample complexity matches existing bounds.
Develops an anytime-valid framework for optimal policy identification from logged contextual bandit data.
problem Selecting the optimal policy from a candidate policy class while monitoring evidence continuously.
method Constructs a time-indexed set that retains the true optimal policy set uniformly over time.
result The procedure allows the analyst to monitor policy values, eliminate clearly suboptimal policies, and stop at data-dependent times without invalidating inference.
New approach for reward-free exploration reduces estimation error.
problem Reward-free exploration in reinforcement learning.
method Adaptive approach reducing MDP estimation error.
result Reward-free UCRL algorithm improves sample complexity.
Study best arm identification in restless bandits with unknown TPMs.
problem Identify the best arm with fixed confidence in restless bandits with unknown TPMs.
method Proposed a policy for best arm identification and proved its expected stopping time matches the lower bound.
result The state-action visitation proportions match the optimal proportions under any asymptotically optimal policy.
Study non-asymptotic BPI guarantees for online RL.
problem Identify optimal policy in MDP with high confidence.
method Non-asymptotic sample complexity guarantees for NaS algorithm.
result Sample complexity depends on MDP connectivity and curvature.
Study best arm identification in restless Markov multi-armed bandits with state-dependent transitions.
problem Identify the best arm in a multi-armed bandit with time-varying states.
method Propose a sequential policy to select arms without knowing their exact TPMs.
result Upper and lower bounds on expected time to find the best arm match in a special case.
A new method for identifying the best arm in multi-armed bandits with mediator feedback.
problem Sequential decision-making with unknown mediator policies.
method Best-arm identification under mediators' feedback (BAI-MF) with statistical lower bound and sequential decision-making strategy.
result The proposed algorithm matches the lower bound on sample complexity for identifying the best arm.
Study on policy testing in MDPs with lower bounds and new algorithm.
problem Deciding if policy value exceeds a threshold with limited samples.
method Derived lower bound, proposed new algorithm, reformulated problem, used policy optimization in reversed MDP.
result New algorithm outperforms existing methods in policy testing.
Opt-BBAI identifies the best arm with minimal batches and pulls, optimizing both sample and batch complexity.
problem Batched best arm identification (BBAI) problem, aiming to minimize policy switches and resource usage.
method Proposed Opt-BBAI algorithm, achieving near-optimal sample and batch complexity in non-asymptotic settings.
result First algorithm to achieve near-optimal sample and batch complexity in non-asymptotic settings.
New RLHF algorithm identifies optimal policies from human feedback without explicit reward inference.
problem Training large language models with human feedback without reward inference.
method Model-free RLHF algorithm BSAD that identifies optimal policies directly from human preference. result Provable, instance-dependent sample complexity ildeO(cMSA3H3Mlogδ1). Paper proposes a policy gradient method for confounded POMDPs.
problem Estimating policy gradients for confounded POMDPs with continuous state and observation spaces.
method Developed a novel identification result to estimate policy gradients using offline data, solved conditional moment restrictions, and applied min-max learning with function approximation.
result Showed global convergence of the proposed algorithm in finding the optimal policy.
New policy minimizes error in finding best arm with privacy constraints.
problem Best arm identification with differential privacy constraints.
method Maximum absolute determinants principle for ε-DP policy. result Error probability decays exponentially in T with matching exponents. AE-LSVI identifies near-optimal policies in complex systems with minimal data.
problem Identifying near-optimal policies in complex, costly data acquisition systems.
method Combines optimism and pessimism for active exploration in a generative model setting.
result Proves near-optimal policy identification over entire state spaces with polynomial sample complexity.
Efficient algorithms for low-rank bandits using subspace recovery.
problem Contextual bandits with low-rank reward matrices.
method Spectral methods for subspace recovery, reformulating as linear bandits.
result Nearly optimal policy evaluation and best policy identification, minimax guarantees for regret minimization.
PRL improves off-policy evaluation in partially observed MDPs.
problem Confounding and bias in offline reinforcement learning with unobserved state factors.
method Extends proximal causal inference to POMDPs, identifying and estimating target policy value.
result Semiparametrically efficient estimators for PRL in partially observed MDPs.
Combines offline and online learning for identifying the best arm in bandits.
problem Identifying the best arm in stochastic bandits with offline data.
method Lower bound analysis and algorithm development for optimal best-arm identification.
result Developed algorithms matching lower bound on sample complexity for small δ.
A new algorithm identifies one of several nearly optimal arms in linear bandits.
problem Identifying one arm that is close to the best arm in linear bandits.
method Developed a procedure to adapt best-arm identification algorithms for ε-best-answer identification in transductive linear stochastic bandits. result Proposed an asymptotically optimal algorithm for ε-best-answer identification. Optimal best-arm identification in linear bandits reduces sampling budget.
problem Identifying the best arm with fixed confidence in stochastic linear bandits.
method A simple algorithm that tracks an optimal proportion of arm draws, updated as rarely as desired.
result The algorithm's sampling complexity matches known lower bounds, asymptotically almost surely and in expectation.
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy. We introduce an approach based on statistical testing to identify the set of policy parameters the agent can control, within a larger parametric policy space. After present…
The paper provides a non-asymptotic error bound for linear system identification under nonlinear policies.
problem System identification for linear systems with nonlinear and/or time-varying policies under i.i.d. random excitation noises.
method Least square estimation with non-asymptotic error bound for bounded state and action trajectories.
result The error bound is consistent with linear policies and generalizes existing guarantees.
Adaptive exploration scheme for evaluating multiple policies with different rewards.
problem Online multi-reward multi-policy evaluation.
method Adapted (ε,δ)-PAC perspective and MR-NaS exploration scheme to minimize sample complexity. result Demonstrated effectiveness of adaptive exploration in tabular domains.
Open problem: fixed-budget best arm identification complexity.
problem Understanding the complexity of identifying the best arm in a fixed budget setting.
method Analyzing existing results and conjectures in the fixed-confidence setting.
result Open questions remain about the fixed-budget setting.
Contextual information helps identify the best arm more efficiently.
problem Best arm identification with contextual covariate information.
method Proposed a context-aware version of the 'Track-and-Stop' strategy.
result Expected number of arm draws matches lower bound asymptotically.
Computer simulation provides an automatic and safe way for training robotic control policies to achieve complex tasks such as locomotion. However, a policy trained in simulation usually does not transfer directly to the real hardware due to the differences between the two environments. Transfer learning using domain ra…
New findings on complexity limits in fixed budget bandit identification.
problem Determining the best possible error rate for fixed budget bandit identification.
method Analyzing the best non-adaptive sampling procedures and showing the existence of complexities.
result No fixed complexity for certain bandit identification tasks.
New algorithm identifies best arm in rested bandit setting.
problem Best arm identification in rested bandit with decreasing losses.
method Introduced a novel best arm identification problem and analyzed an arm elimination algorithm.
result Regret vanishes as time horizon increases, with convergence rate depending on expected loss function.
New algorithm optimizes best arm identification with minimal regret.
problem Best arm identification in multi-armed bandit problems.
method Characterized Bayesian simple regret with continuity conditions of prior, proposed a simple algorithm.
result Proposed algorithm achieves rate-optimal Bayesian simple regret.
We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial…
Optimal multi-fidelity best-arm identification reduces cost with better accuracy.
problem Finding the best arm with highest mean reward at minimum cost.
method Gradient-based approach with asymptotically optimal cost complexity.
result Asymptotically optimal cost complexity compared to existing methods.
Study robust best-arm identification in linear bandits with lower bounds and algorithms.
problem Identify a near-optimal robust arm in linear bandits with adversarial actions.
method Propose instance-dependent lower bounds and both static and adaptive bandit algorithms.
result Sample complexity matches the lower bound and algorithms effectively identify robust arms.
Study best arm identification with safety constraints in bandit problems.
problem Real-world decision-making with safety constraints.
method Analyzed linear and monotonic reward and safety constraints, proposed algorithms.
result Guaranteed safe learning in both linear and general reward/safety constraint settings.
Motivated by the task of hyperparameter optimization, we introduce the non-stochastic best-arm identification problem. Within the multi-armed bandit literature, the cumulative regret objective enjoys algorithms and analyses for both the non-stochastic and stochastic settings while to the best of our knowledge, the best…
Motivated by drug design, we consider the best-arm identification problem in generalized linear bandits. More specifically, we assume each arm has a vector of covariates, there is an unknown vector of parameters that is common across the arms, and a generalized linear model captures the dependence of rewards on the cov…
A new algorithm for identifying the best arm in linear feedback with safety constraints.
problem Identifying the best arm in linear feedback with safety constraints.
method A gap-based algorithm that ensures safety while minimizing sample complexity.
result The algorithm achieves meaningful sample complexity while ensuring safety.
Improved theoretical guarantees for Top Two algorithms.
problem Theoretical support for best arm identification with bounded distributions.
method General analysis of Top Two methods, identifying desirable properties and replacing sampling step.
result Theoretical support for Top Two algorithms with bounded distributions.
Study evaluates policies in partially observable environments without full model specification.
problem Evaluating policies in partially observable environments without full model specification.
method Developed non-parametric identification and recursive fitted-Q-evaluation algorithm.
result Established finite-sample error bounds for policy value estimation.
Paper tackles best arm identification with cost consideration.
problem Best arm identification with cost consideration in product development.
method Derives a theoretical lower bound and proposes algorithms CTAS and CO.
result Simple algorithms can deliver near-optimal performance.
Paper tackles best mixed arm identification with cost constraints in bandit models.
problem Finding the best mixed arm with cost constraints in a stochastic bandit model.
method Proposes SFSR algorithm combining successive reject and score-function-based rejection criteria.
result Upper and lower bounds on mis-identification probability show exponential decay with budget.
The paper identifies the best treatment to maximize NDPO, a key outcome in causal mediation analysis.
problem Identifying the treatment that maximizes the expected natural direct potential outcome (NDPO) in causal mediation analysis.
method Developed a fixed-confidence best-arm identification (BAI) algorithm based on the Track-and-Stop (TaS) framework, using a cutting-set method to solve a semi-infinite optimization problem.
result The proposed algorithm achieves sample-efficient identification with a high-probability correctness guarantee and asymptotic optimality.