On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

9 results for “UCBVI”

A Refined Analysis of UCBVIstat.ML

Improved analysis of UCBVI algorithm with better empirical performance.

problem Improving the UCBVI algorithm's performance and understanding its bounds.

method Refined analysis of UCBVI algorithm with improved bonus terms and regret analysis.

result Improving multiplicative constants in UCBVI bounds enhances empirical performance.

UCBVI regret analysis empirical performance bonus terms

2025-02-24abs ↗pdf ↗

Federated UCBVI: Communication-Efficient Federated Regret Minimization with Heterogeneous Agentscs.LG

Federated UCBVI reduces communication costs while minimizing regret in multi-agent settings.

problem Minimizing regret in federated learning with heterogeneous agents.

method Federated Upper Confidence Bound Value Iteration (Fed-UCBVI) algorithm.

result Regret bound scales as

ilde{\mathcal{O}}(\sqrt{H^3 |\mathcal{S}| |\mathcal{A}| T / M})

with small additional term due to heterogeneity.

Federated Learning Reinforcement Learning UCBVI Heterogeneity Communication Efficiency

2024-10-30abs ↗pdf ↗

Nearly Minimax Optimal Reinforcement Learning for Discounted MDPscs.LG

UCBVI-γ algorithm minimizes regret in discounted MDPs.

problem Minimizing regret in discounted MDPs.

method Optimism in the face of uncertainty principle and Bernstein-type bonus.

result UCBVI-γ achieves nearly minimax optimal regret.

reinforcement learning Markov Decision Processes discounted MDPs UCBVI-γ regret minimization

2020-10-01abs ↗pdf ↗

Causal Markov Decision Processes: Learning Good Interventions Efficientlystat.ML

New formalism for decision making combines causal structures with MDPs, improving reinforcement learning performance.

problem Sequential decision making with causal knowledge to improve performance.

method Causal Markov Decision Processes (C-MDPs) and C-UCBVI algorithm exploiting causal structure.

result C-UCBVI achieves an

ilde{O}(HS\sqrt{ZT})

regret bound, independent of actions.

Causal Markov Decision Processes Reinforcement Learning Sequential Decision Making Causal Structures UCBVI

2021-02-15abs ↗pdf ↗

From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusesstat.ML

Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.

problem Optimizing exploration in reinforcement learning without bonuses.

method Bayes-UCBVI uses a quantile of a Q-value function posterior as an upper confidence bound.

result Proves a regret bound of order

\widetilde{O}(\sqrt{H^3SAT})

for tabular reinforcement learning.

reinforcement learning upper confidence bound Bayesian methods tabular settings anti-concentration bound

2022-05-16abs ↗pdf ↗

Kernel-Based Reinforcement Learning: A Finite-Time Analysiscs.LG

Kernel-UCBVI algorithm balances exploration and exploitation in metric state-action spaces.

problem Exploration-exploitation dilemma in finite-horizon reinforcement learning with metric state-action spaces.

method Kernel-UCBVI, leveraging smoothness and kernel estimators of rewards and transitions.

result First regret bound for kernel-based RL using smoothing kernels,

O(H^3 K^{2d/(2d+1)})

Kernel-based reinforcement learning Metric state-action spaces Exploration-exploitation Regret bound Finite-horizon problems

2020-04-12abs ↗pdf ↗

Unified Framework of Distributional Regret in Multi-Armed Bandits and Reinforcement Learningcs.LG

Unified framework for distributional regret in bandits and reinforcement learning.

problem Characterizing the distribution of regret in multi-armed bandits and reinforcement learning.

method Unified framework with a UCBVI-style algorithm and distributional regret bounds.

result Distributional regret bounds with optimal trade-offs between expected and distributional regret.

distributional regret multi-armed bandits reinforcement learning UCBVI exploration bonus

2026-05-06abs ↗pdf ↗

Learning Upper Lower Value Envelopes to Shape Online RL: A Principled Approachstat.ML

The paper introduces a method to learn and apply value envelopes for faster online reinforcement learning.

problem Accelerating online reinforcement learning using offline data with theoretical grounding.

method A two-stage framework: offline data for learning value bounds, online algorithms for applying them.

result Substantial regret reductions in empirical tests on tabular MDPs.

reinforcement learning online algorithms value envelopes offline data tabular MDPs

2025-10-22abs ↗pdf ↗

A Theoretical Framework for Partially Observed Reward-States in RLHFcs.LG

Develops a new model for RLHF accounting for partially observed states and intermediate feedback.

problem Lack of models for partially observed states and intermediate feedback in RLHF.

method PORRL model with cardinal and dueling feedback methods.

result Demonstrates improved learning and alignment with new model-based and model-free methods.

reinforcement learning human feedback partial observability cardinal feedback dueling feedback

2024-02-05abs ↗pdf ↗