On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,694 papers · 148 categories

Trend · papers per month

count % of corpus

15 results for “non-episodic”

Ecological Reinforcement Learningcs.LG

This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.

problem Challenges in reinforcement learning with continuous interaction and sparse delayed rewards.

method Analysis of environment shaping and dynamism properties to simplify learning.

result Properties like environment shaping and dynamism can significantly ease learning in non-episodic, sparse reward settings.

reinforcement learning non-episodic sparse rewards environment shaping dynamism

2020-06-22abs ↗pdf ↗

Reinforcement Learning in Factored MDPs: Oracle-Efficient Algorithms and Tighter Regret Bounds for the Non-Episodic Settingstat.ML

We study reinforcement learning in non-episodic factored Markov decision processes (FMDPs). We propose two near-optimal and oracle-efficient algorithms for FMDPs. Assuming oracle access to an FMDP planner, they enjoy a Bayesian and a frequentist regret bound respectively, both of which reduce to the near-optimal bound …

2020-02-06abs ↗pdf ↗

Regret Bounds for Discounted MDPscs.LG

Reinforcement learning (RL) has traditionally been understood from an episodic perspective; the concept of non-episodic RL, where there is no restart and therefore no reliable recovery, remains elusive. A fundamental question in non-episodic RL is how to measure the performance of a learner and derive algorithms to max…

2020-02-12abs ↗pdf ↗

Regret Bounds for Reinforcement Learning via Markov Chain Concentrationcs.LG

We give a simple optimistic algorithm for which it is easy to derive regret bounds of $\tilde{O}(\sqrt{t_{\rm mix} SAT})$ after $T$ steps in uniformly ergodic Markov decision processes with $S$ states, $A$ actions, and mixing time parameter $t_{\rm mix}$ . These bounds are the first regret bounds in the general, non-epi…

2018-08-06abs ↗pdf ↗

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholdingcs.LG

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

problem Challenges in online reinforcement learning for non-episodic, finite-horizon MDPs.

method Introduces a K-step lookahead Q-function with a time-varying threshold for selecting actions.

result Achieves minimax optimal constant regret for K=1 and

\mathcal{O}(\max((K-1),C_{K-1})\sqrt{SAT\log(T)})

regret for K ≥ 2.

reinforcement learning finite horizon lookahead Q-function thresholding tabular learning

2026-01-31abs ↗pdf ↗

Dual Control for Approximate Bayesian Reinforcement Learningstat.ML

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

2015-10-13abs ↗pdf ↗

Thompson Sampling in Non-Episodic Restless Banditscs.LG

Restless bandit problems assume time-varying reward distributions of the arms, which adds flexibility to the model but makes the analysis more challenging. We study learning algorithms over the unknown reward distributions and prove a sub-linear, $O(\sqrt{T}\log T)$ , regret bound for a variant of Thompson sampling. Our…

2019-10-12abs ↗pdf ↗

Oracle-Efficient Regret Minimization in Factored MDPs with Unknown Structurecs.LG

New algorithm learns FMDP structure while minimizing regret.

problem Regret minimization in FMDPs with unknown structure.

method Optimism in face of uncertainty principle combined with statistical structure learning.

result First algorithm to learn FMDP structure while minimizing regret.

regret minimization factored MDPs structure learning optimism principle statistical methods

2020-09-13abs ↗pdf ↗

Learning to Steer Markovian Agents under Model Uncertaintycs.LG

Study designs incentives for adapting multi-agent systems without knowing their learning dynamics.

problem Designing incentives for an adapting population in multi-agent systems without prior knowledge of their learning dynamics.

method Introduces a model-based non-episodic Reinforcement Learning (RL) formulation for steering Markovian agents towards desired policies, focusing on history-dependent strategies to handle model uncertainty.

result Identifies conditions for the existence of steering strategies to guide agents to desired policies and provides empirical algorithms to approximately solve the objective.

Reinforcement Learning Markovian Agents Model Uncertainty Incentives Multi-Agent Systems

2024-07-14abs ↗pdf ↗

A Dataless Reinforcement Learning Approach to Rounding Hyperplane Optimization for Max-Cutcs.LG

A new reinforcement learning method improves Max-Cut solutions without needing training data.

problem Max-Cut problem is NP-hard, and existing methods struggle with generalizability and scalability.

method Training-data-free reinforcement learning approach to hyperplane rounding for Max-Cut optimization.

result Our method consistently achieves better Max-Cut solutions across various graph types.

Max-Cut NP-hard reinforcement learning hyperplane rounding Markov Decision Process

2025-05-19abs ↗pdf ↗

Optimistic Q-learning for average reward and episodic reinforcement learningcs.LG

New algorithm for average reward learning with bounded hitting time assumption.

problem Minimizing regret in average reward reinforcement learning with bounded hitting time.

method Optimistic Q-learning with a novel

\overline{L}

operator for bounded hitting time.

result Regret bound of

ilde{O}(H^5 S\sqrt{AT})

for average reward learning.

reinforcement learning average reward bounded hitting time Q-learning episodic

2024-07-18abs ↗pdf ↗

Settling the Communication Complexity for Distributed Offline Reinforcement Learningstat.ML

This paper sets communication complexity bounds for distributed RL.

problem Establishing minimum communication requirements for distributed RL.

method Information-theoretic lower bounds and algorithm development.

result Developed algorithms achieving optimal risk up to logarithmic factors.

Reinforcement Learning Distributed Systems Communication Complexity Offline Learning Contextual Bandits

2022-02-10abs ↗pdf ↗

The Sample Complexity of Online Reinforcement Learning: A Multi-model Perspectivecs.LG

New algorithm reduces sample complexity for online reinforcement learning.

problem Reducing sample complexity for online reinforcement learning in nonlinear systems.

method Generalized algorithm for various dynamical systems, including neural networks.

result Achieves policy regret of O(Nε^2 + d_u ln(m(ε))/ε^2) in general settings.

Reinforcement Learning Online Learning Sample Complexity Nonlinear Systems Dynamical Models

2025-01-27abs ↗pdf ↗

Multi-armed Bandit Learning on a Graphcs.LG

A graph bandit algorithm learns optimal paths on unknown graphs.

problem Optimal path selection on unknown graphs under uncertainty.

method G-UCB algorithm based on offline graph planning and optimism principle.

result Achieves tight regret bound of

O(\sqrt{|S|T\log(T)}+D|S|\log T)

multi-armed bandit graph bandit optimism principle regret bound graph exploration

2022-09-20abs ↗pdf ↗

Learning in Markov Decision Processes under Constraintscs.LG

UCRL-CMDP algorithm optimizes RL with constraints on average costs.

problem Optimizing RL in MDPs with average cost constraints.

method Model-based RL algorithms maximizing reward while keeping costs within bounds.

result UCRL-CMDP algorithm's expected regret is upper-bounded by $T^{2\slash 3}$.

reinforcement learning Markov decision processes average cost constraints UCRL-CMDP regret analysis

2020-02-27abs ↗pdf ↗