Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,051 papers · 148 categories

Trend · papers per month

0111 · Oct 201719922001200920172026
6 results for tree-backup

TBQ(σ) improves trace utilization in off-policy reinforcement learning.

problem Efficiency of trace utilization under greedy target policies.
method Introduces TBQ(σ) that unifies tree-backup and Naive Q(λ) with a new parameter σ.
result TBQ(σ) improves efficiency in trace utilization, accelerating learning and performance.

The paper analyzes off-policy TD-learning using generalized Bellman operators and provides finite-sample bounds.

problem High variance in off-policy TD-learning due to importance sampling.
method Derives finite-sample bounds for off-policy TD-like algorithms using generalized Bellman operators.
result First-known finite-sample guarantees for several off-policy TD algorithms.

This study evaluates multi-step reinforcement learning methods in the Mountain Car environment.

problem Lack of statistical significance in evaluating multi-step reinforcement learning methods.
method Combines n-step action-value algorithms with DQN architecture, tests in Mountain Car environment.
result Performance varies with off-policy correction, backup length, and target network update frequency.

Reinforcement learning methods carry a well known bias-variance trade-off in n-step algorithms for optimal control. Unfortunately, this has rarely been addressed in current research. This trade-off principle holds independent of the choice of the algorithm, such as n-step SARSA, n-step Expected SARSA or n-step Tree bac…

2018-10-16abs ↗pdf ↗