On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “tabular RL”

Q-Star Meets Scalable Posterior Sampling: Bridging Theory and Practice via HyperAgentcs.LG

HyperAgent improves RL exploration in large-scale problems.

problem Efficient exploration in large-scale reinforcement learning problems.

method Hypermodel framework for incremental posterior approximation without conjugacy.

result HyperAgent achieves logarithmic per-step computational complexity and sublinear regret.

reinforcement learning hypermodel posterior sampling large-scale exploration

2024-02-05abs ↗pdf ↗

TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasetscs.LG

TabNAS improves neural architecture search for tabular datasets by rejecting suboptimal architectures.

problem Finding optimal neural architectures for tabular datasets with resource constraints.

method Develops a reinforcement learning controller motivated by rejection sampling to handle resource constraints.

result TabNAS finds better models that obey resource constraints compared to previous methods.

Neural Architecture Search Tabular Datasets Reinforcement Learning Resource Constraints Rejection Sampling

2022-04-15abs ↗pdf ↗

Reinforcement Learning for Monetary Policy Under Macroeconomic Uncertainty: Analyzing Tabular and Function Approximation Methodsq-fin.ST

Tabular Q-learning outperforms advanced RL methods in monetary policy.

problem Dynamic setting of short-term interest rates to stabilize inflation and unemployment under uncertain macroeconomic conditions.

method Discrete-action Markov Decision Process with tabular Q-learning, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning, POMDP formulations.

result Standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return) compared to advanced RL methods and traditional policy rules.

Monetary Policy Reinforcement Learning Tabular Q-learning Bayesian Q-learning Deep Q-Networks

2025-12-09abs ↗pdf ↗

DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guaranteescs.LG

DARLING tackles non-stationary RL with guarantees, improving dynamic regret.

problem Non-stationary reinforcement learning in unknown change points.

method Detection Augmented Reinforcement Learning (DARLING) for tabular and linear MDPs.

result DARLING matches minimax lower bounds in tabular and linear MDPs.

reinforcement learning non-stationary dynamic regret tabular MDPs linear MDPs

2026-04-17abs ↗pdf ↗

Missing Data Multiple Imputation for Tabular Q-Learning in Online RLstat.ML

Paper proposes methods to handle missing data in online RL, improving efficiency and uncertainty capture.

problem Missing data in online RL poses challenges due to the need to impute and act at each time step.

method Proposes fully online imputation ensembles and multiple imputation pathways to balance uncertainty and efficiency.

result Preliminary evidence suggests multiple imputation pathways can be a useful framework for simple and efficient online missing data RL methods.

missing data online RL imputation tabular Q-learning reinforcement learning

2025-10-12abs ↗pdf ↗

Minimax Optimal Reinforcement Learning with Quasi-Optimismcs.LG

EQO uses a simple bonus term for efficient exploration in tabular RL.

problem Achieving minimax optimal reinforcement learning with practical efficiency.

method Exploration via Quasi-Optimism, employing a bonus proportional to state-action visit count.

result Achieves the sharpest known regret bound for tabular RL.

reinforcement learning minimax optimal tabular RL quasi-optimism exploration

2025-03-02abs ↗pdf ↗

A Comparative Analysis of Expected and Distributional Reinforcement Learningcs.LG

Since their introduction a year ago, distributional approaches to reinforcement learning (distributional RL) have produced strong results relative to the standard approach which models expected values (expected RL). However, aside from convergence guarantees, there have been few theoretical results investigating the re…

2019-01-30abs ↗pdf ↗

The Laplacian in RL: Learning Representations with Efficient Approximationscs.LG

The smallest eigenvectors of the graph Laplacian are well-known to provide a succinct representation of the geometry of a weighted graph. In reinforcement learning (RL), where the weighted graph may be interpreted as the state transition process induced by a behavior policy acting on the environment, approximating the …

2018-10-10abs ↗pdf ↗

Hybrid Reinforcement Learning Breaks Sample Size Barriers in Linear MDPsstat.ML

Hybrid RL algorithms improve offline and online RL in linear MDPs.

problem Improving RL performance without single-policy concentrability.

method Developed computationally efficient algorithms for PAC and regret-minimizing RL in linear MDPs.

result Achieved sharper error or regret bounds for linear MDPs.

Hybrid RL PAC RL Regret-minimizing RL Linear MDPs Offline RL

2024-08-08abs ↗pdf ↗

Nearly Optimal Policy Optimization with Stable at Any Time Guaranteecs.LG

New algorithm reduces RL policy optimization gap.

problem Insufficient theoretical understanding of policy optimization methods.

method Reference-based Policy Optimization with Stable at Any Time guarantee (RPO-SAT)

result Achieves nearly minimax optimal policy-based algorithm for tabular RL.

Reinforcement Learning Policy Optimization Tabular RL Stable at Any Time Minimax Optimal

2021-12-21abs ↗pdf ↗

MADE: Exploration via Maximizing Deviation from Explored Regionscs.LG

MADE improves exploration in RL by maximizing deviation from explored regions.

problem Efficient exploration in high-dimensional RL tasks with sparse rewards.

method Proposes a new exploration approach via maximizing the deviation of the occupancy of the next policy from explored regions, adding it as an adaptive regularizer to the RL objective.

result Significantly improves sample efficiency in navigation and locomotion tasks.

reinforcement learning exploration occupancy maximization sparse rewards online learning

2021-06-18abs ↗pdf ↗

Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coveragecs.LG

This work bridges offline RL and DRL to address distributional shift.

problem Distributional shift in offline RL due to difference in state-action visitation distributions.

method Proposes offline RL algorithms using DRL framework, characterizes sample complexity under single policy concentrability.

result Demonstrates superior performance of proposed algorithms through simulations.

offline RL distributionally robust learning distributional shift sample complexity tabular RL

2023-10-27abs ↗pdf ↗

A Reduction-Based Framework for Conservative Bandits and Reinforcement Learningcs.LG

Paper presents a reduction-based framework for conservative bandits and RL with improved lower and upper bounds.

problem Conservative bandits and reinforcement learning problems.

method Reduction technique to calculate necessary and sufficient budget from baseline policy.

result Improved lower and upper bounds for various conservative settings.

Conservative bandits Reinforcement learning Reduction technique Lower bounds Upper bounds

2021-06-22abs ↗pdf ↗

Provably Efficient Exploration for Reinforcement Learning Using Unsupervised Learningcs.LG

New framework uses unsupervised learning for efficient exploration in RL.

problem Efficient exploration in reinforcement learning with rich observations.

method Combines unsupervised learning and no-regret RL algorithms.

result Proves sample complexity for finding near-optimal policies.

reinforcement learning unsupervised learning exploration Markov decision processes tabular RL

2020-03-15abs ↗pdf ↗

Safe Exploration Incurs Nearly No Additional Sample Complexity for Reward-free RLcs.LG

Safe exploration in RF-RL doesn't increase sample complexity.

problem Achieving optimal policies with safety constraints in reward-free RL.

method Proposed SWEET framework for tabular and low-rank MDP settings, leveraging truncated value functions.

result Sample complexities match or outperform constraint-free counterparts, proving safety constraints have little impact.

reward-free reinforcement learning safe exploration sample complexity truncated value functions tabular MDP

2022-06-28abs ↗pdf ↗

Tabular and Deep Reinforcement Learning for Gittins Indexcs.LG

New RL algorithms learn Gittins indices for unknown Markovian states.

problem Learning Gittins indices for unknown Markovian state transitions.

method Tabular (QGI) and Deep RL (DGN) algorithms based on retirement formulation.

result Lower run time, less storage space, better convergence to Gittins index.

Gittins index multi-arm bandit reinforcement learning tabular deep learning

2024-05-02abs ↗pdf ↗

Optimal Sample Complexity of Reinforcement Learning for Mixing Discounted Markov Decision Processescs.LG

New RL theory reduces sample complexity for mixing MDPs.

problem Optimal sample complexity for reinforcement learning in mixing MDPs.

method Regeneration-type ideas to analyze mixing times.

result Optimal sample complexity depends on mixing time, not just discount factor.

reinforcement learning Markov decision process mixing sample complexity regeneration

2023-02-15abs ↗pdf ↗

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholdingcs.LG

New RL method learns K-step lookahead Q-functions for fixed-horizon MDPs.

problem Challenges in online reinforcement learning for non-episodic, finite-horizon MDPs.

method Introduces a K-step lookahead Q-function with a time-varying threshold for selecting actions.

result Achieves minimax optimal constant regret for K=1 and

\mathcal{O}(\max((K-1),C_{K-1})\sqrt{SAT\log(T)})

regret for K ≥ 2.

reinforcement learning finite horizon lookahead Q-function thresholding tabular learning

2026-01-31abs ↗pdf ↗

Optimistic Exploration even with a Pessimistic Initialisationcs.LG

Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient …

2020-02-26abs ↗pdf ↗

List Replicable Reinforcement Learningcs.LG

New RL algorithm ensures stable, replicable policies.

problem Stability and replicability issues in RL algorithms.

method Introduced weak and strong forms of list replicability, developed a novel planning strategy, and tested state reachability.

result Proved efficient tabular RL algorithm with polynomial list complexity.

reinforcement learning replicability tabular RL stability planning strategy

2025-11-29abs ↗pdf ↗

Exact Unlearning in Reinforcement Learningcs.LG

A new RL framework allows removing user data without affecting performance.

problem Efficiently removing user data from a reinforcement learning model without affecting performance.

method Formulated a

ρ

-TV-stable RL algorithm for tabular MDPs that supports exact unlearning.

result Achieved a nearly minimax optimal regret bound of

Ω(H\sqrt{\!SAT}\! +\! {SAH}/ρ)

for

ρ

-TV-stable RL algorithms.

reinforcement learning unlearning tabular MDPs TV-stability regret bound

2026-06-02abs ↗pdf ↗

Many-Goals Reinforcement Learningcs.LG

All-goals updating exploits the off-policy nature of Q-learning to update all possible goals an agent could have from each transition in the world, and was introduced into Reinforcement Learning (RL) by Kaelbling (1993). In prior work this was mostly explored in small-state RL problems that allowed tabular representati…

2018-06-22abs ↗pdf ↗

Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learningcs.LG

Paper proposes a hybrid RL algorithm that combines offline and online data without needing reward info.

problem How to efficiently use online data to improve RL policies using only offline data.

method A three-stage hybrid RL algorithm that uses reward-agnostic exploration and model-based offline RL.

result The hybrid RL algorithm outperforms both pure offline and pure online RL in sample complexity.

hybrid reinforcement learning reward-agnostic offline RL online RL sample complexity

2023-05-17abs ↗pdf ↗

Differentially Private Deep Model-Based Reinforcement Learningcs.LG

PriMORL trains private RL policies on offline data.

problem Private reinforcement learning on offline data.

method PriMORL learns DP models of the environment and optimizes a policy on the penalized private model.

result PriMORL enables training of private RL agents on complex tasks.

differential privacy deep reinforcement learning model-based RL private RL offline RL

2024-02-08abs ↗pdf ↗

Is RLHF More Difficult than Standard RL?cs.LG

Paper shows RLHF can be solved similarly to standard RL.

problem Difficulty of RLHF compared to standard RL.

method Reduction to reward-based RL techniques.

result RLHF can be solved using existing algorithms for reward-based RL.

Reinforcement Learning Human Feedback Reward-based RL Preference-based RL Robust RL

2023-06-25abs ↗pdf ↗

GAN Q-learningstat.ML

Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we p…

2018-05-13abs ↗pdf ↗

Towards Tractable Optimism in Model-Based Reinforcement Learningcs.LG

Optimistic RL algorithms are simplified for deep RL with competitive performance.

problem Achieving accurate optimism in model-based RL for large-scale problems.

method Interpreting scalable optimistic model-based algorithms as solving a tractable noise augmented MDP.

result Competitive regret bound of

ilde{\mathcal{O}}( |\mathcal{S}|H\sqrt{|\mathcal{A}| T } )

for Gaussian noise augmentation.

optimism model-based RL regret bound noise augmented MDP deep RL

2020-06-21abs ↗pdf ↗

TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learningcs.LG

Our understanding of reinforcement learning (RL) has been shaped by theoretical and empirical results that were obtained decades ago using tabular representations and linear function approximators. These results suggest that RL methods that use temporal differencing (TD) are superior to direct Monte Carlo estimation (M…

2018-06-04abs ↗pdf ↗

Optimistic Distributionally Robust Policy Optimizationcs.LG

Develops ODRPO to improve RL algorithms with better performance and stability.

problem RL algorithms converge to sub-optimal solutions due to limited policy representation.

method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.

result Achieves globally optimal policy update and higher sample efficiency.

RL Policy Optimization DRO Trust Region Stability

2020-06-14abs ↗pdf ↗

A Reduction from Reinforcement Learning to No-Regret Online Learningcs.LG

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

2019-11-14abs ↗pdf ↗

Distributional Bellman Operators over Mean Embeddingsstat.ML

New algorithm for RL using mean embeddings of return distributions.

problem Improving reinforcement learning algorithms for dynamic programming.

method Mean embeddings of return distributions, novel algorithms for RL.

result Asymptotic convergence and improved performance in deep RL.

reinforcement learning mean embeddings dynamic programming temporal-difference

2023-12-09abs ↗pdf ↗

Offline Reinforcement Learning with Differential Privacycs.LG

Offline RL algorithms protect privacy while learning from sensitive data.

problem Learning sensitive data in financial, legal, and healthcare applications.

method Differential privacy guarantees for offline RL algorithms.

result Provable prevention of privacy risks with strong learning bounds.

differential privacy offline reinforcement learning privacy protection sensitive data financial applications

2022-06-02abs ↗pdf ↗

Self-Consistent Models and Valuescs.LG

Self-consistent models improve reinforcement learning by aligning predictions with future values.

problem Improving reinforcement learning by aligning model predictions with future values.

method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.

result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.

reinforcement learning self-consistency model-based RL value function policy evaluation

2021-10-25abs ↗pdf ↗

Mutual-Information Regularization in Markov Decision Processes and Actor-Critic Learningcs.LG

Cumulative entropy regularization introduces a regulatory signal to the reinforcement learning (RL) problem that encourages policies with high-entropy actions, which is equivalent to enforcing small deviations from a uniform reference marginal policy. This has been shown to improve exploration and robustness, and it ta…

2019-09-11abs ↗pdf ↗

Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goalscs.LG

Consider mutli-goal tasks that involve static environments and dynamic goals. Examples of such tasks, such as goal-directed navigation and pick-and-place in robotics, abound. Two types of Reinforcement Learning (RL) algorithms are used for such tasks: model-free or model-based. Each of these approaches has limitations.…

2018-09-25abs ↗pdf ↗

Active Reinforcement Learning with Monte-Carlo Tree Searchcs.LG

Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …

2018-03-13abs ↗pdf ↗

Offline Reinforcement Learning via Linear-Programming with Error-Bound Induced Constraintscs.LG

New offline RL algorithm with optimal sample complexity using LP and error bounds.

problem Finding optimal policies from offline data with limited coverage and function approximation.

method Developed a new LP reformulation with error bounds and constraints for offline RL.

result Achieved optimal

O(1/\sqrt{n})

sample complexity under various assumptions.

Offline Reinforcement Learning Linear Programming Error Bounds Markov Decision Processes Sample Complexity

2022-12-28abs ↗pdf ↗

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Gamecs.LG

New algorithm for offline RL with linear approx in MDPs and MGs, nearly optimal.

problem Offline RL with linear function approximation in MDPs and MGs.

method Pessimism-based algorithm with uncertainty decomposition via reference function.

result Nearly minimax optimal performance in offline RL for MDPs and MGs.

Offline RL Linear function approximation MDP MG Pessimism

2022-05-31abs ↗pdf ↗

Near-Optimal Offline Reinforcement Learning via Double Variance Reductioncs.LG

New algorithm reduces offline RL data requirements significantly.

problem Optimizing policies using only historical data in reinforcement learning.

method Off-Policy Double Variance Reduction (OPDVR) algorithm.

result OPDVR achieves optimal sample complexity with

O(H^2/d_mε^2)

episodes.

reinforcement learning offline RL variance reduction policy optimization sample complexity

2021-02-02abs ↗pdf ↗

Reward Estimation for Variance Reduction in Deep Reinforcement Learningcs.LG

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

2018-05-09abs ↗pdf ↗

Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RLcs.LG

Optimistic NPG improves policy optimization in online RL with efficient sample complexity.

problem Limited theoretical understanding of policy optimization, especially in online RL.

method Combines natural policy gradient with optimistic policy evaluation.

result Achieves optimal dimension dependence sample complexity for learning near-optimal policies.

Reinforcement Learning Policy Optimization Online RL Natural Policy Gradient Optimistic Policy Evaluation

2023-05-18abs ↗pdf ↗

Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscs.LG

New BE dimension measure reveals rich RL problems with sample-efficient algorithms.

problem Finding sample-efficient algorithms for complex RL problems.

method Introducing Bellman Eluder (BE) dimension and designing GOLF and OLIVE algorithms.

result GOLF and OLIVE algorithms learn near-optimal policies for low BE dimension problems with polynomial samples.

Bellman Eluder dimension sample-efficient learning Reinforcement Learning optimization-based algorithms hypothesis elimination

2021-02-01abs ↗pdf ↗

Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regretquant-ph

Quantum RL algorithm achieves logarithmic regret for exploration.

problem Designing efficient quantum RL algorithms for exploration.

method UCRL-style quantum algorithm with lazy updating and quantum estimation.

result Proves

\mathcal{O}(\mathrm{poly}(S, A, H, \log T))

worst-case regret.

quantum reinforcement learning exploration-exploitation regret minimization lazy updating quantum estimation

2023-02-21abs ↗pdf ↗

Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learningcs.LG

Deep RL controls anesthesia more accurately than traditional methods.

problem Controlling the level of unconsciousness during anesthesia.

method Deep Reinforcement Learning (DRL) to map patient state to propofol dosage.

result Deep RL model outperformed traditional controllers (1.7% vs 3.4% median absolute performance error).

Reinforcement Learning Anesthesia Propofol Deep Neural Network Pharmacokinetics

2020-08-27abs ↗pdf ↗

The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescs.LG

New algorithm finds Nash equilibrium in multi-agent games.

problem Extending RL theory to multi-agent settings with large state spaces.

method Proposes an algorithm using an exploiter to find Nash equilibrium policies.

result Proves finding Nash equilibrium is possible with polynomial samples.

reinforcement learning multi-agent Markov Games Nash equilibrium exploiter

2021-06-07abs ↗pdf ↗

Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcs.LG

Improved RL algorithm for robustness against parameter mismatches.

problem Learning robust control policies against parameter mismatches between training and testing environments.

method Formulated as DR-RL problem, proposed RPVL algorithm for tabular episodic learning with four divergences.

result Achieved

ilde{\mathcal{O}}(|\mathcal{S}||\mathcal{A}| H^{5})

sample complexity uniformly better than existing results.

Reinforcement Learning Robustness Distributionally Robust Tabular Episodic Wasserstein

2023-03-05abs ↗pdf ↗

Provably Efficient CVaR RL in Low-rank MDPscs.LG

New RL algorithm maximizes CVaR in low-rank MDPs with provable efficiency.

problem Maximizing CVaR in large state spaces with function approximation.

method Upper Confidence Bound (UCB) bonus-driven algorithm for low-rank MDPs.

result Achieves sample complexity of O(H^7 A^2 d^4 / τ^2 ε^2) for ε-optimal CVaR.

Reinforcement Learning Conditional Value at Risk Low-rank MDPs Upper Confidence Bound Function Approximation

2023-11-20abs ↗pdf ↗

On Reward-Free Reinforcement Learning with Linear Function Approximationcs.LG

New algorithm for reward-free RL with linear function approximation, reducing sample complexity.

problem Efficiently learning optimal policies without prior reward information in complex environments.

method Developed an algorithm for reward-free RL in linear Markov decision processes, proving sample complexity bounds.

result Polynomial sample complexity in feature dimension and planning horizon, independent of states and actions.

reinforcement learning reward-free linear function approximation Markov decision processes sample complexity

2020-06-19abs ↗pdf ↗