On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “Bayesian Reinforcement Learning”

Inferential Induction: A Novel Framework for Bayesian Reinforcement Learningcs.LG

Novel framework for Bayesian reinforcement learning infers value function distributions.

problem Bayesian reinforcement learning's challenges in inferring value function distributions.

method Inferential Induction framework for Bayesian reinforcement learning, developing Bayesian Backwards Induction algorithm.

result Proposed algorithm is competitive with state-of-the-art methods.

Bayesian reinforcement learning value function distribution Bayesian Backwards Induction Inferential Induction

2020-02-08abs ↗pdf ↗

Bayesian Meta-Reinforcement Learning with Laplace Variational Recurrent Networkscs.LG

Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.

problem Improving meta-reinforcement learning by providing full posterior distributions.

method Augmenting point estimates with Laplace approximation for full posterior distributions.

result Our method performs similarly to variational baselines with fewer parameters.

Bayesian meta-reinforcement learning Laplace approximation Recurrent neural networks Meta-learning Variational inference

2025-05-24abs ↗pdf ↗

An Information-Theoretic Analysis of Bayesian Reinforcement Learningcs.LG

Study on Bayesian reinforcement learning performance bounds.

problem Achieving optimal performance in model-based Bayesian reinforcement learning.

method Defining minimum Bayesian regret, deriving upper bounds using relative entropy and Wasserstein distance, and applying these to specific MDP cases.

result Upper bounds on minimum Bayesian regret for MDPs, including specific cases like MAB and online optimization with partial feedback.

Bayesian reinforcement learning Markov decision processes Minimum Bayesian regret Relative entropy Wasserstein distance

2022-07-18abs ↗pdf ↗

Bayesian Meta-reinforcement Learning for Traffic Signal Controlcs.LG

Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.

problem Lack of robustness and stability in adaptation for traffic signal control.

method Value-based Bayesian meta-reinforcement learning framework BM-DQN with fast-adaptation variation and DQN fast-update advantage.

result Framework adapts more quickly and robustly to new scenarios than previous methods.

Bayesian meta-reinforcement learning traffic signal control robustness continual learning heterogeneous scenarios

2020-10-01abs ↗pdf ↗

Gaussian Process Policy Optimizationcs.LG

Bayesian approach uses Gaussian process for reinforcement learning.

problem Robotic locomotion environments

method Bayesian actor-critic, model-free reinforcement learning with Gaussian process for exploration and policy optimization.

result Gaussian process method outperforms current algorithms in robotic locomotion environments.

Bayesian reinforcement learning Gaussian process actor-critic reinforcement learning robotics

2020-03-02abs ↗pdf ↗

Improved Bayesian Regret Bounds for Thompson Sampling in Reinforcement Learningstat.ML

Paper improves Bayesian regret bounds for Thompson Sampling in reinforcement learning.

problem Improving Bayesian regret bounds for Thompson Sampling in reinforcement learning.

method Using a discrete set of surrogate environments and posterior consistency analysis, the authors derive an upper bound of order

O(H\sqrt{d_{l_1}T})

result The derived upper bound of

O(H\sqrt{d_{l_1}T})

is a significant improvement over previous bounds.

Thompson Sampling Reinforcement Learning Bayesian Regret Information Ratio Surrogate Environments

2023-10-30abs ↗pdf ↗

PAC-Bayesian Reinforcement Learning Trains Generalizable Policiescs.LG

New reinforcement learning bound improves generalization for sequential data.

problem Challenges in obtaining generalization guarantees for reinforcement learning due to sequential data.

method PAC-Bayesian reinforcement learning with consideration of Markov dependencies and mixing time.

result Demonstrated practical utility through PB-SAC, providing meaningful confidence certificates.

PAC-Bayesian Reinforcement Learning Generalization Sequential Data Soft Actor-Critic

2025-10-12abs ↗pdf ↗

Foundations of Reinforcement Learning and Interactive Decision Makingcs.LG

Lecture notes on reinforcement learning using statistical methods.

problem Addressing the exploration-exploitation dilemma in decision making.

method Frequentist and Bayesian approaches, function approximation, neural networks.

result Unified framework for decision making and estimation.

reinforcement learning exploration-exploitation Bayesian statistics frequentist statistics function approximation

2023-12-27abs ↗pdf ↗

PAC-Bayesian Policy Evaluation for Reinforcement Learningcs.LG

Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors. PAC-Bayesian methods overcome this problem by providing bounds that hold regardless …

2012-02-14abs ↗pdf ↗

Prior-dependent analysis of posterior sampling reinforcement learning with function approximationstat.ML

Enhances RL with function approximation, improving regret bounds.

problem Improving exploration in reinforcement learning with function approximation.

method Prior-dependent Bayesian regret bound for PSRL with linear mixture MDPs, using value-targeted model learning and variance reduction.

result Established an upper bound of

{\mathcal{O}}(d\sqrt{H^3 T \log T})

for PSRL.

reinforcement learning function approximation PSRL Bayesian regret linear mixture MDPs

2024-03-17abs ↗pdf ↗

Full Bayesian Reinforcement Learning via LF-IBISstat.ML

LF-IBIS learns optimal policies online without explicit likelihood.

problem Bayesian RL challenges due to intractable likelihood functions.

method Combines ABC with IBIS for online belief updates.

result Approximates posterior distributions for policies and parameters.

Bayesian RL Reinforcement Learning Likelihood-Free Approximate Bayesian Computation Iterated Batch Importance Sampling

2026-07-02abs ↗pdf ↗

Variational Sequential Optimal Experimental Design using Reinforcement Learningstat.ML

vsOED optimizes experiment design with reinforcement learning for Bayesian models.

problem Optimizing the sequence of experiments in Bayesian models for efficient data collection.

method Reinforcement learning with variational posterior approximations to optimize design policy.

result vsOED achieves superior sample efficiency compared to existing methods.

reinforcement learning Bayesian optimization experimental design sequential decision making variational inference

2023-06-17abs ↗pdf ↗

Quantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learningcs.LG

Bayesian optimization outperforms other methods in hyperparameter tuning for reinforcement learning.

problem Finding optimal hyperparameters that generalize across random seeds in reinforcement learning.

method Benchmarked Successive Halving, Random Search, and Bayesian Optimization with and without repetitions on PPO2 algorithms for Cartpole and Inverted Pendulum tasks.

result Bayesian optimization with noise robust acquisition function is the best choice.

reinforcement learning hyperparameter optimization Bayesian optimization successive halving random search

2020-07-29abs ↗pdf ↗

Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problemscs.LG

The paper solves IRL for Bayesian stopping time problems.

problem Identifying optimal actions in Bayesian stopping time problems.

method Novel IRL framework using Bayesian revealed preferences.

result Identifies optimality and constructs cost function estimates.

Inverse Reinforcement Learning Bayesian Stopping Time Revealed Preferences YouTube Engagement Hypothesis Testing

2020-07-07abs ↗pdf ↗

Robust Bayesian reinforcement learning through tight lower boundscs.LG

In the Bayesian approach to sequential decision making, exact calculation of the (subjective) utility is intractable. This extends to most special cases of interest, such as reinforcement learning problems. While utility bounds are known to exist for this problem, so far none of them were particularly tight. In this pa…

2011-06-18abs ↗pdf ↗

Exploration by Distributional Reinforcement Learningcs.LG

We propose a framework based on distributional reinforcement learning and recent attempts to combine Bayesian parameter updates with deep reinforcement learning. We show that our proposed framework conceptually unifies multiple previous methods in exploration. We also derive a practical algorithm that achieves efficien…

2018-05-04abs ↗pdf ↗

Optimizing Sequential Experimental Design with Deep Reinforcement Learningcs.LG

We use deep reinforcement learning to optimize experimental designs efficiently.

problem Optimizing sequential experimental designs with limited exploration and black-box models.

method Reduced the optimal design problem to an MDP and solved it with deep reinforcement learning.

result Our approach achieves state-of-the-art performance on both continuous and discrete design spaces.

reinforcement learning sequential design experimental optimization deep learning Markov decision process

2022-02-02abs ↗pdf ↗

Generalized Bayesian deep reinforcement learningstat.ML

Bayesian RL tackles uncertainty with deep generative models and sequential samplers.

problem Optimal decision-making in uncertain environments with limited data.

method Bayesian approach using deep generative models and prequential scoring rule for posterior inference. Policy learning via expected Thompson sampling.

result Improves policy learning in high-dimensional parameter spaces and continuous action spaces.

Bayesian reinforcement learning deep generative models sequential Monte Carlo prequential scoring rule Thompson sampling

2024-12-16abs ↗pdf ↗

Preference elicitation and inverse reinforcement learningstat.ML

We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent's preferences, policy and optio…

2011-04-29abs ↗pdf ↗

Monte-Carlo utility estimates for Bayesian reinforcement learningcs.LG

This paper introduces a set of algorithms for Monte-Carlo Bayesian reinforcement learning. Firstly, Monte-Carlo estimation of upper bounds on the Bayes-optimal value function is employed to construct an optimistic policy. Secondly, gradient-based algorithms for approximate upper and lower bounds are introduced. Finally…

2013-03-11abs ↗pdf ↗

Gradient-EM Bayesian Meta-learningcs.LG

Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.

problem Efficient and robust adaptation to new tasks with uncertainty assessment.

method Extends Bayesian meta-learning with gradient-EM algorithm, decoupling inner-update from meta-update.

result Improves accuracy with less computation cost and enhanced robustness to uncertainty.

Bayesian meta-learning gradient-EM deep neural networks few-shot learning reinforcement learning

2020-06-21abs ↗pdf ↗

Bayesian Reinforcement Learning: A Surveycs.AI

Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporati…

2016-09-14abs ↗pdf ↗

ε-BMC: A Bayesian Ensemble Approach to Epsilon-Greedy Exploration in Model-Free Reinforcement Learningcs.LG

Bayesian approach improves $\varepsilon$ -greedy exploration in RL.

problem Improving

\varepsilon

-greedy exploration in model-free RL.

method Introducing a Bayesian model update for

\varepsilon

based on BMC.

result Proposed

\varepsilon

- exttt{BMC} algorithm efficiently balances exploration and exploitation.

Bayesian Reinforcement Learning Epsilon-Greedy Exploration Model-Free RL Monotone Convergence Q-value Function

2020-07-02abs ↗pdf ↗

Sample-efficient reinforcement learning using deep Gaussian processesstat.ML

Improved sample efficiency in reinforcement learning with deep Gaussian processes.

problem Efficiently learn to control actions with limited interaction data.

method Deep Gaussian processes that simulate dynamics with depth and prior knowledge.

result Significantly improved early sample efficiency across various tasks, including half-cheetah control.

reinforcement learning sample efficiency Gaussian processes model-based Bayesian posterior

2020-11-02abs ↗pdf ↗

Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcs.LG

Bayesian sOED uses PG reinforcement learning for efficient experiment design.

problem Optimizing sequential experiments for nonlinear models with limited data.

method Formulated as POMDP, solved via PG methods with neural network parameterization.

result Demonstrated advantages over batch and greedy designs in contaminant source inversion.

Bayesian Optimal Experimental Design Reinforcement Learning Sequential Design Nonlinear Models

2021-10-28abs ↗pdf ↗

ABC Reinforcement Learningstat.ML

This paper introduces a simple, general framework for likelihood-free Bayesian reinforcement learning, through Approximate Bayesian Computation (ABC). The main advantage is that we only require a prior distribution on a class of simulators (generative models). This is useful in domains where an analytical probabilistic…

2013-03-27abs ↗pdf ↗

Cover Tree Bayesian Reinforcement Learningstat.ML

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tr…

2013-05-08abs ↗pdf ↗

Bayesian Inverse Reinforcement Learning for Collective Animal Movementcs.LG

Bayesian method infers local rules for collective animal movement.

problem Learn local rules governing long-term group behaviors.

method Bayesian Inverse Reinforcement Learning with Linearly-Solvable Markov Decision Process.

result Recover true costs and find value of collective movement.

Bayesian Inverse Reinforcement Learning Collective Animal Movement Linearly-Solvable Markov Decision Process Agent-based Models Markov Decision Process

2020-09-08abs ↗pdf ↗

Bayesian Inverse Transition Learning: Learning Dynamics From Near-Optimal Trajectoriescs.LG

Bayesian method estimates dynamics from near-optimal trajectories.

problem Estimating dynamics from near-optimal expert trajectories in reinforcement learning.

method Constraint-based Bayesian approach integrating expert near-optimality.

result Significant improvements in decision-making and transfer success.

Bayesian Inverse Transition Learning Reinforcement Learning Near-Optimal Trajectories Dynamics Estimation Constraint-Based Methods

2024-11-07abs ↗pdf ↗

ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisitionstat.ML

ALINE optimizes Bayesian inference and data acquisition by strategically querying informative data.

problem Strategic acquisition of informative data for Bayesian inference in challenging tasks.

method Unified framework combining amortized Bayesian inference and active data acquisition using a transformer architecture trained via reinforcement learning.

result ALINE delivers both instant and accurate inference along with efficient selection of informative points.

Bayesian inference active learning reinforcement learning transformer information gain

2025-06-08abs ↗pdf ↗

Continuous-Time Model-Based Reinforcement Learningcs.LG

Continuous-time MBRL framework tackles control systems with Bayesian ODEs.

problem Discretization of continuous-time systems in MBRL.

method Novel actor-critic method with Bayesian ODEs for state inference.

result Model robust against irregular and noisy data, sample-efficient, solves challenging control problems.

continuous-time model-based reinforcement learning Bayesian ODEs actor-critic control systems

2021-02-09abs ↗pdf ↗

Searching for ribbons with machine learningmath.GT

Machine learning finds knots that bound ribbon disks.

problem Detecting ribbon knots in topology.

method Bayesian optimization and reinforcement learning.

result Successfully detected many ribbon knots up to 70 crossings.

knot theory ribbons machine learning topology Bayesian optimization

2023-04-18abs ↗pdf ↗

Entropy-regularized Gradient Estimators for Approximate Bayesian Inferencecs.LG

This paper improves Bayesian inference for predictive models with limited data.

problem Effective uncertainty quantification for training predictive models with limited data.

method Entropy-regularized gradient estimators to approximate the Bayesian posterior.

result The method generates diverse samples from the posterior distribution efficiently.

Bayesian inference Approximate Bayesian computation Gradient estimation Entropy regularization Model-based reinforcement learning

2025-03-15abs ↗pdf ↗

Analytically Tractable Bayesian Deep Q-Learningcs.LG

Paper presents a new method for Bayesian deep learning that scales to Atari games.

problem Training neural networks on complex environments like Atari games is challenging.

method Adapted temporal difference Q-learning to work with Bayesian inference.

result TAGI allows for analytical inference of neural network parameters, achieving performance comparable to gradient-based methods.

Bayesian deep learning Atari games temporal difference Q-learning TAGI neural network parameters

2021-06-21abs ↗pdf ↗

A Bayesian latent class reinforcement learning framework to capture adaptive, feedback-driven travel behaviourcs.LG

Bayesian model identifies three types of travelers adapting to feedback.

problem Capturing adaptive, feedback-driven travel behavior in heterogeneous individuals.

method Latent Class Reinforcement Learning (LCRL) model with Variational Bayes estimation.

result Three distinct traveler classes identified: context-dependent, persistent exploitative, and exploratory.

travel behavior adaptive learning reinforcement learning latent class Bayesian statistics

2025-12-08abs ↗pdf ↗

SymCircuit: Bayesian Structure Inference for Tractable Probabilistic Circuits via Entropy-Regularized Reinforcement Learningcs.LG

SymCircuit learns PC structure via entropy-regularized RL, improving inference efficiency and accuracy.

problem Greedy algorithms in PC structure learning lead to suboptimal solutions.

method Entropy-regularized reinforcement learning to train a learned generative policy for PC structure inference.

result SymCircuit learns the optimal policy as a tempered Bayesian posterior, improving inference efficiency and accuracy.

probabilistic circuits reinforcement learning structure learning Bayesian inference entropy regularization

2026-03-20abs ↗pdf ↗

PAC-Bayesian Soft Actor-Critic Learningcs.LG

New PAC-Bayesian approach stabilizes actor-critic learning.

problem Training instability in actor-critic algorithms.

method Employing PAC-Bayesian bound as the critic training objective.

result Significant improvement in online learning performance.

actor-critic reinforcement learning PAC-Bayesian Soft Actor-Critic stability

2023-01-30abs ↗pdf ↗

Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescs.LG

Bayesian optimization technique scaled using Vecchia approximations.

problem Scalability issue with Gaussian process surrogate models in Bayesian optimization.

method Adapted Vecchia approximation from spatial statistics to Gaussian processes, developed improvements and extensions.

result Methods compared favorably to state-of-the-art on various test functions and reinforcement learning problems.

Bayesian optimization Gaussian processes Vecchia approximation scalability Thompson sampling

2022-03-02abs ↗pdf ↗

Optimizing Algorithms for Mobile Health Interventions with Active Querying Optimizationcs.LG

Bayesian ATM improves stability and efficiency in mobile health interventions.

problem Balancing intervention efficacy with user burden in mobile health interventions.

method Bayesian extension to ATM using Kalman filter-style updates.

result Bayesian ATM achieves comparable or improved scalarized returns with lower variance and more stable policy behavior.

Bayesian Reinforcement Learning Mobile Health Kalman Filter Active Querying Optimization Reinforcement Learning

2025-11-27abs ↗pdf ↗

Bayesian Robust Optimization for Imitation Learningcs.LG

Bayesian Robust Optimization for Imitation Learning (BROIL) balances risk and reward.

problem Learning robust policies for new states in imitation learning.

method Bayesian reward function inference and user-specific risk tolerance.

result BROIL outperforms risk-sensitive and risk-neutral algorithms.

Bayesian Robust Optimization Imitation Learning Risk-Tolerance Inverse Reinforcement Learning

2020-07-24abs ↗pdf ↗

In-context Exploration-Exploitation for Reinforcement Learningcs.LG

ICEE learns new RL tasks in less time with a Transformer model.

problem Efficient in-context policy learning for reinforcement learning.

method In-context Exploration-Exploitation (ICEE) algorithm that optimizes efficiency without explicit Bayesian inference.

result ICEE solves Bayesian optimization problems as efficiently as Gaussian process biased methods but in significantly less time.

reinforcement learning in-context learning exploration-exploitation Transformer model Bayesian inference

2024-03-11abs ↗pdf ↗

Using Image Transformations to Learn Network Structurestat.ML

Image compression techniques reveal network structure for shipping box optimization.

problem Designing and planning shipping box networks between nodes.

method Image transformations and Bayesian reinforcement learning.

result Learned network structure can recommend future connectivity.

image compression network structure reinforcement learning shipping networks Bayesian inference

2021-12-06abs ↗pdf ↗

Robust Bayesian Dynamic Programming for On-policy Risk-sensitive Reinforcement Learningq-fin.RM

Novel framework for risk-sensitive reinforcement learning with robustness against uncertainty.

problem Risk-sensitive reinforcement learning with uncertainty in transition dynamics.

method Developed a risk-sensitive robust Markov decision process (RSRMDP), derived its Bellman equation, and proposed a Bayesian Dynamic Programming (Bayesian DP) algorithm.

result Demonstrated convergence to near-optimal policies and analyzed sample and computational complexities.

reinforcement learning risk-sensitive dynamic programming robustness Bayesian

2025-12-31abs ↗pdf ↗

Bayesian Nonparametrics for Offline Skill Discoverycs.LG

Bayesian nonparametrics enables dynamic skill discovery from expert data.

problem Fixed K for offline skill discovery in reinforcement learning.

method Variational inference, continuous relaxations, Bayesian nonparametrics.

result Nonparametric model with dynamically-changing number of options.

Bayesian nonparametrics offline skill discovery reinforcement learning variational inference continuous relaxations

2022-02-09abs ↗pdf ↗

Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stabilitycs.LG

Regularization improves generalization in Bayesian RL, shown through algorithmic stability.

problem Ensuring good generalization in Bayesian reinforcement learning.

method Algorithmic stability, using regularization and fast convergence rates for mirror descent.

result Regularization makes the optimal policy stable, improving generalization.

Bayesian reinforcement learning algorithmic stability meta-RL PAC framework Markov decision processes

2021-09-24abs ↗pdf ↗

Bayesian Transfer Reinforcement Learning with Prior Knowledge Rulescs.LG

We propose a probabilistic framework to directly insert prior knowledge in reinforcement learning (RL) algorithms by defining the behaviour policy as a Bayesian posterior distribution. Such a posterior combines task specific information with prior knowledge, thus allowing to achieve transfer learning across tasks. The …

2018-09-30abs ↗pdf ↗

MFRL-BI: Design of a Model-free Reinforcement Learning Process Control Scheme by Using Bayesian Inferencecs.LG

MFRL-BI controls manufacturing processes without needing accurate models.

problem Model inaccuracies in complex manufacturing systems.

method Model-free reinforcement learning with Bayesian inference.

result Demonstrated to perform well in a nonlinear CMP process.

reinforcement learning Bayesian inference model-free control manufacturing process Bayesian disturbance

2023-09-17abs ↗pdf ↗

Variance-Based Rewards for Approximate Bayesian Reinforcement Learningcs.LG

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

2012-03-15abs ↗pdf ↗