arXiv research

This paper shows RL with KL penalties is equivalent to Bayesian inference for fine-tuning LMs.

Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporati…

Regularization improves generalization in Bayesian RL, shown through algorithmic stability.

Paper uses RL and diffusion models to solve Bayesian inverse problems.

Bayesian RL enhances LLMs to reflectively explore and correct errors.

We propose a probabilistic framework to directly insert prior knowledge in reinforcement learning (RL) algorithms by defining the behaviour policy as a Bayesian posterior distribution. Such a posterior combines task specific information with prior knowledge, thus allowing to achieve transfer learning across tasks. The …

Bayesian approach improves $\varepsilon$-greedy exploration in RL.

Offline RL policies should adapt to unknown aspects of the environment.

Paper uses RL to optimize ICU load during COVID-19.

BCPO optimizes offline RL policies by converting uncertainty into conservative bounds.

In preference-based reinforcement learning (RL), an agent interacts with the environment while receiving preferences instead of absolute feedback. While there is increasing research activity in preference-based RL, the design of formal frameworks that admit tractable theoretical analysis remains an open challenge. Buil…

Enhances RL with function approximation, improving regret bounds.

We study reinforcement learning (RL) in high dimensional episodic Markov decision processes (MDP). We consider value-based RL when the optimal Q-value is a linear function of d-dimensional state-action feature representation. For instance, in deep-Q networks (DQN), the Q-value is a linear function of the feature repres…

New methods use RL and DA to improve dosing precision and reduce side effects.

LF-IBIS learns optimal policies online without explicit likelihood.

Bayesian Reinforcement Learning (RL) is capable of not only incorporating domain knowledge, but also solving the exploration-exploitation dilemma in a natural way. As Bayesian RL is intractable except for special cases, previous work has proposed several approximation methods. However, these methods are usually too sen…

Extends Thompson sampling for RL with fewer episodes.

Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We describe and study in these models a decomposition of predictive uncertainty into its epistemic and aleatoric components. First, we show how such a decomposition ar…

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

BOMS enhances offline MBRL by improving model selection with Bayesian optimization.

Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then…

ICEE learns new RL tasks in less time with a Transformer model.

A common problem in Machine Learning and statistics consists in detecting whether the current sample in a stream of data belongs to the same distribution as previous ones, is an isolated outlier or inaugurates a new distribution of data. We present a hierarchical Bayesian algorithm that aims at learning a time-specific…

This study assesses model influence on RL algorithm performance.

Tabular Q-learning outperforms advanced RL methods in monetary policy.

CAGES optimizes expensive RL problems by efficiently learning gradients from multiple sources.

Meta-agent learns effective exploration from offline data.

Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors. PAC-Bayesian methods overcome this problem by providing bounds that hold regardless …

This work uses RL to optimize batch experiments in SDOE.

We introduce a deep generative model for functions. Our model provides a joint distribution p(f, z) over functions f and latent variables z which lets us efficiently sample from the marginal p(f) and maximize a variational lower bound on the entropy H(f). We can thus maximize objectives of the form E_{f~p(f)}[R(f)] + c…

ContraBAR uses contrastive learning to learn Bayes-optimal policies in RL.

Scalable and effective exploration remains a key challenge in reinforcement learning (RL). While there are methods with optimality guarantees in the setting of discrete state and action spaces, these methods cannot be applied in high-dimensional deep RL scenarios. As such, most contemporary RL relies on simple heuristi…

Reinforcement learning (RL) methods often rely on massive exploration data to search optimal policies, and suffer from poor sampling efficiency. This paper presents a mixed reinforcement learning (mixed RL) algorithm by simultaneously using dual representations of environmental dynamics to search the optimal policy wit…

Develops RL algorithm for non-Markovian, non-stationary reward streams.

In record linkage (RL), or exact file matching, the goal is to identify the links between entities with information on two or more files. RL is an important activity in areas including counting the population, enhancing survey frames and data, and conducting epidemiological and follow-up studies. RL is challenging when…

Reinforcement Learning (RL) has demonstrated state-of-the-art results in a number of autonomous system applications, however many of the underlying algorithms rely on black-box predictions. This results in poor explainability of the behaviour of these systems, raising concerns as to their use in safety-critical applica…

Bayesian optimization improves policy search in reinforcement learning.

Exploration in environments with continuous control and sparse rewards remains a key challenge in reinforcement learning (RL). Recently, surprise has been used as an intrinsic reward that encourages systematic and efficient exploration. We introduce a new definition of surprise and its RL implementation named Variation…

Bayesian RL tackles uncertainty with deep generative models and sequential samplers.

Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessary - a prohibitive operation with large models, and an impossible one with RL. We …

Develops a personalized reinforcement learning algorithm for dyadic health interventions.

SymCircuit learns PC structure via entropy-regularized RL, improving inference efficiency and accuracy.

In reinforcement learning (RL), agents often operate in partially observed and uncertain environments. Model-based RL suggests that this is best achieved by learning and exploiting a probabilistic model of the world. 'Active inference' is an emerging normative framework in cognitive and computational neuroscience that …

This work tackles model-based RL by optimizing state-action queries to learn policies with minimal data.

New PAC-Bayesian approach stabilizes actor-critic learning.

This study uses HMM and RL to dynamically allocate equities, Treasuries, and gold based on market regimes.

New bounds on IDS for RL show how to balance computation and learning efficiency.

Novel MBRL method for large-scale RL with reduced posterior complexity.