arXiv research

This work improves policy-based training by proposing an evaluation balance objective for GFlowNets.

New GFlowNet training framework using policy gradients for combinatorial object generation.

Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.

Step-DAD improves BED by periodically updating a design policy during experiments.

The goal of reinforcement learning (RL) is to let an agent learn an optimal control policy in an unknown environment so that future expected rewards are maximized. The model-free RL approach directly learns the policy based on data samples. Although using many samples tends to improve the accuracy of policy learning, c…

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

This work highlights problems with off-policy estimation in recommender systems due to unobserved confounders.

The study optimizes free trial lengths to boost subscriptions and consumer loyalty.

Modern deep learning methods provide effective means to learn good representations. However, is a good representation itself sufficient for sample efficient reinforcement learning? This question has largely been studied only with respect to (worst-case) approximation error, in the more classical approximate dynamic pro…

New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.

New method defends RL agents from poisoning attacks without MDP knowledge.

We study the problem of off-policy value evaluation in reinforcement learning (RL), where one aims to estimate the value of a new policy based on data collected by a different policy. This problem is often a critical step when applying RL in real-world problems. Despite its importance, existing general methods either h…

Combines multiple OPE estimators into a more accurate and efficient estimate.

New RL algorithm tackles non-stationary environments with flexible policy updates.

Develops ODRPO to improve RL algorithms with better performance and stability.

New Bellman error estimator improves offline model selection performance.

We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…

Active learning method improves local model validity estimation.

Tackling large approximate dynamic programming or reinforcement learning problems requires methods that can exploit regularities, or intrinsic structure, of the problem in hand. Most current methods are geared towards exploiting the regularities of either the value function or the policy. We introduce a general classif…

VA-OPE improves OPE by incorporating variance information, achieving tighter error bounds.

Study improves off-policy evaluation from non-i.i.d. bandit samples.

New algorithm reduces RL policy optimization gap.

Paper addresses OPE for dependent bandit samples using MDS and batch updates.

We solve POMDPs by approximating them as finite-state MDPs.

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) stra…

Survey of RL in finance, tackling complex decision-making.

We study the problem of offline policy optimization in stochastic contextual bandit problems, where the goal is to learn a near-optimal policy based on a dataset of decision data collected by a suboptimal behavior policy. Rather than making any structural assumptions on the reward function, we assume access to a given …

The paper analyzes RLHF with human feedback and provides convergence results for MLE and pessimistic MLE.

Many complex domains, such as robotics control and real-time strategy (RTS) games, require an agent to learn a continuous control. In the former, an agent learns a policy over $\mathbb{R}^d$ and in the latter, over a discrete set of actions each of which is parametrized by a continuous parameter. Such problems are natu…

Study on adversarial training's impact on deep neural reinforcement learning policies.

Off-policy deep reinforcement learning (RL) algorithms are incapable of learning solely from batch offline data without online interactions with the environment, due to the phenomenon known as \textit{extrapolation error}. This is often due to past data available in the replay buffer that may be quite different from th…

NK bandits improve performance on nonlinear tasks.

Reinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision making problems. The goodness of a policy is measured by its value function starting from some initial state. The focus of this paper is to construct confidence intervals…

RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.

The paper analyzes optimal liquidation strategies for cryptocurrencies considering both temporary and permanent price impacts.

The problem of retrosynthetic planning can be framed as one player game, in which the chemist (or a computer program) works backwards from a molecular target to simpler starting materials though a series of choices regarding which reactions to perform. This game is challenging as the combinatorial space of possible cho…

A new imitation learning method uses random search for simple policies, outperforming complex models.

Off-policy learning in dynamic decision problems is essential for providing strong evidence that a new policy is better than the one in use. But how can we prove superiority without testing the new policy? To answer this question, we introduce the G-SCOPE algorithm that evaluates a new policy based on data generated by…

This work develops confidence intervals for off-policy evaluation.

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …

As the Portable Document Format (PDF) file format increases in popularity, research in analysing its structure for text extraction and analysis is necessary. Detecting headings can be a crucial component of classifying and extracting meaningful data. This research involves training a supervised learning model to detect…

In decision making problems for continuous state and action spaces, linear dynamical models are widely employed. Specifically, policies for stochastic linear systems subject to quadratic cost functions capture a large number of applications in reinforcement learning. Selected randomized policies have been studied in th…

POTEC tackles off-policy learning in large action spaces, improving effectiveness.

New algorithm finds near-optimal policies efficiently in zero-sum games.

Privileged Information Dropout improves RL performance without distillation.

Neuroscientific theory suggests that dopaminergic neurons broadcast global reward prediction errors to large areas of the brain influencing the synaptic plasticity of the neurons in those regions. We build on this theory to propose a multi-agent learning framework with spiking neurons in the generalized linear model (G…

BLOB combines organic and bandit signals for better user interest estimation.

We introduce Bayesian least-squares policy iteration (BLSPI), an off-policy, model-free, policy iteration algorithm that uses the Bayesian least-squares temporal-difference (BLSTD) learning algorithm to evaluate policies. An online variant of BLSPI has been also proposed, called randomised BLSPI (RBLSPI), that improves…