arXiv research

Step-DAD improves BED by periodically updating a design policy during experiments.

Overparameterized models generalize well in offline contextual bandits, but policy-based algorithms struggle.

This work improves policy-based training by proposing an evaluation balance objective for GFlowNets.

New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.

New GFlowNet training framework using policy gradients for combinatorial object generation.

New method defends RL agents from poisoning attacks without MDP knowledge.

This work explores representation complexity in RL paradigms, revealing model-based RL as the easiest task.

New findings show good representations alone are insufficient for efficient reinforcement learning.

MAVEN improves multi-agent exploration by hybridizing value and policy-based methods.

Develops ODRPO to improve RL algorithms with better performance and stability.

The goal of reinforcement learning (RL) is to let an agent learn an optimal control policy in an unknown environment so that future expected rewards are maximized. The model-free RL approach directly learns the policy based on data samples. Although using many samples tends to improve the accuracy of policy learning, c…

New algorithm reduces RL policy optimization gap.

The study optimizes free trial lengths to boost subscriptions and consumer loyalty.

Survey of RL in finance, tackling complex decision-making.

New RL algorithm tackles non-stationary environments with flexible policy updates.

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for most applications. In this work, we approach an alternative Interactive Machine Learning (IML) stra…

We establish a new connection between value and policy based reinforcement learning (RL) based on a relationship between softmax temporal value consistency and policy optimality under entropy regularization. Specifically, we show that softmax consistent action values correspond to optimal entropy regularized policy pro…

A new imitation learning method uses random search for simple policies, outperforming complex models.

Study on adversarial training's impact on deep neural reinforcement learning policies.

Optimizing option exercise policies based on variance optimal martingale measure can lead to unappealing results.

We solve POMDPs by approximating them as finite-state MDPs.

This work highlights problems with off-policy estimation in recommender systems due to unobserved confounders.

Policy analysts wish to visualize a range of policies for large simulator-defined Markov Decision Processes (MDPs). One visualization approach is to invoke the simulator to generate on-policy trajectories and then visualize those trajectories. When the simulator is expensive, this is not practical, and some method is r…

Learning goal-oriented dialogues by means of deep reinforcement learning has recently become a popular research topic. However, commonly used policy-based dialogue agents often end up focusing on simple utterances and suboptimal policies. To mitigate this problem, we propose a class of novel temperature-based extension…

Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.

New algorithm finds near-optimal policies efficiently in zero-sum games.

The paper reinterprets policy gradient methods using operator theory.

In artificial intelligence, we often specify tasks through a reward function. While this works well in some settings, many tasks are hard to specify this way. In deep reinforcement learning, for example, directly specifying a reward as a function of a high-dimensional observation is challenging. Instead, we present an …

Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate. Though often achieving encouraging empirical success, its underlying mathematical principle on {\em policy-distri…

Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse. In this paper, we consider a challenging setting where an agent and an expert use different actions from each other. We assume that the agent has access to a sparse reward function and state-only expert observa…

In this paper, we study a multi-step interactive recommendation problem, where the item recommended at current step may affect the quality of future recommendations. To address the problem, we develop a novel and effective approach, named CFRL, which seamlessly integrates the ideas of both collaborative filtering (CF) …

Data augmentation is commonly used to encode invariances in learning methods. However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set. The resulting explosion of the dataset size can be an issue in te…

Active learning method improves local model validity estimation.

Off-policy learning in dynamic decision problems is essential for providing strong evidence that a new policy is better than the one in use. But how can we prove superiority without testing the new policy? To answer this question, we introduce the G-SCOPE algorithm that evaluates a new policy based on data generated by…

Tackling large approximate dynamic programming or reinforcement learning problems requires methods that can exploit regularities, or intrinsic structure, of the problem in hand. Most current methods are geared towards exploiting the regularities of either the value function or the policy. We introduce a general classif…

Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algor…

A new multi-agent learning method improves performance in complex games.

We study the problem of off-policy value evaluation in reinforcement learning (RL), where one aims to estimate the value of a new policy based on data collected by a different policy. This problem is often a critical step when applying RL in real-world problems. Despite its importance, existing general methods either h…

As the Portable Document Format (PDF) file format increases in popularity, research in analysing its structure for text extraction and analysis is necessary. Detecting headings can be a crucial component of classifying and extracting meaningful data. This research involves training a supervised learning model to detect…

New Bellman error estimator improves offline model selection performance.

iDAD uses neural networks to quickly adapt experiments without likelihoods.

Paper addresses OPE for dependent bandit samples using MDS and batch updates.

In decision making problems for continuous state and action spaces, linear dynamical models are widely employed. Specifically, policies for stochastic linear systems subject to quadratic cost functions capture a large number of applications in reinforcement learning. Selected randomized policies have been studied in th…

New method optimizes policies in non-stationary environments.

Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to p…

Combines multiple OPE estimators into a more accurate and efficient estimate.

ENIAC method optimizes and explores complex RL problems with non-linear policies.

New method combines value function decomposition and policy gradients for cooperative multi-agent reinforcement learning.