arXiv research

PQR estimates reward functions from actions and states without assuming state-only rewards.

Estimates rewards from a demonstrator's learning process.

New algorithms improve efficiency in learning from personalized rewards.

Study improves unbiased recommender learning by addressing missing-reward bias.

New algorithm estimates treatment effects for more efficient contextual bandits.

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

This paper addresses reward estimation and incentive design for agents with hidden rewards.

Paper tackles robust offline RL with heavy-tailed rewards.

New approach for reward-free exploration reduces estimation error.

A new method recovers rewards from behavior policies using classification and regression.

Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.

Extends reinforcement learning alignment to scalar rewards, improving math reasoning.

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

PROWL uses robust reward estimates to improve ITR selection.

This paper balances short-term and long-term rewards in policy learning.

The paper tackles reward-relevance in offline RL with sparse decision dynamics.

Off-policy evaluation for MNAR rewards in MDPs

Unified framework for estimating reward functions in competitive games.

Optimizing rewards under budget constraints with correlated costs and rewards.

Study of repeated games with unobserved agent rewards using MAB framework.

The paper develops a robust algorithm for contextual bandits with heavy-tailed rewards.

Unified LP framework for offline reward learning from human demonstrations and feedback.

COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.

The multi-armed bandit (MAB) problem is a classic example of the exploration-exploitation dilemma. It is concerned with maximising the total rewards for a gambler by sequentially pulling an arm from a multi-armed slot machine where each arm is associated with a reward distribution. In static MABs, the reward distributi…

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …

Agents cooperate to make decisions in multi-armed bandits over a graph.

There exist a number of reinforcement learning algorithms which learnby climbing the gradient of expected reward. Their long-runconvergence has been proved, even in partially observableenvironments with non-deterministic actions, and without the need fora system model. However, the variance of the gradient estimator ha…

New algorithms handle heavy-tailed rewards in reinforcement learning.

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …

Self-supervised reward prediction improves RL in sparse reward settings.

New algorithm for shareable arms with load-dependent rewards in stochastic bandits.

Algorithm estimates human decision-making in high-dimensional states with finite-time guarantees.

A new UCB policy improves reward-cost ratio estimation in budgeted MAB.

Novel algorithm reduces computational burden in IRL with finite-time guarantees.

New method for contextual bandit with missing rewards.

Paper develops robust estimators and strategies for stochastic MABs with heavy-tailed rewards.

A new method resolves non-identifiability in reward modeling using anchor labels.

Backtesting framework for CLMMs on Uniswap V3 reduces reward estimation error.

Reinforcement learning methods require careful design involving a reward function to obtain the desired action policy for a given task. In the absence of hand-crafted reward functions, prior work on the topic has proposed several methods for reward estimation by using expert state trajectories and action pairs. However…

Recent reinforcement learning algorithms, though achieving impressive results in various fields, suffer from brittle training effects such as regression in results and high sensitivity to initialization and parameters. We claim that some of the brittleness stems from variance differences, i.e. when different environmen…

New algorithm for RL with horizon-free reward-free exploration for linear MDPs.

Theory for RLHF generalization under reward shift and clipped KL.

Study on deep neural networks for reward modeling with pairwise comparison data.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

Scalable model for slate recommendation learns reward probabilities.

Bayesian inverse reinforcement learning (IRL) methods are ideal for safe imitation learning, as they allow a learning agent to reason about reward uncertainty and the safety of a learned policy. However, Bayesian IRL is computationally intractable for high-dimensional problems because each sample from the posterior req…

At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of states with samples from a Markov reward process induced by following a Markov policy in a Markov decision process. We propose a simple and…