arXiv research

New reward function improves GAIL performance in task-based environments.

Study improves unbiased recommender learning by addressing missing-reward bias.

Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…

There exist a number of reinforcement learning algorithms which learnby climbing the gradient of expected reward. Their long-runconvergence has been proved, even in partially observableenvironments with non-deterministic actions, and without the need fora system model. However, the variance of the gradient estimator ha…

Q-Learning overestimation bias influenced by learning rate, discount factor, and reward signal.

New algorithm achieves optimal regret in average reward MDPs without prior bias information.

Revisits VIC method to correct intrinsic reward bias in stochastic environments.

A RL framework selects features to balance bias and accuracy dynamically.

CausalRM models rewards from user feedback, overcoming noise and bias.

This study mainly investigates two common decoding problems in neural keyphrase generation: sequence length bias and beam diversity. To tackle the problems, we introduce a beam search decoding strategy based on word-level and ngram-level reward function to constrain and refine Seq2Seq inference at test time. Results sh…

This work compares human feedback methods for reward learning in bandits.

SAIL improves AIL by weighting adversarial rewards with support estimation.

New RLHF approach mitigates bias in aligning LLMs with human preferences.

Deep Reinforcement Learning has been shown to be very successful in complex games, e.g. Atari or Go. These games have clearly defined rules, and hence allow simulation. In many practical applications, however, interactions with the environment are costly and a good simulator of the environment is not available. Further…

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework. The first problem is implicit bias present in the reward functions used in these algorithms. While these biases might work well for some environments, they can also lead to sub-optimal behavior in others. Secondl…

New algorithm mitigates affinity bias in hiring feedback loops.

A new algorithm for offline RL with trajectory-wise reward reduces bias and variance errors.

Paper tackles transfer RL under unobserved context, developing methods to reduce bias.

The paper tackles bandit problems with biased offline data by using causal methods.

New method decomposes Markov chain rewards into persistent and transient components.

SA-GFN corrects biases in GFlowNets due to graph symmetries.

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, …

This work overcomes bias in concave multi-objective reinforcement learning.

Paper resolves bias in ALFT training using generalized alignment games.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

New method learns diverse solutions in reinforcement learning without gradient bias.

New method for evaluating sequential recommendations with lower variance.

New method separates model and non-model risks for more practical asset pricing.

New method reduces sample complexity for robust reinforcement learning.

Study tackles infinitely many-armed bandits with rotting rewards, achieving tight regret bounds.

Study intrinsic motivation for synergistic tasks in reinforcement learning.

Off-policy evaluation for MNAR rewards in MDPs

PRISM integrates diverse rewards in MORL, improving sample efficiency and Pareto coverage.

New insights on bias in multi-armed bandits under conditional sampling.

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

Our goal is for agents to optimize the right reward function, despite how difficult it is for us to specify what that is. Inverse Reinforcement Learning (IRL) enables us to infer reward functions from demonstrations, but it usually assumes that the expert is noisily optimal. Real people, on the other hand, often have s…

A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.

Mitigates biases in reward models using variational inference.

A new Q-learning variant reduces underestimation bias in deep reinforcement learning.

A new algorithm CAP learns optimal policies from observational data with confounding bias and missing observations.

New method optimizes policies without assuming known link functions between preferences and rewards.

New estimator improves off-policy evaluation for large action spaces.

A new estimator reduces variance in slate bandit OPE.

Ranking metrics are a family of metrics largely used to evaluate recommender systems. However they typically suffer from the fact the reward is affected by the order in which recommended items are displayed to the user. A classical way to overcome this position bias is to uniformly shuffle a proportion of the recommend…

COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.

This paper tackles distribution shift in model-based offline RL, proposing a shifts-aware reward method.

Paper proposes a method to learn and exceed expert demonstrations in unknown reward environments.