arXiv research

Enhances reward specification in RL with a novel language-based approach.

Many cooperative multiagent reinforcement learning environments provide agents with a sparse team-based reward, as well as a dense agent-specific reward that incentivizes learning basic skills. Training policies solely on the team-based reward is often difficult due to its sparsity. Furthermore, relying solely on the a…

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

A language for specifying complex reinforcement learning tasks.

New bandit algorithms improve sparse reward learning.

Optimizes reward learning design for complex tasks using nonparametric methods.

Study optimal reward schemes for inducing desired player performance in risky contests.

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

Develops HMRL for sparse reward RL problems, improving meta policy efficiency and transferability.

New RL method uses distance between states instead of rewards for sparse reward environments.

SharedRep-RLHF learns shared traits for diverse groups, improving fairness and performance.

RoME optimizes mobile health interventions by modeling user and time-specific effects.

Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.

We propose a bandit algorithm that explores by randomizing its history of rewards. Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards. We design the pseudo rewards such that the bootstrap mean is optimistic with a sufficiently high probabi…

Adaptive exploration scheme for evaluating multiple policies with different rewards.

New lower bounds for combinatorial multi-armed bandits for general reward functions.

Enhanced financial reward with shuffled feature CNN-DRL.

A novel framework refines diffusion models iteratively for better downstream reward optimization.

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, …

Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularizatio…

SARA uses similarity to learn rewards robustly and adaptively.

Unified framework improves diffusion model rewards without full trajectories.

New algorithm uses Whittle index to improve Q-learning for restless bandits.

Solving tasks with sparse rewards is one of the most important challenges in reinforcement learning. In the single-agent setting, this challenge is addressed by introducing intrinsic rewards that motivate agents to explore unseen regions of their state spaces; however, applying these techniques naively to the multi-age…

Imitation by observation is an approach for learning from expert demonstrations that lack action information, such as videos. Recent approaches to this problem can be placed into two broad categories: training dynamics models that aim to predict the actions taken between states, and learning rewards or features for com…

Study quantile reward identification with 1-bit feedback constraints.

Reinforcement learning agents are prone to undesired behaviors due to reward mis-specification. Finding a set of reward functions to properly guide agent behaviors is particularly challenging in multi-agent scenarios. Inverse reinforcement learning provides a framework to automatically acquire suitable reward functions…

Paper shows RLHF can be solved similarly to standard RL.

Adjoint Matching improves flow and diffusion models with reward fine-tuning.

A new method recovers rewards from behavior policies using classification and regression.

Study of multi-armed bandits with state-switching rewards using Markov models.

Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total reward over the inputs. Contextual bandits are commonly used to solve recommendation or ranking problems. This paper considers a learning s…

A significant challenge for the practical application of reinforcement learning in the real world is the need to specify an oracle reward function that correctly defines a task. Inverse reinforcement learning (IRL) seeks to avoid this challenge by instead inferring a reward function from expert behavior. While appealin…

New RL method explores environments without rewards, achieving efficient policy generation.

Two novel methods identify influential features in CMABs for better reward distribution.

Optimal sample complexity analysis for plug-in approach in average-reward MDPs.

The Multi-Armed Bandits (MAB) framework highlights the tension between acquiring new knowledge (Exploration) and leveraging available knowledge (Exploitation). In the classical MAB problem, a decision maker must choose an arm at each time step, upon which she receives a reward. The decision maker's objective is to maxi…

Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of…

Study on deep neural networks for reward modeling with pairwise comparison data.

COCOA improves credit assignment in reinforcement learning by measuring contributions to rewards.

Improves text-to-image diffusion models using GFlowNets.

Proposes a new theoretical framework for PbRL that requires less human feedback.

Investigates risk measures for DC pension decumulation.

New RL algorithms use lookahead info to maximize rewards.

CARDS improves decoding efficiency and alignment quality for LLMs.

In many settings (e.g., robotics) demonstrations provide a natural way to specify tasks; however, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the tasks, such as rewards or policies, can be safely composed and/or do not explicitly capture history dependen…

This work frames reward modelling from preferences as a causal problem.

Complex networks are often either too large for full exploration, partially accessible, or partially observed. Downstream learning tasks on these incomplete networks can produce low quality results. In addition, reducing the incompleteness of the network can be costly and nontrivial. As a result, network discovery algo…