arXiv research

Optimizing rewards under budget constraints with correlated costs and rewards.

ADAC uses analogous policies to improve RL exploration without sacrificing stability.

Reinforcement learning methods require careful design involving a reward function to obtain the desired action policy for a given task. In the absence of hand-crafted reward functions, prior work on the topic has proposed several methods for reward estimation by using expert state trajectories and action pairs. However…

This work compares human feedback methods for reward learning in bandits.

Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of…

DDO-RM improves reward-based policies by converting reward scores into a target distribution.

MAXMINLCB optimizes unknown target functions with preference feedback using a Stackelberg game approach.

Enhances reward specification in RL with a novel language-based approach.

Paper tackles offline preference-based RL with human feedback.

New concept: reward hacking, where optimizing a flawed reward function can hurt performance.

Method tackles uncertainty in reward models for LLMs from heterogeneous human feedback.

Current reinforcement learning methods fail if the reward function is imperfect, i.e. if the agent observes reward different from what it actually receives. We study this problem within the formalism of Corrupt Reward Markov Decision Processes (CRMDPs). We show that if the reward corruption in a CRMDP is sufficiently "…

New framework for efficient query-based imitation learning.

Reinforcement learning with trajectory feedback instead of state-action rewards.

A new method recovers rewards from behavior policies using classification and regression.

The paper explores a Multi-Objective RL approach for trading that generalizes reward functions.

GRPO optimizes LLMs with verifiable rewards, amplifying policy success.

This work frames reward modelling from preferences as a causal problem.

Proposes a new theoretical framework for PbRL that requires less human feedback.

Reward suffices for convex MDPs, expanding RL to new problems.

Task-specific scores are often used to optimize for and evaluate the performance of conditional text generation systems. However, such scores are non-differentiable and cannot be used in the standard supervised learning paradigm. Hence, policy gradient methods are used since the gradient can be computed without requiri…

Develops statistical framework for resolving reward function ambiguity in inverse reinforcement learning.

A new algorithm for offline RL with trajectory-wise reward reduces bias and variance errors.

PROBE optimizes best-arm identification with cheap proxies, improving sample complexity.

Capsule Networks improve autonomous navigation in sparse environments.

Backtesting framework for CLMMs on Uniswap V3 reduces reward estimation error.

RL enhances cryptocurrency trading profits.

Study optimal offline RL with uncertainty sets and distribution shifts.

Exploration efficiency is a challenging problem in multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the collaborative approach among multiple agents. Another important problem is the less informative reward restricts the learning speed of MARL compared with the informative…

In this paper we model the problem of learning preferences of a population as an active learning problem. We propose an algorithm can adaptively choose pairs of items to show to users coming from a heterogeneous population, and use the obtained reward to decide which pair of items to show next. We provide computational…

MADE improves exploration in RL by maximizing deviation from explored regions.

In many environments only a tiny subset of all states yield high reward. In these cases, few of the interactions with the environment provide a relevant learning signal. Hence, we may want to preferentially train on those high-reward states and the probable trajectories leading to them. To this end, we advocate for the…

Best-of-N sampling reveals reward targets from preference data, influencing N and base distribution choices.

The pair-matching problem appears in many applications where one wants to discover good matches between pairs of entities or individuals. Formally, the set of individuals is represented by the nodes of a graph where the edges, unobserved at first, represent the good matches. The algorithm queries pairs of nodes and obs…

Improved DPO framework penalizes preference uncertainty to avoid overoptimization.

This work optimizes identifying good arms in nonparametric multi-armed bandits.

A dueling bandit problem with resource constraints is solved using EXP3.

CREDIT learns to master pair trading with risk-aware RL, outperforming existing methods.

A fair policy for hiring candidates from different groups is proposed in a linear contextual bandit problem.

We tackle the Multi-task Batch Reinforcement Learning problem. Given multiple datasets collected from different tasks, we train a multi-task policy to perform well in unseen tasks sampled from the same distribution. The task identities of the unseen tasks are not provided. To perform well, the policy must infer the tas…

Rank aggregation based on pairwise comparisons over a set of items has a wide range of applications. Although considerable research has been devoted to the development of rank aggregation algorithms, one basic question is how to efficiently collect a large amount of high-quality pairwise comparisons for the ranking pur…

In this paper we propose the multi-objective contextual bandit problem with similarity information. This problem extends the classical contextual bandit problem with similarity information by introducing multiple and possibly conflicting objectives. Since the best arm in each objective can be different given the contex…

New RL method improves on standard discounted RL for operations research.

The focal point of this paper is the issue of "drawdown" which arises in recursive betting scenarios and related applications in the stock market. Roughly speaking, drawdown is understood to mean drops in wealth over time from peaks to subsequent lows. Motivated by the fact that this issue is of paramount concern to co…

A novel approach uses an ensemble of Gaussian processes for robust and adaptive reinforcement learning.

New method decomposes Markov chain rewards into persistent and transient components.

A novel bandit problem with context-dependent rewards and blocking.

New algorithms reduce contextual bandits' regret without knowing reward noise variances.