arXiv research

Trade-R1 bridges verifiable rewards to stochastic financial markets via process-level reasoning verification.

Extends linear MDP to handle nonlinear rewards.

This work shows how approximate reward models can significantly improve inference-time scaling.

Modeling decision-making dynamics in MDD patients using RL-HMM.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

Novel framework for risk-sensitive reinforcement learning using martingale decomposition.

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-proce…

The paper tackles reinforcement learning with exogenous variables and rewards.

The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.

Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.

Unified framework for human-like decision making in various sequential tasks.

New findings on universal learning in contextual bandits with adversarial rewards.

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement learning problem, which extends the standard Q-learning approach to incorporate a two-stream framework of reward processing with biases biologically associated with several neurolo…

Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-reward Markov Decision Processes (MDPs). The first algorithm reduces the problem to the discounted-r…

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus…

Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and identify conditions under which an MDP with exogenous state can be decomposed into an exogenous Markov Reward Process involving on…

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

ARL uses queries to learn rewards, focusing on cost vs. reward value.

A novel framework refines diffusion models iteratively for better downstream reward optimization.

New acquisition function for extreme rewards in bandits.

TLRS improves predictive power of mined formulaic alpha factors.

The paper studies reward concentration in MDPs, covering asymptotic and non-asymptotic settings.

Unified framework improves diffusion model rewards without full trajectories.

This work extends reinforcement learning to handle non-cumulative objectives.

We propose RUDDER, a novel reinforcement learning approach for delayed rewards in finite Markov decision processes (MDPs). In MDPs the Q-values are equal to the expected immediate reward plus the expected future rewards. The latter are related to bias problems in temporal difference (TD) learning and to high variance p…

At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of states with samples from a Markov reward process induced by following a Markov policy in a Markov decision process. We propose a simple and…

Adjoint Matching improves flow and diffusion models with reward fine-tuning.

A new algorithm uses concavity in Gaussian processes to optimize decisions in bandit problems.

Improves generative models by optimizing rewards and sample editing.

Multi-agent learning is a promising method to simulate aggregate competitive behaviour in finance. Learning expert agents' reward functions through their external demonstrations is hence particularly relevant for subsequent design of realistic agent-based simulations. Inverse Reinforcement Learning (IRL) aims at acquir…

Mitigates biases in reward models using variational inference.

Estimates rewards from a demonstrator's learning process.

We propose a new complexity measure for Markov decision processes (MDPs), the maximum expected hitting cost (MEHC). This measure tightens the closely related notion of diameter [JOA10] by accounting for the reward structure. We show that this parameter replaces diameter in the upper bound on the optimal value span of a…

A new algorithm balances global reward and group constraints in federated multi-armed bandits.

Method predicts future rewards from past actions in a linear Gaussian system.

We consider reinforcement learning (RL) in Markov Decision Processes in which an agent repeatedly interacts with an environment that is modeled by a controlled Markov process. At each time step $t$, it earns a reward, and also incurs a cost-vector consisting of $M$ costs. We design model-based RL algorithms that maximi…

New algorithm learns policies without explicit rewards for MDPs.

Off-policy evaluation for MNAR rewards in MDPs

Many cooperative multiagent reinforcement learning environments provide agents with a sparse team-based reward, as well as a dense agent-specific reward that incentivizes learning basic skills. Training policies solely on the team-based reward is often difficult due to its sparsity. Furthermore, relying solely on the a…

New algorithm for recommending best arms with aggregated feedback.

New algorithm learns optimal policy for average reward MDPs with sample complexity matching lower bound.

This paper formalizes reward training in RLHF and provides theoretical guarantees.

New algorithm for reward-free RL with linear function approximation, reducing sample complexity.

Trajectory-level supervision allows efficient offline reinforcement learning.

Develops first-order methods for average-reward MDPs with strong guarantees.

A stochastic model helps maintain insufficiently funded pension funds.

This work improves Q-learning for average-reward MDPs, reducing sample and communication complexities in federated settings.

Algorithm learns diffusion processes with high-dimensional state spaces.