arXiv research

Study shows mutual information can reward structure learning agents without expert systems.

New IRL algorithm identifies optimal reward and policy from expert demonstrations.

Study reward-free RL in non-linear settings, improving efficiency and removing assumptions.

PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.

Flexible algorithms for maximizing rewards in structured bandits.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

A model learns rewards from diverse demonstrations for structurally similar tasks.

Proposes a method to boost deep reinforcement learning with sparse rewards.

We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round. The reward of the complex action is some function of the basic arms' rewards, and the feedback observed may not necessarily be the rewa…

Paper simplifies complex AI exploration by predicting future rewards.

In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching th…

A new IRL model recovers reward and state structure from expert demonstrations.

Enhances reward specification in RL with a novel language-based approach.

Adaptive policies solve a linear program to maximize rewards while minimizing costs in sales with discounts.

New bandit algorithms improve sparse reward learning.

Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.

We consider an adversarial online learning setting where a decision maker can choose an action in every stage of the game. In addition to observing the reward of the chosen action, the decision maker gets side observations on the reward he would have obtained had he chosen some of the other actions. The observation str…

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus…

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

Study adapts combinatorial semi-bandit for piecewise stationary, causally related rewards.

New complexity measure MEHC refines MDP upper bounds and rewards informativeness.

A new algorithm tackles delayed combinatorial semi-bandit with causal relations.

New algorithm reduces regret in multi-armed bandit problems with Gaussian rewards.

Trade-R1 bridges verifiable rewards to stochastic financial markets via process-level reasoning verification.

We propose a bandit algorithm that explores by randomizing its history of rewards. Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards. We design the pseudo rewards such that the bootstrap mean is optimistic with a sufficiently high probabi…

New algorithm uses GNNs to optimize rewards in graph-structured data.

Study non-linear combinatorial bandits with polynomial rewards, finding significant differences from linear cases.

Extends linear MDP to handle nonlinear rewards.

The paper tackles reward-relevance in offline RL with sparse decision dynamics.

New algorithm DUSA minimizes regret in structured bandits with structural information.

Two algorithms tackle heavy-tailed rewards in reinforcement learning with linear function approximation.

SA-GFN corrects biases in GFlowNets due to graph symmetries.

Generative Flow Networks use submodular upper bounds to generate more data.

Improves generative models by optimizing rewards and sample editing.

Algorithm estimates human decision-making in high-dimensional states with finite-time guarantees.

We consider a problem of learning the reward and policy from expert examples under unknown dynamics. Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies. Empowe…

Algorithm learns similarities to optimize bandit decisions in unknown metric space.

Improved exploration in factored average-reward MDPs reduces regret.

DeepSynth synthesizes automata to guide deep RL agents through sparse, non-Markovian rewards.

Study shows fast rates for inverse reinforcement learning with linear rewards.

New algorithm uses Whittle index to improve Q-learning for restless bandits.

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

A new estimator for state values in reinforcement learning reduces complexity and improves convergence.

New algorithm learns expert reward structures from batch data.

Reinforcement learning is widely used for dialogue policy optimization where the reward function often consists of more than one component, e.g., the dialogue success and the dialogue length. In this work, we propose a structured method for finding a good balance between these components by searching for the optimal re…

Novel algorithm reduces computational burden in IRL with finite-time guarantees.

We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter $θ^*$. Since we do not place any restrictions of these functions, the problem setting subsumes several previously studied frameworks that assume linear or invertible reward funct…

Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outpu…