arXiv research

Extends linear MDP to handle nonlinear rewards.

This paper studies GAIL's global convergence for general MDP and nonlinear rewards.

LNUCB-TA improves MAB performance by dynamically adjusting exploration rates and recognizing spatiotemporal patterns.

We consider the combinatorial multi-armed bandit (CMAB) problem, where the reward function is nonlinear. In this setting, the agent chooses a batch of arms on each round and receives feedback from each arm of the batch. The reward that the agent aims to maximize is a function of the selected arms and their expectations…

We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …

We consider the best-arm identification problem in multi-armed bandits, which focuses purely on exploration. A player is given a fixed budget to explore a finite set of arms, and the rewards of each arm are drawn independently from a fixed, unknown distribution. The player aims to identify the arm with the largest expe…

We address the M-best-arm identification problem in multi-armed bandits. A player has a limited budget to explore K arms (M<K), and once pulled, each arm yields a reward drawn (independently) from a fixed, unknown distribution. The goal is to find the top M arms in the sense of expected reward. We develop an algorithm …

This work explains RL policies using causal models, revealing important patterns and failures.

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

NK bandits improve performance on nonlinear tasks.

The question of how to explore, i.e., take actions with uncertain outcomes to learn about possible future rewards, is a key question in reinforcement learning (RL). Here, we show a surprising result: We show that Q-learning with nonlinear Q-function and no explicit exploration (i.e., a purely greedy policy) can learn s…

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…

We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round. The reward of the complex action is some function of the basic arms' rewards, and the feedback observed may not necessarily be the rewa…

RoME optimizes mobile health interventions by modeling user and time-specific effects.

New method modifies diffusions for singular rewards.

New method learns multiple reward functions for complex tasks.

VO$Q$L optimizes RL with sparse rewards using weighted bounds.

Our understanding of reinforcement learning (RL) has been shaped by theoretical and empirical results that were obtained decades ago using tabular representations and linear function approximators. These results suggest that RL methods that use temporal differencing (TD) are superior to direct Monte Carlo estimation (M…

Paper closes the gap in MP-MAB problems with novel adaptive communication and exploration.

Unified approach to time-inconsistent problems with distribution-dependent rewards.

Inverse reinforcement learning (IRL) is the problem of inferring the reward function of an agent, given its policy or observed behavior. Analogous to RL, IRL is perceived both as a problem and as a class of methods. By categorically surveying the current literature in IRL, this article serves as a reference for researc…

Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.

AIF reformulated as convex MDP for adaptive behavior.

New method learns optimal policies in presence of unmeasured confounders.

PFN-TS uses Thompson sampling with PFNs to improve contextual bandit performance.

Trajectory-level supervision allows efficient offline reinforcement learning.

New method identifies flawed internal models of the world in animals.

MAXMINLCB optimizes unknown target functions with preference feedback using a Stackelberg game approach.

DSAC improves cooperative MARL with general utilities, converging faster than existing methods.

This work overcomes bias in concave multi-objective reinforcement learning.

Kernel method estimates long-term effects from short-term data.

New MAB model incentivizes user arm-pulling with self-reinforcing preferences.

Nonlinear optimal control problems are often solved with numerical methods that require knowledge of system's dynamics which may be difficult to infer, and that carry a large computational cost associated with iterative calculations. We present a novel neurobiologically inspired hierarchical learning framework, Reinfor…

Study pure exploration in high-dimensional feature spaces using adaptive embeddings.

We characterize value functions in partially observable MDPs as semi-algebraic sets.

This thesis proposes a derivatives hedging framework using deep learning and reinforcement learning.

Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.

Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.

We discuss the approximation of the value function for infinite-horizon discounted Markov Reward Processes (MRP) with nonlinear functions trained with the Temporal-Difference (TD) learning algorithm. We first consider this problem under a certain scaling of the approximating function, leading to a regime called lazy tr…

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

This work analyzes the value of future reward information in RL.

New algorithm for nonstationary GLBs reduces computation and memory costs.

Self-supervised reward prediction improves RL in sparse reward settings.

The study categorizes reward errors in reinforcement learning, finding some can be beneficial.

Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.

Reward models need more than just accuracy for effective RLHF.

Action guidance helps agents learn true objectives in games with sparse rewards.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.