arXiv research

Study non-linear combinatorial bandits with polynomial rewards, finding significant differences from linear cases.

Study reward-free RL in non-linear settings, improving efficiency and removing assumptions.

DART optimizes subset selection in non-linear bandit problems.

We develop a theory for solving continuous time optimal stopping problems for non-linear expectations. Our motivation is to consider problems in which the stopper uses risk measures to evaluate future rewards.

Multi-agent learning is a promising method to simulate aggregate competitive behaviour in finance. Learning expert agents' reward functions through their external demonstrations is hence particularly relevant for subsequent design of realistic agent-based simulations. Inverse Reinforcement Learning (IRL) aims at acquir…

This paper studies semiparametric contextual bandits, a generalization of the linear stochastic bandit problem where the reward for an action is modeled as a linear function of known action features confounded by an non-linear action-independent term. We design new algorithms that achieve $\tilde{O}(d\sqrt{T})$ regret …

New framework for analyzing games with multi-dimensional singular controls and non-linear jumps.

Finding optimal policies which maximize long term rewards of Markov Decision Processes requires the use of dynamic programming and backward induction to solve the Bellman optimality equation. However, many real-world problems require optimization of an objective that is non-linear in cumulative rewards for which dynami…

Unified framework for multi-user bandits using Laplacian kernels.

Bayesian optimization improved for biased data.

INF-clip optimizes heavy-tailed MAB problems with improved performance.

Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus…

Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…

Improved regret bound for multinomial logistic bandits with non-linearity.

We study the Combinatorial Pure Exploration problem with Continuous and Separable reward functions (CPE-CS) in the stochastic multi-armed bandit setting. In a CPE-CS instance, we are given several stochastic arms with unknown distributions, as well as a collection of possible decisions. Each decision has a reward accor…

New algorithms improve contextual bandits with neural networks and energy models.

New algorithm achieves nearly optimal regret with one-pass updates for GLB problems.

NeuralRBMLE tackles explore-exploit trade-offs in contextual bandits with neural networks.

Paper introduces G-LowTESTR for efficient tensor bandits.

Neural algorithms optimize arm selection with human preference feedback for complex reward functions.

VO$Q$L optimizes RL with sparse rewards using weighted bounds.

Thompson Sampling fails to perform well in high dimensions.

Improved algorithm for logistic bandits with better regret bounds.

Online learning has traditionally focused on the expected rewards. In this paper, a risk-averse online learning problem under the performance measure of the mean-variance of the rewards is studied. Both the bandit and full information settings are considered. The performance of several existing policies is analyzed, an…

We reveal an interesting convex duality relationship between two problems: (a) minimizing the probability of lifetime ruin when the rate of consumption is stochastic and when the individual can invest in a Black-Scholes financial market; (b) a controller-and-stopper problem, in which the controller controls the drift a…

MADE improves exploration in RL by maximizing deviation from explored regions.

Study contextual bandits with stage-wise constraints, proving regret bounds and extending results.

We study the regret minimization problem in the novel setting of generalized kernelized bandits (GKBs), where we optimize an unknown function $f^*$ belonging to a reproducing kernel Hilbert space (RKHS) having access to samples generated by an exponential family (EF) reward model whose mean is a non-linear function $μ(…

New algorithm reduces regret for logistic bandits without $κ$ dependency.

Oracle-efficient algorithms reduce combinatorial semi-bandit regret to logarithmic time.

We develop an option pricing model based on a tug-of-war game. This two-player zero-sum stochastic differential game is formulated in the context of a multi-dimensional financial market. The issuer and the holder try to manipulate asset price processes in order to minimize and maximize the expected discounted reward. W…

A new estimator reduces variance in slate bandit OPE.

Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.

We consider the optimal stopping problem with non-linear $f$-expectation (induced by a BSDE) without making any regularity assumptions on the reward process $ξ$. and with general filtration. We show that the value family can be aggregated by an optional process $Y$. We characterize the process $Y$ as the $\mathcal{E}^f…

Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding of such methods has been rather limited. This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algo…

We propose combinatorial cascading bandits, a class of partial monitoring problems where at each step a learning agent chooses a tuple of ground items subject to constraints and receives a reward if and only if the weights of all chosen items are one. The weights of the items are binary, stochastic, and drawn independe…

Improved control approach for correlated bandits with better performance.

We study two randomized algorithms for generalized linear bandits. The first, GLM-TSL, samples a generalized linear model (GLM) from the Laplace approximation to the posterior distribution. The second, GLM-FPL, fits a GLM to a randomly perturbed history of past rewards. We analyze both algorithms and derive $\tilde{O}(…

New algorithm tackles non-stationary combinatorial semi-bandit problems with optimal regret bounds.

A new algorithm for conversational recommendation systems using dueling bandits in GLMs.

Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

This work analyzes the value of future reward information in RL.

Self-supervised reward prediction improves RL in sparse reward settings.

The study categorizes reward errors in reinforcement learning, finding some can be beneficial.

Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.

Reward models need more than just accuracy for effective RLHF.

Action guidance helps agents learn true objectives in games with sparse rewards.