arXiv research

Study non-linear combinatorial bandits with polynomial rewards, finding significant differences from linear cases.

Thompson Sampling shows polynomial regret for combinatorial semi-bandits with subgaussian rewards.

Polynomial-time method solves complex combinatorial semi-bandits.

Optimizes algorithms for non-concave bandit problems.

Reward-poisoning attacks can force RL agents to learn bad policies, and we categorize and quantify their feasibility.

New algorithm for reward-free RL with linear function approximation, reducing sample complexity.

New algorithms for efficient learning with long-term rewards in contextual bandits.

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

Paper tackles efficient IRL in offline settings with polynomial samples and runtime.

We consider Markov Decision Processes (MDPs) where the rewards are unknown and may change in an adversarial manner. We provide an algorithm that achieves state-of-the-art regret bound of $O( \sqrt{τ(\ln|S|+\ln|A|)T}\ln(T))$, where $S$ is the state space, $A$ is the action space, $τ$ is the mixing time of the MDP, and $…

The paper develops a robust algorithm for contextual bandits with heavy-tailed rewards.

Improves RLHF sample efficiency by scaling reward complexity polynomially.

This work shows how approximate reward models can significantly improve inference-time scaling.

Polynomial chaos surrogates quantify epistemic uncertainty in AI-driven scientific models.

New algorithm finds optimal policies without knowing reward functions.

ETGL-DDPG improves DDPG for sparse reward control with new exploration and replay techniques.

New algorithm reduces reinforcement learning regret to sqrt(T) without strong dynamics assumptions.

The paper enhances preference learning by incorporating response time data.

Paper develops methods to optimize policies directly from human feedback without reward inference.

New algorithm for RL with horizon-free reward-free exploration for linear MDPs.

The explore{exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approxi…

Paper resolves bias in ALFT training using generalized alignment games.

This paper investigates teacher hacking during language model distillation and proposes methods to mitigate it.

We study the problem of stochastic combinatorial pure exploration (CPE), where an agent sequentially pulls a set of single arms (a.k.a. a super arm) and tries to find the best super arm. Among a variety of problem settings of the CPE, we focus on the full-bandit setting, where we cannot observe the reward of each singl…

Algorithm learns near-optimal policies for reward-mixing MDPs with few latent contexts.

We consider a novel stochastic multi-armed bandit setting, where playing an arm makes it unavailable for a fixed number of time slots thereafter. This models situations where reusing an arm too often is undesirable (e.g. making the same product recommendation repeatedly) or infeasible (e.g. compute job scheduling on ma…

The paper tackles non-stationary MAB with periodic rewards.

New method predicts and optimizes test-time scaling for LLMs.

Paper tackles offline preference-based RL with human feedback.

New method for efficient online exploration in RLHF reduces regret.

New algorithm reduces combinatorial semi-bandit regret efficiently.

Trajectory-level supervision allows efficient offline reinforcement learning.

We tackle the problem of acting in an unknown finite and discrete Markov Decision Process (MDP) for which the expected shortest path from any state to any other state is bounded by a finite number $D$. An MDP consists of $S$ states and $A$ possible actions per state. Upon choosing an action $a_t$ at state $s_t$, one re…

New algorithm reduces sleeping bandits' regret to O(sqrt(T)).

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

Algorithm learns diffusion processes with high-dimensional state spaces.

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…

In many settings (e.g., robotics) demonstrations provide a natural way to specify tasks; however, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the tasks, such as rewards or policies, can be safely composed and/or do not explicitly capture history dependen…

We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…

We consider a stochastic continuum armed bandit problem where the arms are indexed by the $\ell_2$ ball $B_{d}(1+ν)$ of radius $1+ν$ in $\mathbb{R}^d$. The reward functions $r :B_{d}(1+ν) \rightarrow \mathbb{R}$ are considered to intrinsically depend on $k \ll d$ unknown linear parameters so that $r(\mathbf{x}) = g(\ma…

New algorithm identifies optimal actions in large reward spaces efficiently.

We describe a novel algorithm for noisy global optimisation and continuum-armed bandits, with good convergence properties over any continuous reward function having finitely many polynomial maxima. Over such functions, our algorithm achieves square-root regret in bandits, and inverse-square-root error in optimisation, …

We introduce a new class of reinforcement learning methods referred to as {\em episodic multi-armed bandits} (eMAB). In eMAB the learner proceeds in {\em episodes}, each composed of several {\em steps}, in which it chooses an action and observes a feedback signal. Moreover, in each step, it can take a special action, c…

Adaptive algorithm identifies best arm with abstention, showing phase transition from polynomial to exponential error probability.

New algorithm learns causal graph to minimize regret in bandits without full structure.

Contextual multi-armed bandit algorithms are widely used in sequential decision tasks such as news article recommendation systems, web page ad placement algorithms, and mobile health. Most of the existing algorithms have regret proportional to a polynomial function of the context dimension, $d$. In many applications ho…

We characterize value functions in partially observable MDPs as semi-algebraic sets.

Efficient algorithm for online learning with Massart noise achieves near-optimal mistake bound.