arXiv research

Algorithm reduces regret in misspecified linear contextual bandits.

Logarithmic regret achieved in Q-learning with positive gap.

Unified hybrid RL algorithm improves online RL performance with offline data.

New algorithm reduces sample complexity for planning in MDPs.

Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.

We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system …

New measure quantifies function similarity for optimization.

Logarithmic regret achieved in RL with linear function approximation.

Study gap-dependent regret bounds for risk-sensitive RL.

Optimal algorithms identify non-dominated arms in multi-output linear bandit models.

AdMRL improves meta-reinforcement learning by minimizing worst-case sub-optimality gap.

We consider the dynamic assortment optimization problem under the multinomial logit model (MNL) with unknown utility parameters. The main question investigated in this paper is model mis-specification under the $\varepsilon$-contamination model, which is a fundamental model in robust statistics and machine learning. In…

This paper tackles robust policy learning under concept drifts, improving upon existing methods.

New model predicts optimal control for restless bandit problems.

In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle. We study the effectiveness of the near-optimal cost-to-go oracle on the planning horizon and demonstrate that the cost-to-go oracle shortens the learner's planning horizon as function of its accurac…

We consider the problem of \textit{best arm identification} with a \textit{fixed budget $T$}, in the $K$-armed stochastic bandit setting, with arms distribution defined on $[0,1]$. We prove that any bandit strategy, for at least one bandit problem characterized by a complexity $H$, will misidentify the best arm with pr…

In many environments, only a relatively small subset of the complete state space is necessary in order to accomplish a given task. We develop a simple technique using emergency stops (e-stops) to exploit this phenomenon. Using e-stops significantly improves sample complexity by reducing the amount of required explorati…

Previous work on recommender systems mainly focus on fitting the ratings provided by users. However, the response patterns, i.e., some items are rated while others not, are generally ignored. We argue that failing to observe such response patterns can lead to biased parameter estimation and sub-optimal model performanc…

Paper bounds PAC RL sample complexity in deterministic MDPs.

New algorithms identify best policies in discounted linear MDPs efficiently.

Optimizes ranking of top-k players from partial comparison data.

One of the key approaches to save samples in reinforcement learning (RL) is to use knowledge from an approximate model such as its simulator. However, how much does an approximate model help to learn a near-optimal policy of the true unknown model? Despite numerous empirical studies of transfer reinforcement learning, …

New algorithm for identifying optimal arms in stochastic bandit problems.

We consider effort allocation in crowdsourcing, where we wish to assign labeling tasks to imperfect homogeneous crowd workers to maximize overall accuracy in a continuous-time Bayesian setting, subject to budget and time constraints. The Bayes-optimal policy for this problem is the solution to a partially observable Ma…

This work develops a unified framework for RLHF with general $f$-divergence regularization.

The paper tackles system identification via Hankel nuclear norm regularization, improving estimation rates and singular value gaps.

Algorithm finds a simplified model for reinforcement learning under agent limitations.

IRM fails to capture natural invariances on simple problems.

Aims to create safe reinforcement learning policies by considering individual harm.

We consider a finite-armed structured bandit problem in which mean rewards of different arms are known functions of a common hidden parameter $θ^*$. Since we do not place any restrictions of these functions, the problem setting subsumes several previously studied frameworks that assume linear or invertible reward funct…

SAIL learns from sub-optimal demonstrations to improve sample efficiency in sparse reward tasks.

The paper sets fundamental limits for ERM in high dimensions.

Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference between wide and narrow networks. In this work, we prove that from narrow to wide networks, there is a phase transition from havi…

Improves robust transfer learning with side information.

LinMED is a new linear bandit algorithm with near-optimal regret bound.

Two efficient algorithms improve online item recommendation for large user-item matrices.

LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.

Paper establishes first instance-dependent lower bound for PAC reinforcement learning.

LoCo-RLHF models diverse human feedback with contextual information.

ELM improves neural model embeddings for long-tail learning.

Does over-parameterization eliminate sub-optimal local minima for neural networks? An affirmative answer was given by a classical result in [59] for 1-hidden-layer wide neural networks. A few recent works have extended the setting to multi-layer neural networks, but none of them has proved every local minimum is global…

The paper compares theoretical and empirical performance of imputation methods for missing data.

We address reinforcement learning problems with finite state and action spaces where the underlying MDP has some known structure that could be potentially exploited to minimize the exploration rates of suboptimal (state, action) pairs. For any arbitrary structure, we derive problem-specific regret lower bounds satisfie…

Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.

Neural networks outperform NTK on compositional tasks, revealing a complexity gap.

In autonomous vehicle (AV) control, allowing mistakes can be quite dangerous and costly in the real world. For this reason we investigate methods of training an AV without allowing the agent to explore and instead having a human explorer collect the data. Supervised learning has been explored for AV control, but it enc…

Flexible algorithms for maximizing rewards in structured bandits.

Paper improves risk bound for MTL with graph-dependent data.