arXiv research

New algorithm optimally identifies best arm in both stochastic and adversarial settings.

Study non-linear combinatorial bandits with polynomial rewards, finding significant differences from linear cases.

New findings on universal learning in contextual bandits with adversarial rewards.

New reward function improves GAIL performance in task-based environments.

A multi-player bandit system resists adversarial attacks with near-optimal regret.

Improves imitation learning in RL by learning reward function efficiently.

The paper shows how policy regularization acts like an adversary to improve robustness.

WDAIL uses Wasserstein distance for more effective reward shaping in IL.

Adaptive MAB algorithms handle composite, anonymous feedback without reward interval knowledge.

This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproduc…

Explicit engineering of reward functions for given environments has been a major hindrance to reinforcement learning methods. While Inverse Reinforcement Learning (IRL) is a solution to recover reward functions from demonstrations only, these learned rewards are generally heavily \textit{entangled} with the dynamics of…

Regularized policies are robust to adversarial rewards.

Algorithm maximizes total reward in multi-agent bandits with adversarial corruptions.

Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.

Study on reward poisoning attacks on CMAB, revealing attackability depends on adversary's knowledge.

We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performanc…

We consider a problem of learning the reward and policy from expert examples under unknown dynamics. Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies. Empowe…

AMIGo uses adversarial intrinsic goals to teach RL agents new skills.

Adversarial CBO optimizes under interventions by adversaries and non-stationarities.

Paper improves AIRL by enhancing policy imitation and addressing reward recovery issues.

Algorithm tackles adaptive discretization in adversarial Lipschitz bandits for dynamic pricing and auctions.

New models for bandit problems with fidelity rewards are introduced and analyzed.

Paper optimizes learning MDPs with adversarial rewards using linear models.

Adversarial Imitation Learning (AIL) is a broad family of imitation learning methods designed to mimic expert behaviors from demonstrations. While AIL has shown state-of-the-art performance on imitation learning with only small number of demonstrations, it faces several practical challenges such as potential training i…

New RL algorithm tackles adversarial RMAB with unknown transitions and bandit feedback.

This paper studies GAIL's global convergence for general MDP and nonlinear rewards.

This paper optimizes attacks on stochastic bandits and proposes defenses against them.

Contextual bandits study how reward variance affects regret bounds.

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, …

Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularizatio…

New model for display advertising with stochastic and adversarial components.

We seek to align agent policy with human expert behavior in a reinforcement learning (RL) setting, without any prior knowledge about dynamics, reward function, and unsafe states. There is a human expert knowing the rewards and unsafe states based on his preference and objective, but querying that human expert is expens…

Reinforcement learning agents are prone to undesired behaviors due to reward mis-specification. Finding a set of reward functions to properly guide agent behaviors is particularly challenging in multi-agent scenarios. Inverse reinforcement learning provides a framework to automatically acquire suitable reward functions…

Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments. Recent advances in adversarial learning have allowed extending inverse RL to applications with non-stationary environment …

Motivated by applications of bandit algorithms in education, we consider a stochastic multi-armed bandit problem with $\varepsilon$-contaminated rewards. We allow an adversary to give arbitrary unbounded contaminated rewards with full knowledge of the past and future. We impose the constraint that for each time $t$ the…

INF-clip optimizes heavy-tailed MAB problems with improved performance.

The growing use of virtual autonomous agents in applications like games and entertainment demands better control policies for natural-looking movements and actions. Unlike the conventional approach of hard-coding motion routines, we propose a deep learning method for obtaining control policies by directly mimicking raw…

This paper investigates the adversarial Bandits with Knapsack (BwK) online learning problem, where a player repeatedly chooses to perform an action, pays the corresponding cost, and receives a reward associated with the action. The player is constrained by the maximum budget $B$ that can be spent to perform actions, an…

I describe an optimal control view of adversarial machine learning, where the dynamical system is the machine learner, the input are adversarial actions, and the control costs are defined by the adversary's goals to do harm and be hard to detect. This view encompasses many types of adversarial machine learning, includi…

New algorithm resists corruption in linear contextual bandits.

Multi-armed bandits a simple but very powerful framework for algorithms that make decisions over time under uncertainty. An enormous body of work has accumulated over the years, covered in several books and surveys. This book provides a more introductory, textbook-like treatment of the subject. Each chapter tackles a p…

This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.

There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a …

Robust Q-learning algorithm resists corrupted rewards.

New algorithm tackles adversarial corruption in Lipschitz bandits with sub-linear regret.

It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement l…

New algorithm tackles adversarial RL without horizon constraints.

Paper addresses privacy and robustness in stochastic linear bandits.