arXiv research

The study examines robust decision-making in volatile financial markets, finding action robustness is more impactful than uncertainty tolerance.

A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action $a$, and (i) with probability $α$, an alt…

Proposes MDR estimator for unbiased OPE with large action spaces.

A new reinforcement learning method reduces action complexity for robust control.

This work improves deep reinforcement learning robustness to adversarial state uncertainty.

Robust reinforcement learning aims to produce policies that have strong guarantees even in the face of environments/transition models whose parameters have strong uncertainty. Existing work uses value-based methods and the usual primitive action setting. In this paper, we propose robust methods for learning temporally …

New approach to meaningful and robust algorithmic recourse.

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. …

Study robust control for systems with continuous states using adversarial perturbations.

Study robust best-arm identification in linear bandits with lower bounds and algorithms.

A scheme robust to action erasures improves MAB performance.

In his 1985 paper Sullivan sketched a proof of his structural stability theorem for differentiable group actions satisfying certain expansion-hyperbolicity axioms. In this paper we relax Sullivan's axioms and introduce a notion of "meandering hyperbolicity" for group actions on geodesic metric spaces. This generalizati…

Paper proposes a framework to assess model robustness against adversarial actions.

Learning how to act when there are many available actions in each state is a challenging task for Reinforcement Learning (RL) agents, especially when many of the actions are redundant or irrelevant. In such cases, it is sometimes easier to learn which actions not to take. In this work, we propose the Action-Elimination…

Paper robustifies reinforcement learning agents against action space perturbations.

Study optimal offline RL with uncertainty sets and distribution shifts.

Robust algorithm optimizes corrupted Gaussian process bandits.

We study decision making in environments where the reward is only partially observed, but can be modeled as a function of an action and an observed context. This setting, known as contextual bandits, encompasses a wide variety of applications including health-care policy and Internet advertising. A central task is eval…

New attack manipulates UCB algorithm, new defense algorithm reduces pseudo-regret.

This paper improves causal inference using deep neural networks for low-dimensional covariates.

RRPI improves offline RL by optimizing policies against worst-case dynamics.

New method improves DRL robustness against adversarial state observations.

This study examines how learning algorithms affect collective action in machine learning.

Linear disentangled representations improve unsupervised action estimation.

New approach makes deep reinforcement learning robust without assuming adversary knowledge.

New methods estimate policy value and gradients for deterministic policies from off-policy data.

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how i…

Paper robustifies reinforcement learning with risk-averse methods.

The paper develops a method to create robust control policies for robots using information bottlenecks.

Unified framework for corruption-robust linear bandits with optimal gap-dependent misspecification bounds.

New definition of interpretability makes model design more actionable.

Temporal-difference (TD) networks are a class of predictive state representations that use well-established TD methods to learn models of partially observable dynamical systems. Previous research with TD networks has dealt only with dynamical systems with finite sets of observations and actions. We present an algorithm…

Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.

Certifiably robust VAEs are trained with bounds on input perturbations.

We consider a new family of operators for reinforcement learning with the goal of alleviating the negative effects and becoming more robust to approximation or estimation errors. Various theoretical results are established, which include showing on a sample path basis that our family of operators preserve optimality an…

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expe…

Training deep reinforcement learning agents complex behaviors in 3D virtual environments requires significant computational resources. This is especially true in environments with high degrees of aliasing, where many states share nearly identical visual features. Minecraft is an exemplar of such an environment. We hypo…

RORL improves offline RL robustness with conservative smoothing.

Paper addresses privacy and robustness in stochastic linear bandits.

Action-bisimulation learns long-horizon controllability for reinforcement learning.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

New method uses hindsight to make exploration robust in stochastic environments.

Improved GFlowNets learn more efficiently with trajectory balance.

This paper studies semiparametric contextual bandits, a generalization of the linear stochastic bandit problem where the reward for an action is modeled as a linear function of known action features confounded by an non-linear action-independent term. We design new algorithms that achieve $\tilde{O}(d\sqrt{T})$ regret …

A reinforcement learning agent tries to maximize its cumulative payoff by interacting in an unknown environment. It is important for the agent to explore suboptimal actions as well as to pick actions with highest known rewards. Yet, in sensitive domains, collecting more data with exploration is not always possible, but…

Value-based reinforcement learning (RL) methods like Q-learning have shown success in a variety of domains. One challenge in applying Q-learning to continuous-action RL problems, however, is the continuous action maximization (max-Q) required for optimal Bellman backup. In this work, we develop CAQL, a (class of) algor…

New algorithm reduces sample size for robust reinforcement learning.

SDM Policy accelerates inference for robotic tasks while maintaining high action quality.