A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This paper studies poisoning attacks in episodic RL and discovers their effectiveness depends on reward bounds.
A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.
Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. Howe…
Manipulating data, such as weighting data examples or augmenting with new instances, has been increasingly used to improve model training. Previous work has studied various rule- or learning-based approaches designed for specific types of data manipulation. In this work, we propose a new method that supports learning d…
A new method for robot manipulation tasks using imagined object goals.
New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.
Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward shaping method through predictive coding to tackle sparse reward problems. By learning predictive repr…
Manipulation tasks such as preparing a meal or assembling furniture remain highly challenging for robotics and vision. Traditional task and motion planning (TAMP) methods can solve complex tasks but require full state observability and are not adapted to dynamic scene changes. Recent learning methods can operate direct…
New attack manipulates UCB algorithm, new defense algorithm reduces pseudo-regret.
We develop an option pricing model based on a tug-of-war game. This two-player zero-sum stochastic differential game is formulated in the context of a multi-dimensional financial market. The issuer and the holder try to manipulate asset price processes in order to minimize and maximize the expected discounted reward. W…
PixL2R maps natural language to pixel-based rewards for RL, improving sample efficiency.
We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempt…
Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficien…
Malicious agents can manipulate linear contextual bandits to pull desired arms with logarithmic overhead.
Motivated by economic applications such as recommender systems, we study the behavior of stochastic bandits algorithms under \emph{strategic behavior} conducted by rational actors, i.e., the arms. Each arm is a \emph{self-interested} strategic player who can modify its own reward whenever pulled, subject to a cross-per…
We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performanc…
ORIL learns a reward function from unlabeled data to improve robot learning.
Shaping in humans and animals has been shown to be a powerful tool for learning complex tasks as compared to learning in a randomized fashion. This makes the problem less complex and enables one to solve the easier sub task at hand first. Generating a curriculum for such guided learning involves subjecting the agent to…
Transforming sparse outcomes into dense process rewards for efficient reinforcement learning.
Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, …
We study adversarial attacks that manipulate the reward signals to control the actions chosen by a stochastic multi-armed bandit algorithm. We propose the first attack against two popular bandit algorithms: -greedy and UCB, \emph{without} knowledge of the mean rewards. The attacker is able to spend only logarithmic …
This paper tackles fair online decision-making in contextual bandits, achieving optimal performance and fairness.
Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of…
Efficient exploration is necessary to achieve good sample efficiency for reinforcement learning in general. From small, tabular settings such as gridworlds to large, continuous and sparse reward settings such as robotic object manipulation tasks, exploration through adding an uncertainty bonus to the reward function ha…
Optimizes reward learning design for complex tasks using nonparametric methods.
New RL approach infers optimal policies via variational inference.
New method uses explicit human demonstrations to teach missing features in reward learning.
Algorithm learns optimal coordination for strategic agents in uncertain settings.
PlanGAN uses GANs to plan efficient trajectories for multi-goal tasks in sparse reward environments.
SEMI uses multisensory incongruity to self-supervise exploration in reinforcement learning.
This work tackles real-world robotic reinforcement learning challenges.
Attackers can significantly reduce team rewards in cooperative multi-agent reinforcement learning.
Self-supervised policy adapts after deployment without rewards.
Recent reinforcement learning algorithms, though achieving impressive results in various fields, suffer from brittle training effects such as regression in results and high sensitivity to initialization and parameters. We claim that some of the brittleness stems from variance differences, i.e. when different environmen…
Robot learns to juggle two balls from 56 minutes of experience.
This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.
This paper optimizes attacks on stochastic bandits and proposes defenses against them.
A new reinforcement learning method uses mutual information to encourage agents to control their environment.
The combination of deep neural network models and reinforcement learning algorithms can make it possible to learn policies for robotic behaviors that directly read in raw sensory inputs, such as camera images, effectively subsuming both estimation and control into one model. However, real-world applications of reinforc…
Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gati…
Study intrinsic motivation for synergistic tasks in reinforcement learning.
While reinforcement learning (RL) has the potential to enable robots to autonomously acquire a wide range of skills, in practice, RL usually requires manual, per-task engineering of reward functions, especially in real world settings where aspects of the environment needed to compute progress are not directly accessibl…
AI learns market manipulation through simulation, suggesting regulation.
Reinforcement learning (RL) has advanced greatly in the past few years with the employment of effective deep neural networks (DNNs) on the policy networks. With the great effectiveness came serious vulnerability issues with DNNs that small adversarial perturbations on the input can change the output of the network. Sev…
The paper analyzes how leverage affects manipulation in event-linked markets, offering new insights into regulation.
Eikonal-Constrained QRL improves goal-reaching in reinforcement learning.
A Markov Decision Process (MDP) is a popular model for reinforcement learning. However, its commonly used assumption of stationary dynamics and rewards is too stringent and fails to hold in adversarial, nonstationary, or multi-agent problems. We study an episodic setting where the parameters of an MDP can differ across…