A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method estimates state-action stationary distribution for better off-policy policy evaluation.
Aims to learn from multiple unpredictable teachers with minimal interaction.
SafeMIL learns safer policies by avoiding risky behavior from non-preferred trajectories.
Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.
ICIL learns policies invariant to multiple environments, improving generalization.
We study the problem of controllable generation of long-term sequential behaviors, where the goal is to calibrate to multiple behavior styles simultaneously. In contrast to the well-studied areas of controllable generation of images, text, and speech, there are two questions that pose significant challenges when genera…
Paper tackles offline RL from mixed datasets with adaptive KL regularizer.
Humans are able to perform a myriad of sophisticated tasks by drawing upon skills acquired through prior experience. For autonomous agents to have this capability, they must be able to extract reusable skills from past experience that can be recombined in new ways for subsequent tasks. Furthermore, when controlling com…
When learning policies for real-world domains, two important questions arise: (i) how to efficiently use pre-collected off-policy, non-optimal behavior data; and (ii) how to mediate among different competing objectives and constraints. We thus study the problem of batch policy learning under multiple constraints, and o…
This article develops a deep reinforcement learning (Deep-RL) framework for dynamic pricing on managed lanes with multiple access locations and heterogeneity in travelers' value of time, origin, and destination. This framework relaxes assumptions in the literature by considering multiple origins and destinations, multi…
We consider the problem of off-policy evaluation in Markov decision processes. Off-policy evaluation is the task of evaluating the expected return of one policy with data generated by a different, behavior policy. Importance sampling is a technique for off-policy evaluation that re-weights off-policy returns to account…
Industrial recommender systems deal with extremely large action spaces -- many millions of items to recommend. Moreover, they need to serve billions of users, who are unique at any point in time, making a complex user state space. Luckily, huge quantities of logged implicit feedback (e.g., user clicks, dwell time) are …
Standard reinforcement learning methods aim to master one way of solving a task whereas there may exist multiple near-optimal policies. Being able to identify this collection of near-optimal policies can allow a domain expert to efficiently explore the space of reasonable solutions. Unfortunately, existing approaches t…
We propose a policy improvement algorithm for Reinforcement Learning (RL) which is called Rerouted Behavior Improvement (RBI). RBI is designed to take into account the evaluation errors of the Q-function. Such errors are common in RL when learning the -value from finite past experience data. Greedy policies or even …
The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.
The paper tackles robust policy learning in MDPs using statistical methods.
We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space. We show that by utilizing the dual formulation of the WD, we can learn score functions over policy behaviors that can in turn be used to lead policy optimization towar…
New rule reduces exploration regret to logarithmic, improving bad episode handling.
The study models life insurance policy cancellations using statistical and machine learning methods.
A new pricing strategy minimizes regret by controlling strategic buyer behavior.
Learning a policy using only observational data is challenging because the distribution of states it induces at execution time may differ from the distribution observed during training. We propose to train a policy by unrolling a learned model of the environment dynamics over multiple time steps while explicitly penali…
Hierarchical Reinforcement Learning (HRL) exploits temporally extended actions, or options, to make decisions from a higher-dimensional perspective to alleviate the sparse reward problem, one of the most challenging problems in reinforcement learning. The majority of existing HRL algorithms require either significant m…
ABPS improves RL training efficiency by sharing policies and evolving hyper-params.
Paper proposes a framework for reliable off-policy evaluation in reinforcement learning.
Estimates RL data for dynamic treatment effects using GMM.
We propose a novel approach to train a multi-modal policy from mixed demonstrations without their behavior labels. We develop a method to discover the latent factors of variation in the demonstrations. Specifically, our method is based on the variational autoencoder with a categorical latent variable. The encoder infer…
We make policy optimization algorithms batch size-invariant by decoupling proximal and behavior policies.
AIPS improves ranking policy evaluation by adapting to diverse user behavior.
CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.
A method for a single policy to solve various tasks across diverse agent morphologies.
We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempt…
Off-policy learning exhibits greater instability when compared to on-policy learning in reinforcement learning (RL). The difference in probability distribution between the target policy () and the behavior policy (b) is a major cause of instability. High variance also originates from distributional mismatch. The var…
Learning algorithms are enabling robots to solve increasingly challenging real-world tasks. These approaches often rely on demonstrations and reproduce the behavior shown. Unexpected changes in the environment may require using different behaviors to achieve the same effect, for instance to reach and grasp an object in…
Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-…
FOCOPS optimizes agent's behavior while adhering to constraints.
CQL (ReDS) learns from varied driving behaviors, improving offline RL performance.
Study strategic dynamic pricing for buyers with unknown manipulation costs.
Paper proposes Cycle-of-Learning framework for better reinforcement learning performance.
Deep learning analyzes healthcare provider actions and patient outcomes.
Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …
KL-regularized RL from expert demos can lead to slow, unstable learning.
New method estimates and optimizes policy differences using orthogonal learning.
New estimator GMIPS reduces variance in ranking policy evaluation.
We study the problem of off-policy policy optimization in Markov decision processes, and develop a novel off-policy policy gradient method. Prior off-policy policy gradient approaches have generally ignored the mismatch between the distribution of states visited under the behavior policy used to collect data, and what …
Paper finds efficient OPE estimator for multiple logging policies with minimum variance.
This paper solves steady-state planning for multichain MDPs.
Method learns evolving policies in healthcare contexts.
Study improves off-policy evaluation from non-i.i.d. bandit samples.