A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In thes…
We extend the classical risk minimization model with scalar risk measures to the general case of set-valued risk measures. The problem we obtain is a set-valued optimization model and we propose a goal programming-based approach with satisfaction function to obtain a solution which represents the best compromise betwee…
For an autonomous agent to fulfill a wide range of user-specified goals at test time, it must be able to learn broadly applicable and general-purpose skill repertoires. Furthermore, to provide the requisite level of generality, these skills must handle raw sensory input such as images. In this paper, we propose an algo…
A variety of cooperative multi-agent control problems require agents to achieve individual goals while contributing to collective success. This multi-goal multi-agent setting poses difficulties for recent algorithms, which primarily target settings with a single global reward, due to two new challenges: efficient explo…
This work improves imitation learning and goal-conditioned RL by estimating value densities.
problem Effective solutions for imitation and goal-conditioned reinforcement learning require reliably reaching specified states or demonstrations.
method The approach uses recent advances in density estimation to learn value functions efficiently and without hindsight bias.
result The method achieves state-of-the-art demonstration sample-efficiency in imitation learning and is both efficient and bias-free in goal-conditioned reinforcement learning.
All-goals updating exploits the off-policy nature of Q-learning to update all possible goals an agent could have from each transition in the world, and was introduced into Reinforcement Learning (RL) by Kaelbling (1993). In prior work this was mostly explored in small-state RL problems that allowed tabular representati…
Generating novel graph structures that optimize given objectives while obeying some given underlying rules is fundamental for chemistry, biology and social science research. This is especially important in the task of molecular graph generation, whose goal is to discover novel molecules with desired properties such as …
In this paper, we study a new learning paradigm for Neural Machine Translation (NMT). Instead of maximizing the likelihood of the human translation as in previous works, we minimize the distinction between human translation and the translation given by an NMT model. To achieve this goal, inspired by the recent success …
A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization. To this end, we introduce universal planning networks (UPN). UPNs embed differentiable planning within a goal-directed policy. This planning computation unrolls a for…
The goals of this article are twofold : 1) to compute the conjugate locus of a geodesic that lies in the center of a simply connected, 2-step nilpotent Lie group with a left invariant metric 2) compare the isometry types of two such nilpotent Lie groups whose conjugate loci for central geodesics are "the same" in a sui…
Goals for reinforcement learning problems are typically defined through hand-specified rewards. To design such problems, developers of learning algorithms must inherently be aware of what the task goals are, yet we often require agents to discover them on their own without any supervision beyond these sparse rewards. W…
The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.
problem Ensuring an autonomous system achieves the user's goal with calibrated probability estimates.
method Invertible conformal prediction using Probability-space Conformalized Quantile Regression (PCQR) to produce well-calibrated conditional prediction intervals.
result The method produces well-calibrated probabilities that the cumulative reward will fall within a user-specified target interval, with finite-sample guarantees.
We present an active learning architecture that allows a robot to actively learn which data collection strategy is most efficient for acquiring motor skills to achieve multiple outcomes, and generalise over its experience to achieve new outcomes. The robot explores its environment both via interactive learning and goal…
We study the problem of efficient PAC active learning of homogeneous linear classifiers (halfspaces) in Rd, where the goal is to learn a halfspace with low error using as few label queries as possible. Under the extra assumption that there is a t-sparse halfspace that performs well on the data (t≪d)…