arXiv research

We optimize rebalancing options by limiting asset allocations to a few choices, reducing the price and guaranteeing near-optimal performance.

Rewriting history improves RL algorithms for solving multiple tasks.

This paper derives a robust on-line equity trading algorithm that achieves the greatest possible percentage of the final wealth of the best pairs rebalancing rule in hindsight. A pairs rebalancing rule chooses some pair of stocks in the market and then perpetually executes rebalancing trades so as to maintain a target …

HTRPO tackles sparse rewards in RL with improved stability and performance.

Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…

Motivated by the pricing of lookback options in exponential Lévy models, we study the difference between the continuous and discrete supremum of Lévy processes. In particular, we extend the results of Broadie et al. (1999) to jump-diffusion models. We also derive bounds for general exponential Lévy models.

New algorithms learn POMDPs efficiently with hindsight observability.

Framework uses hindsight regret to audit marketing budget allocations.

Improved Q-learning with hindsight factor reduces overestimation errors.

Generalized Hindsight improves RL by reusing data from one task for another.

Efficient RL in partially observable risk-sensitive environments with hindsight observations.

This paper prices and replicates the financial derivative whose payoff at $T$ is the wealth that would have accrued to a $\$1$ deposit into the best continuously-rebalanced portfolio (or fixed-fraction betting scheme) determined in hindsight. For the single-stock Black-Scholes market, Ordentlich and Cover (1998) only p…

Interactive learning with hindsight instruction feedback achieves better performance than traditional methods.

A new EM framework for goal-conditioned RL improves performance on sparse reward tasks.

New method combines curiosity and hindsight for stacking blocks.

Paper introduces a new $G^\star$ regret measure for online convex optimization with smooth losses.

Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In thes…

PCHID improves sample efficiency in reinforcement learning tasks.

New algorithms assign credit to past decisions based on hindsight.

HGG generates goals to improve sample efficiency in robotic tasks.

Dynamical-VAE learns causal dynamics from POMDPs using future information.

New method uses hindsight to make exploration robust in stochastic environments.

HL algorithms improve resource allocation in cloud environments.

In this paper, we consider an online optimization process, where the objective functions are not convex (nor concave) but instead belong to a broad class of continuous submodular functions. We first propose a variant of the Frank-Wolfe algorithm that has access to the full gradient of the objective functions. We show t…

A new algorithm optimizes multiple molecular properties efficiently.

I introduce and analyse an anytime version of the Optimally Confident UCB (OCUCB) algorithm designed for minimising the cumulative regret in finite-armed stochastic bandits with subgaussian noise. The new algorithm is simple, intuitive (in hindsight) and comes with the strongest finite-time regret guarantees for a hori…

New method improves meta-reinforcement learning efficiency.

In Hindsight Experience Replay (HER), a reinforcement learning agent is trained by treating whatever it has achieved as virtual goals. However, in previous work, the experience was replayed at random, without considering which episode might be the most valuable for learning. In this paper, we develop an energy-based fr…

Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…

HO2 learns options from data efficiently, improving robot manipulation tasks.

This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.

HGAIL learns policies without expert demonstrations.

New algorithm combines curriculum learning with HER for complex object manipulation tasks.

Efficient algorithm controls unknown systems with adversarial perturbations.

ACTRCE uses natural language to improve reinforcement learning performance.

Efficient algorithm for unknown linear systems with convex costs.

VRM uses mixtures to adaptively reduce variance in stochastic optimization.

The paper improves HER by prioritizing virtual goals and removing misleading samples.

Optimal control in changing systems without strong convexity assumptions.

HIGhER uses language to generate new instructions for better learning from mistakes.

This note improves on universal portfolios by using bilinear strategies.

In the standard setting of approachability there are two players and a target set. The players play repeatedly a known vector-valued game where the first player wants to have the average vector-valued payoff converge to the target set which the other player tries to exclude it from this set. We revisit this setting in …

GDT improves reinforcement learning by matching future state information efficiently.

We propose algorithms for online principal component analysis (PCA) and variance minimization for adaptive settings. Previous literature has focused on upper bounding the static adversarial regret, whose comparator is the optimal fixed action in hindsight. However, static regret is not an appropriate metric when the un…

We consider a basic problem at the interface of two fundamental fields: submodular optimization and online learning. In the online unconstrained submodular maximization (online USM) problem, there is a universe $[n]=\{1,2,...,n\}$ and a sequence of $T$ nonnegative (not necessarily monotone) submodular functions arrive …

Paper develops online learning algorithms for quaternion ARMA models.

In E-commerce advertising, where product recommendations and product ads are presented to users simultaneously, the traditional setting is to display ads at fixed positions. However, under such a setting, the advertising system loses the flexibility to control the number and positions of ads, resulting in sub-optimal p…

An online framework optimizes efficiency in conformal prediction with a target miscoverage rate.