arXiv research

Interactive learning with hindsight instruction feedback achieves better performance than traditional methods.

HIGhER uses language to generate new instructions for better learning from mistakes.

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and un…

Hindsight Experience Replay (HER) is a multi-goal reinforcement learning algorithm for sparse reward functions. The algorithm treats every failure as a success for an alternative (virtual) goal that has been achieved in the episode. Virtual goals are randomly selected, irrespective of which are most instructive for the…

Improves reinforcement learning for complex tasks with sparse feedback.

Method constructs finance LLMs without instruction data using pretraining and model merging.

New algorithms learn POMDPs efficiently with hindsight observability.

Reinforcement Learning(RL) with sparse rewards is a major challenge. We propose \emph{Hindsight Trust Region Policy Optimization}(HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with \emph{hindsight} to tackle the challenge of sparse rewards. Hindsight refers to the algorithm's ability to l…

Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In thes…

PCHID improves sample efficiency in reinforcement learning tasks.

We study T. Cover's rebalancing option (Ordentlich and Cover 1998) under discrete hindsight optimization in continuous time. The payoff in question is equal to the final wealth that would have accrued to a $\$1$ deposit into the best of some finite set of (perhaps levered) rebalancing rules determined in hindsight. A r…

Framework uses hindsight regret to audit marketing budget allocations.

New algorithms assign credit to past decisions based on hindsight.

Generalized Hindsight improves RL by reusing data from one task for another.

Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images o…

Efficient RL in partially observable risk-sensitive environments with hindsight observations.

Learning in environments with large state and action spaces, and sparse rewards, can hinder a Reinforcement Learning (RL) agent's learning through trial-and-error. For instance, following natural language instructions on the Web (such as booking a flight ticket) leads to RL settings where input vocabulary and number of…

Dynamical-VAE learns causal dynamics from POMDPs using future information.

In this paper, we propose a framework that enables a human teacher to shape a robot behaviour by interactively providing it with unlabeled instructions. We ground the meaning of instruction signals in the task-learning process, and use them simultaneously for guiding the latter. We implement our framework as a modular …

A method to improve LLMs by automating the construction of a mixture of expert prompts.

New method uses hindsight to make exploration robust in stochastic environments.

Improved financial sentiment analysis using simple instruction tuning of LLMs.

This paper derives a robust on-line equity trading algorithm that achieves the greatest possible percentage of the final wealth of the best pairs rebalancing rule in hindsight. A pairs rebalancing rule chooses some pair of stocks in the market and then perpetually executes rebalancing trades so as to maintain a target …

Rewriting history improves RL algorithms for solving multiple tasks.

Improves matrix multiplication throughput for asymmetric bit-width operands.

Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficien…

In classical Q-learning, the objective is to maximize the sum of discounted rewards through iteratively using the Bellman equation as an update, in an attempt to estimate the action value function of the optimal policy. Conventionally, the loss function is defined as the temporal difference between the action value and…

Paper fine-tunes LLaMA-3-8B for financial NER using instruction and LoRA.

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the predict…

HL algorithms improve resource allocation in cloud environments.

A new EM framework for goal-conditioned RL improves performance on sparse reward tasks.

RAG-IT automates financial analysis using LLMs and specialized datasets.

New method improves meta-reinforcement learning efficiency.

In Hindsight Experience Replay (HER), a reinforcement learning agent is trained by treating whatever it has achieved as virtual goals. However, in previous work, the experience was replayed at random, without considering which episode might be the most valuable for learning. In this paper, we develop an energy-based fr…

Study shows model size inversely affects few-shot instruction accuracy.

Enhances LLMs for predicting stock movements by considering news dissemination and context.

Experience replay is an important technique for addressing sample-inefficiency in deep reinforcement learning (RL), but faces difficulty in learning from binary and sparse rewards due to disproportionately few successful experiences in the replay buffer. Hindsight experience replay (HER) was recently proposed to tackle…

Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functio…

Compared to reinforcement learning, imitation learning (IL) is a powerful paradigm for training agents to learn control policies efficiently from expert demonstrations. However, in most cases, obtaining demonstration data is costly and laborious, which poses a significant challenge in some scenarios. A promising altern…

Improved robot navigation using multi-head attention for natural language instructions.

This paper prices and replicates the financial derivative whose payoff at $T$ is the wealth that would have accrued to a $\$1$ deposit into the best continuously-rebalanced portfolio (or fixed-fraction betting scheme) determined in hindsight. For the single-stock Black-Scholes market, Ordentlich and Cover (1998) only p…

This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.

This paper benchmarks FinGPT for financial datasets using open-source large language models.

This paper proposes a new method to connect language and physical actions in reinforcement learning.

LLM Pro Finance Suite enhances financial NLP with instruction-tuned models.

SymNoise improves language model fine-tuning by 6.7% over NEFTune, using symmetric noise.

InvestLM is a financial domain LLM tuned on LLaMA-65B for investment advice.

New algorithm combines curriculum learning with HER for complex object manipulation tasks.