A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Framework uses expert intervention to solve long-horizon reinforcement learning tasks.
Max entropy exploration guides reinforcement learning agents to pursue achievable goals.
We present relay policy learning, a method for imitation and reinforcement learning that can solve multi-stage, long-horizon robotic tasks. This general and universally-applicable, two-phase approach consists of an imitation learning stage that produces goal-conditioned hierarchical policies, and a reinforcement learni…
Accelerates TD learning for long-horizon reinforcement learning problems.
Long horizon reinforcement learning is as hard as short horizon learning.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
Action-bisimulation learns long-horizon controllability for reinforcement learning.
The paper improves model-based reinforcement learning by using multi-timestep objectives.
TRM improves long-horizon LLM RL by masking divergent sequences.
This work improves RL for complex robotic tasks by guiding exploration with task-specific goal distributions.
SGM combines deep learning and planning for robust long-horizon tasks.
Q-chunking improves RL for long tasks by chunking actions.
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we pro…
Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated action…
We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the…
Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.
Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combi…
A new algorithm reduces memory and computational needs for reinforcement learning.
Study proposes adaptive RL for dynamic portfolio optimization.
Plan2Vec learns image representations without labels, improving control tasks.
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward shaping method through predictive coding to tackle sparse reward problems. By learning predictive repr…
Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.
A new reinforcement learning method uses model derivatives to improve policy optimization.
Reward tweaking optimizes behavior for long-term goals by adjusting the reward function.
In this work, we take a representation learning perspective on hierarchical reinforcement learning, where the problem of learning lower layers in a hierarchy is transformed into the problem of learning trajectory-level generative models. We show that we can learn continuous latent representations of trajectories, which…
A new method reduces compounding errors in model-based reinforcement learning.
Model-based reinforcement learning (MBRL) aims to learn a dynamic model to reduce the number of interactions with real-world environments. However, due to estimation error, rollouts in the learned model, especially those of long horizons, fail to match the ones in real-world environments. This mismatching has seriously…
HiDe learns hierarchical control for complex tasks by separating planning and control.
New method selects best offline RL policies from logged data.
Representation learning is a central challenge across a range of machine learning areas. In reinforcement learning, effective and functional representations have the potential to tremendously accelerate learning progress and solve more challenging problems. Most prior work on representation learning has focused on gene…
NDPs embed dynamical systems into neural networks for efficient sensorimotor learning.
A new method calibrates value predictions in offline RL to improve reliability.
Latent-state environments with long horizons, such as those faced by recommender systems, pose significant challenges for reinforcement learning (RL). In this work, we identify and analyze several key hurdles for RL in such environments, including belief state error and small action advantage. We develop a general prin…
In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execu…
FDS tackles long horizon hyperparameter optimization issues.
Hierarchical reinforcement learning is a promising approach to tackle long-horizon decision-making problems with sparse rewards. Unfortunately, most methods still decouple the lower-level skill acquisition process and the training of a higher level that controls the skills in a new task. Leaving the skills fixed can le…
Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.
Motivated by the many real-world applications of reinforcement learning (RL) that require safe-policy iterations, we consider the problem of off-policy evaluation (OPE) -- the problem of evaluating a new policy using the historical data obtained by different behavior policies -- under the model of nonstationary episodi…
Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation, requires the use of mobile manipulators: mobile bases with manipulation capabilities. Interactive N…
New method for efficient online exploration in RLHF reduces regret.
Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent ba…
This review tackles long horizon forecasting in time series analysis using deep learning.
New method estimates off-policy data without needing known behavior policy.
State-of-the-art forecasting methods using Recurrent Neural Net- works (RNN) based on Long-Short Term Memory (LSTM) cells have shown exceptional performance targeting short-horizon forecasts, e.g given a set of predictor features, forecast a target value for the next few time steps in the future. However, in many appli…
Paper proposes a new RL approach combining IL and RL methods to improve decision-making.
PRISM integrates diverse rewards in MORL, improving sample efficiency and Pareto coverage.
The objective of this work is to augment the basic abilities of a robot by learning to use new sensorimotor primitives to enable the solution of complex long-horizon problems. Solving long-horizon problems in complex domains requires flexible generative planning that can combine primitive abilities in novel combination…
We explore the use of Evolution Strategies (ES), a class of black box optimization algorithms, as an alternative to popular MDP-based RL techniques such as Q-learning and Policy Gradients. Experiments on MuJoCo and Atari show that ES is a viable solution strategy that scales extremely well with the number of CPUs avail…