On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

18 results for “HRL”

Efficient Exploration through Intrinsic Motivation Learning for Unsupervised Subgoal Discovery in Model-Free Hierarchical Reinforcement Learningcs.LG

Paper shows intrinsic motivation boosts exploration efficiency in HRL.

problem Efficient exploration and subgoal discovery in model-free HRL.

method Unsupervised learning over agent's experiences for subgoal discovery.

result Intrinsic motivation learning improves exploration efficiency.

HRL Intrinsic Motivation Subgoal Discovery Reinforcement Learning

2019-11-18abs ↗pdf ↗

Diversity-Driven Extensible Hierarchical Reinforcement Learningcs.LG

Hierarchical reinforcement learning (HRL) has recently shown promising advances on speeding up learning, improving the exploration, and discovering intertask transferable skills. Most recent works focus on HRL with two levels, i.e., a master policy manipulates subpolicies, which in turn manipulate primitive actions. Ho…

2018-11-10abs ↗pdf ↗

Hierarchical Reinforcement Learning via Advantage-Weighted Information Maximizationcs.LG

Real-world tasks are often highly structured. Hierarchical reinforcement learning (HRL) has attracted research interest as an approach for leveraging the hierarchical structure of a given task in reinforcement learning (RL). However, identifying the hierarchical policy structure that enhances the performance of RL is n…

2019-01-05abs ↗pdf ↗

Heterogeneous Representation Learning: A Reviewcs.LG

This survey reviews Heterogeneous Representation Learning (HRL) for diverse data types.

problem Challenges in learning from heterogeneous data types.

method Unified learning framework for modeling various learning settings.

result Unified mathematical framework for multi-view, transfer, privileged, and multi-task learning.

Representation Learning Heterogeneous Data Multi-View Learning Transfer Learning Privileged Information

2020-04-28abs ↗pdf ↗

Provable Hierarchy-Based Meta-Reinforcement Learningcs.LG

Proves efficient learning of hierarchical structure in meta-reinforcement learning.

problem Lack of provable guarantees for learning hierarchical structures in reinforcement learning.

method Analyzed HRL in meta-RL setting with tabular transition dynamics, providing diversity conditions and regret bounds.

result Sample-efficient recovery of natural hierarchical structure with provable guarantees.

meta-reinforcement learning hierarchical reinforcement learning provability regret bounds diversity conditions

2021-10-18abs ↗pdf ↗

Learning and Exploiting Multiple Subgoals for Fast Exploration in Hierarchical Reinforcement Learningcs.LG

Hierarchical Reinforcement Learning (HRL) exploits temporally extended actions, or options, to make decisions from a higher-dimensional perspective to alleviate the sparse reward problem, one of the most challenging problems in reinforcement learning. The majority of existing HRL algorithms require either significant m…

2019-05-13abs ↗pdf ↗

Data-Efficient Hierarchical Reinforcement Learningcs.LG

Hierarchical reinforcement learning (HRL) is a promising approach to extend traditional reinforcement learning (RL) methods to solve more complex tasks. Yet, the majority of current HRL methods require careful task-specific design and on-policy training, making them difficult to apply in real-world scenarios. In this p…

2018-05-21abs ↗pdf ↗

Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcs.LG

The paper introduces an adjacency constraint to improve goal-conditioned HRL.

problem Training inefficiency in goal-conditioned HRL due to large action space.

method Restricting the high-level action space to a k-step adjacent region of the current state.

result The adjacency constraint preserves optimal hierarchical policies and improves HRL performance.

reinforcement learning hierarchical reinforcement learning subgoal generation adjacency constraint goal-conditioned

2020-06-20abs ↗pdf ↗

PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcs.LG

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may …

2018-04-20abs ↗pdf ↗

HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscs.LG

HRL4IN tackles interactive navigation tasks with mobile manipulators, improving efficiency and performance.

problem Interactive Navigation tasks require mobile manipulators to perform various actions, but choosing the right part of the embodiment is inefficient.

method HRL4IN uses a hierarchical reinforcement learning architecture to handle heterogeneous phases of navigation and manipulation, selecting the appropriate part of the embodiment for each phase.

result HRL4IN significantly outperforms flat PPO and HAC in terms of task performance and energy efficiency.

Hierarchical Reinforcement Learning Interactive Navigation Mobile Manipulators Energy Efficiency Task Performance

2019-10-24abs ↗pdf ↗

Hierarchical Policy Search via Return-Weighted Density Estimationcs.LG

Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gati…

2017-11-28abs ↗pdf ↗

Coagent Networks Revisitedcs.LG

Unified coagent network theory with execution paths.

problem Hierarchical reinforcement learning exploration-exploitation trade-off.

method Formalized coagent networks, introduced execution paths, revisited coagent network theory.

result Shorter proof of policy gradient theorem, generalizable to asynchronous coagents.

coagent networks reinforcement learning hierarchical learning execution paths policy gradient

2020-01-28abs ↗pdf ↗

Object-oriented state editing for HRLcs.LG

Agents use object-oriented reasoning to solve problems more quickly.

problem Finding solutions to problems more efficiently.

method Hierarchical controller directs low-level agent to simulate alternate states of the world.

result Achieves similar reward levels as non-hierarchical agents but with better data efficiency.

Hierarchical control Object-oriented reasoning State editing Hierarchical controller Low-level agent

2019-10-31abs ↗pdf ↗

Benchmarking Framework for Performance-Evaluation of Causal Inference Analysisstat.ME

Causal inference analysis is the estimation of the effects of actions on outcomes. In the context of healthcare data this means estimating the outcome of counter-factual treatments (i.e. including treatments that were not observed) on a patient's outcome. Compared to classic machine learning methods, evaluation and val…

2018-02-14abs ↗pdf ↗

SFCNeXt: a simple fully convolutional network for effective brain age estimation with small sample sizeeess.IV

SFCNeXt estimates brain age from small MRI datasets.

problem Efficiently estimating brain age from limited MRI data.

method Simple fully convolutional network (SFCNeXt) with SPEC and HRL.

result SFCNeXt outperforms complex models in small sample size scenarios.

brain age estimation fully convolutional network small sample size magnetic resonance imaging age prediction

2023-05-30abs ↗pdf ↗

Inter-Level Cooperation in Hierarchical Reinforcement Learningcs.LG

Improved cooperation between levels boosts reinforcement learning performance.

problem Training multi-level policies in hierarchical reinforcement learning.

method Modeling policy optimization as a multi-agent process and inducing cooperation between sub-policies.

result Inducing cooperation between sub-policies leads to stronger and more sample-efficient policies.

Hierarchical Reinforcement Learning Multi-Agent RL Policy Cooperation Sample Efficiency Transfer Learning

2019-12-05abs ↗pdf ↗

Demystifying Linear MDPs and Novel Dynamics Aggregation Frameworkstat.ML

We prove a lower bound for feature dimension in linear MDPs and propose a novel dynamics aggregation framework.

problem The limitation of feature dimension in linear MDPs and the need for efficient hierarchical reinforcement learning.

method We propose a novel dynamics aggregation framework based on structural dynamics and design a provably efficient hierarchical reinforcement learning algorithm.

result Our algorithm achieves a regret of

ilde{O} ( d_ψ^{3/2} H^{3/2}\sqrt{ N T} )

and meets the condition

d_ψ^3 N \ll d^{3}

in most real-world environments.

linear MDPs feature dimension dynamics aggregation hierarchical reinforcement learning provably efficient

2024-10-31abs ↗pdf ↗

MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Tradingcs.LG

MacroHFT uses memory and context-aware reinforcement learning to improve HFT performance.

problem Overfitting and biased decisions in HFT due to rapid market changes.

method Memory Augmented Context-aware Reinforcement Learning (MacroHFT) that trains multiple sub-agents and a hyper-agent.

result MacroHFT achieves state-of-the-art performance on minute-level trading tasks.

High-frequency trading Reinforcement learning Memory mechanism Context-aware Cryptocurrency markets

2024-06-20abs ↗pdf ↗