---
publication_id: "OT-2026-000001734"
title: "Data-efficient hierarchical reinforcement learning"
published_at: "2026-09-15T19:08:03.434986Z"
canonical_url: "https://opentechnical.org/p/OT-2026-000001734"
categories:
  - "robotics-automation/robot-architectures"
keywords:
  - "b25j9/1602"
  - "b25j9/163"
  - "b25j9/163"
  - "g06n3/092"
  - "b25j9/16"
  - "b25j9/16"
---

# Data-efficient hierarchical reinforcement learning

## Abstract

Training and/or utilizing a hierarchical reinforcement learning (HRL) model for robotic control. The HRL model can include at least a higher-level policy model and a lower-level policy model. Some implementations relate to technique(s) that enable more efficient off-policy training to be utilized in training of the higher-level policy model and/or the lower-level policy model. Some of those implementations utilize off-policy correction, which re-labels higher-level actions of experience data, generated in the past utilizing a previously trained version of the HRL model, with modified higher-level actions. The modified higher-level actions are then utilized to off-policy train the higher-level policy model. This can enable effective off-policy training despite the lower-level policy model being a different version at training time (relative to the version when the experience data was collected).