---
publication_id: "OT-2026-000001396"
title: "Training and/or utilizing machine learning model(s) for use in natural language based robotic control"
published_at: "2026-09-15T19:05:34.938258Z"
canonical_url: "https://opentechnical.org/p/OT-2026-000001396"
categories:
  - "robotics-automation/robot-architectures"
keywords:
  - "b25j9/1656"
  - "b25j9/1656"
  - "b25j9/1664"
  - "b25j9/1664"
  - "b25j9/16"
  - "b25j9/163"
---

# Training and/or utilizing machine learning model(s) for use in natural language based robotic control

## Abstract

Techniques are disclosed that enable training a goal-conditioned policy based on multiple data sets, where each of the data sets describes a robot task in a different way. For example, the multiple data sets can include: a goal image data set, where the task is captured in the goal image; a natural language instruction data set, where the task is described in the natural language instruction; a task ID data set, where the task is described by the task ID, etc. In various implementations, each of the multiple data sets has a corresponding encoder, where the encoders are trained to generate a shared latent space representation of the corresponding task description. Additional or alternative techniques are disclosed that enable control of a robot using a goal-conditioned policy network. For example, the robot can be controlled, using the goal-conditioned policy network, based on free-form natural language input describing robot task(s).