OPENTECHNICAL PUBLICATION
OT-2026-000000650

Using affordance plans for robot control

Published: 2026-09-15T19:03:12.485467Z
Permanent URL: https://opentechnical.org/p/OT-2026-000000650
Categories: Robotics & Automation > Robot Architectures

------------------------------------------------------------------------

ABSTRACT

Implementations for robot control are provided. A method involves, based on
vision data depicting an environment of a robot and a natural language
instruction for the robot, determining an affordance plan for performing a
task. The affordance plan comprises a sequence of intermediate representations
of the robot in visual space, such as end effector poses. An action input
prompt is assembled with data indicative of the vision data, the natural
language instruction, and the affordance plan. The action input prompt is
processed using one or more generative models to generate action output
indicative of one or more actions to be performed by the robot. Subsequently,
a robot control signal is generated based on the one or more actions. This
provides a spatially precise and dimensionally concise form of guidance for
robot manipulation tasks, which can improve performance and generalization.

------------------------------------------------------------------------

OpenTechnical publication OT-2026-000000650
Published 2026-09-15T19:03:12.485467Z
https://opentechnical.org/p/OT-2026-000000650