OPENTECHNICAL PUBLICATION
OT-2026-000002449

Semantic robot hazard avoidance with multi-modal prompting

Published: 2026-09-15T19:13:09.408581Z
Permanent URL: https://opentechnical.org/p/OT-2026-000002449
Categories: Robotics & Automation > Robot Architectures

------------------------------------------------------------------------

ABSTRACT

Systems and methods for semantic robot hazard avoidance with multi-modal
prompting are provided. In one aspect, a method includes receiving a user
input indicative of one or more hazards in an environment of the robot and
image data indicative of the one or more hazards in the environment. The
method also includes generating one or more segments of the image data. Each
of the one or more segments corresponds to at least one of the one or more
hazards indicated by the user input. The method further includes identifying a
semantic label for each of the one or more segments, generating a hazard map
including a location of each of the one or more segments and the corresponding
semantic label, and navigating the robot through the environment based at
least in part on the hazard map.

------------------------------------------------------------------------

OpenTechnical publication OT-2026-000002449
Published 2026-09-15T19:13:09.408581Z
https://opentechnical.org/p/OT-2026-000002449