Abstract
A navigation device and method based on multimodal information are provided. The navigation device includes a storage device configured to store a plurality of waypoint images and a plurality of first movement reference instructions previously generated by using a database, a memory and a processor. The navigation method includes a step of receiving an observed image, a step of extracting a goal image from among the plurality of waypoint images stored in the database and extracting a second movement reference instruction applied to autonomous driving of a robot from among the plurality of first movement reference instructions stored in the database, based on the observed image, and a step of inputting the observed image, the goal image, and the second movement reference instruction to a pre-trained autonomous driving path generation model to generate an autonomous driving path and motion to be applied to the robot.