Abstract
A robotic manipulator includes a sensing device with a tactile interface featuring visual markers and a camera for capturing images of these markers. The system includes one or more processors and memories storing instructions which, when executed, direct the tactile interface to contact a target surface, deforming the visual markers. The camera captures visual feedback of this deformation, which is input into a machine learning algorithm. Based on the algorithm's output, the system controls the operation of a manipulator device. This configuration enables adaptive manipulation by interpreting tactile and visual data through machine learning.