Abstract
A method may include receiving training data comprising a plurality of RGB-D images of an object at a plurality of time steps, and a plurality of robot actions associated with the object at the plurality of time steps; and optimizing, using the training data, a dynamics function to predict a future state of the object based on a current state of the object and a robot action. A state of the object is estimated as a plurality of particles comprising 3D Gaussians using particle filtering.