Object and Relation Centric Representations for Push Effect Prediction

A. E. Tekden¹ A. Erdem² E. Erdem³ T. Asfour⁴ E. Ugur¹

¹Boun CoLoRs ²KUIS AI ³HUCVL ⁴(H²T)

Abstract

Pushing is an essential non-prehensile manipulation skill used for tasks ranging from pre-grasp manipulation to scene rearrangement, and thus pushing action has been widely studied in robotics. The effective use of pushing actions often requires an understanding of the dynamics of the manipulated objects and adaptation to the discrepancies between prediction and reality. For this reason, effect prediction and parameter estimation with pushing actions have been heavily investigated in the literature. However, current approaches are limited because they either model systems with a fixed number of objects or use image-based representations whose outputs are not very interpretable and quickly accumulate errors. In this study, we propose a graph neural network based framework for effect prediction and parameter estimation in push manipulation tasks by modeling objects and their relations, which are in the form of contact or articulation. Our framework is validated both in real and simulated environments containing different shaped multi-part objects connected via different types of joints and objects with different masses. Our approach enables the robot to predict and adapt the effect of its pushing actions as it observes the scene. Furthermore, we show that we can model novel tools that are not encountered during training and utilize them for planning in tool manipulation tasks; and we can make 6-D effect prediction in lever-up tasks inspired by hard-disk drive disassembly.

Paper

A. E. Tekden, A. Erdem, E. Erdem, T. Asfour and E. Ugur
Object and Relation Centric Representations for Push Effect Prediction

Supplementary Video

Code and Dataset

Code and dataset is available at:
https://github.com/Fzaero/Object-and-Relation-Centric-Representations-for-Push-Effect-Prediction