Localization and Completion for 3D Object Interactions

Xi Zhao, Ruizhen Hu, Haisong Liu, Taku Komura, Xinyu Yang

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Finding where and what objects to put into an existing scene is a common task for scene synthesis and robot/character motion planning. Existing frameworks require development of hand-crafted features suitable for the task, or full volumetric analysis that could be memory intensive and imprecise. In this paper, we propose a data-driven framework to discover a suitable location and then place the appropriate objects in a scene. Our approach is inspired by computer vision techniques for localizing objects in images: using an all directional depth image (ADD-image) that encodes the 360-degree field of view from samples in the scene, our system regresses the images to the position where the new object can be located. Given several candidate areas around the host object in the scene, our system predicts the partner object whose geometry fit well to the host object. Our approach is highly parallel and memory efficient, and is especially suitable for handling interactions between large and small objects. We show examples where the system can hang bags on hooks, fit chairs in front of desks, put objects into shelves, insert flowers into vases, and put hangers onto laundry rack.

Original languageEnglish
Number of pages13
JournalIEEE Transactions on Visualization and Computer Graphics
Early online date14 Jan 2019
DOIs
Publication statusE-pub ahead of print - 14 Jan 2019

Fingerprint

Dive into the research topics of 'Localization and Completion for 3D Object Interactions'. Together they form a unique fingerprint.

Cite this