Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing

Russell Buchanan, Adrian Rofer, Joao Moura, Abhinav Valada, Sethu Vijayakumar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

From dishwashers to cabinets, humans interact with articulated objects every day, and for a robot to assist in common manipulation tasks, it must learn a representation of articulation. Recent deep learning learning methods can provide powerful vision-based priors on the affordance of articulated objects from previous, possibly simulated, experiences. In contrast, many works estimate articulation by observing the object in motion, requiring the robot to already be interacting with the object. In this work, we propose to use the best of both worlds by introducing an online estimation method that merges vision-based affordance predictions from a neural network with interactive kinematic sensing in an analytical model. Our work has the benefit of using vision to predict an articulation model before touching the object, while also being able to update the model quickly from kinematic sensing during the interaction. In this paper, we implement a full system using shared autonomy for robotic opening of articulated objects, in particular objects in which the articulation is not apparent from vision alone. We implemented our system on a real robot and performed several autonomous closed-loop experiments in which the robot had to open a door with unknown joint while estimating the articulation online. Our system achieved an 80% success rate for autonomous opening of unknown articulated objects.
Original languageEnglish
Title of host publication2024 IEEE International Conference on Robotics and Automation (ICRA)
PublisherIEEE
Pages1-7
Number of pages7
Publication statusAccepted/In press - 31 Jan 2024
Event2024 IEEE International Conference on Robotics and Automation - Yokohama, Japan
Duration: 13 May 202417 May 2024
https://2024.ieee-icra.org/

Conference

Conference2024 IEEE International Conference on Robotics and Automation
Abbreviated titleICRA 2024
Country/TerritoryJapan
CityYokohama
Period13/05/2417/05/24
Internet address

Fingerprint

Dive into the research topics of 'Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing'. Together they form a unique fingerprint.

Cite this