Vision-based system identification and 3D keypoint discovery using dynamics constraints

Miguel Jaques, Martin Asenov, Michael Burke, Timothy M Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

This paper introduces V-SysId, a novel method that enables simultaneous keypoint discovery, 3D system identification, and extrinsic camera calibration from an unlabeled video taken from a static camera, using only the family of equations of motion of the object of interest as weak supervision. V-SysId takes keypoint trajectory proposals and alternates between maximum likelihood parameter estimation and extrinsic camera calibration, before applying a suitable selection criterion to identify the track of interest. This is then used to train a keypoint tracking model using supervised learning. Results on a range of settings (robotics, physics, physiology) highlight the utility of this approach.
Original languageEnglish
Title of host publicationProceedings of The 4th Annual Learning for Dynamics and Control Conference
Subtitle of host publicationVolume 168: Learning for Dynamics and Control Conference, 23-24 June 2022, Stanford University, Stanford, CA, USA
EditorsRoya Firoozi, Negar Mehr, Esen Yel, Rika Antonova, Jeannette Bohg, Mac Schwager, Mykel Kochendorfer
PublisherPMLR
Pages316-329
Number of pages14
Volume168
Publication statusPublished - 11 May 2022
Event4th Annual Learning for Dynamics & Control Conference
- Stanford, United States
Duration: 23 Jun 202224 Jun 2022
Conference number: 4
https://l4dc.stanford.edu/

Conference

Conference4th Annual Learning for Dynamics & Control Conference
Abbreviated titleL4DC 2022
Country/TerritoryUnited States
CityStanford
Period23/06/2224/06/22
Internet address

Fingerprint

Dive into the research topics of 'Vision-based system identification and 3D keypoint discovery using dynamics constraints'. Together they form a unique fingerprint.

Cite this