NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spaces

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Learning low-dimensional latent state space dynamics models has proven powerful for enabling vision-based planning and learning for control. We introduce a latent dynamics learning framework that is uniquely designed to induce proportional controlability in the latent space, thus enabling the use of simple and well-known PID controllers. We show that our learned dynamics model enables proportional control from pixels, dramatically simplifies and accelerates behavioural cloning of vision-based controllers, and provides interpretable goal discovery when applied to imitation learning of switching controllers from demonstration. Notably, such proportional controlability also allows for robust path following from visual demonstrations using Dynamic Movement Primitives in the learned latent space.
Original languageEnglish
Title of host publication2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages4452-4461
Number of pages10
ISBN (Electronic)978-1-6654-4509-2
ISBN (Print)978-1-6654-4510-8
DOIs
Publication statusPublished - 13 Nov 2021
EventIEEE Conference on Computer Vision and Pattern Recognition 2021 - Virtual
Duration: 19 Jun 202125 Jun 2021
http://cvpr2021.thecvf.com/

Publication series

Name
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

ConferenceIEEE Conference on Computer Vision and Pattern Recognition 2021
Abbreviated titleCVPR 2021
Period19/06/2125/06/21
Internet address

Fingerprint

Dive into the research topics of 'NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spaces'. Together they form a unique fingerprint.

Cite this