Overcoming Occlusion with Inverse Graphics

Pol Moreno, Christopher K. I. Williams, Charlie Nash, Pushmeet Kohli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Scene understanding tasks such as the prediction of object pose, shape, appearance and illumination are hampered by the occlusions often found in images. We propose a vision-as-inverse-graphics approach to handle these occlusions by making use of a graphics renderer in combination with a robust generative model (GM). Since searching over scene factors to obtain the best match for an image is very inefficient, we make use of a recognition model (RM) trained on synthetic data to initialize the search. This paper addresses two issues: (i) We study how the inferences are affected by the degree of occlusion of the foreground object, and show that a robust GM which includes an outlier model to account for occlusions works significantly better than a non-robust model. (ii) We characterize the performance of the RM and the gains that can be made by refining the search using the GM, using a new dataset that includes background clutter and occlusions. We find that pose and shape are predicted very well by the RM, but appearance and especially illumination less so. However, accuracy on these latter two factors can be clearly improved with the generative model.
Original languageEnglish
Title of host publicationComputer Vision
Subtitle of host publicationECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III
EditorsGang Hua, Hervé Jégou
Place of PublicationCham
PublisherSpringer International Publishing
Number of pages16
ISBN (Electronic)978-3-319-49409-8
ISBN (Print)978-3-319-49408-1
Publication statusPublished - 16 Nov 2016
EventEuropean Conference on Computer Vision 2016 Workshops - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science (LNCS)
PublisherSpringer International Publishing
ISSN (Print)0302-9743


ConferenceEuropean Conference on Computer Vision 2016 Workshops
Abbreviated titleECCV 2016
Internet address


Dive into the research topics of 'Overcoming Occlusion with Inverse Graphics'. Together they form a unique fingerprint.

Cite this