Unsupervised Learning of Object Landmarks through Conditional Image Generation

Tomas Jakab, Ankush Gupta, Hakan Bilen, Andrea Vedaldi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We propose a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision. We cast this as the problem of generating images that combine the appearance of the object as seen in a first example image with the geometry of the object as seen in a second example image, where the two examples differ by a viewpoint change and/or an object deformation. In order to factorize appearance and geometry, we introduce a tight bottleneck in the geometry-extraction process that selects and distills geometry-related features. Compared to standard image generation problems, which often use generative adversarial networks, our generation task is conditioned on both appearance and geometry and thus is significantly less ambiguous, to the point that adopting a simple perceptual loss formulation is sufficient. We demonstrate that our approach can learn object landmarks from synthetic image deformations or videos, all without manual supervision, while outperforming state-of-the-art unsupervised landmark detectors. We further show that our method is applicable to a large variety of datasets — faces, people, 3D objects, and
digits — without any modifications.
Original languageEnglish
Title of host publicationProceedings of the 32nd Conference on Neural Information Processing Systems (NIPS 2018)
Place of PublicationPalais des Congrès de Montréal, Montréal CANADA
Pages1-12
Number of pages12
Publication statusPublished - 2018
EventThirty-second Conference on Neural Information Processing Systems - Montreal, Canada
Duration: 3 Dec 20188 Dec 2018
https://nips.cc/

Conference

ConferenceThirty-second Conference on Neural Information Processing Systems
Abbreviated titleNIPS 2018
Country/TerritoryCanada
CityMontreal
Period3/12/188/12/18
Internet address

Fingerprint

Dive into the research topics of 'Unsupervised Learning of Object Landmarks through Conditional Image Generation'. Together they form a unique fingerprint.

Cite this