Now You See Me: Deep Face Hallucination for Unviewed Sketches

Conghui Hu, Da Li, Yi-Zhe Song, Timothy M. Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Face hallucination has been well studied in the last decade because of its useful applications in law enforcement and entertainment. Promising results on the problem of sketch-photo face hallucination have been achieved with classic, and increasingly deep learning-based methods. However, synthesized photos still lack the crisp fidelity of real photos. More importantly, good results have primarily been demonstrated on very constrained datasets where the style variability is very low, and crucially the sketches are perfectly align-able traces of the ground-truth photos. However, realistic applications in entertainment or law enforcement require working with more unconstrained sketches drawn from memory or description, which are not rigidly align-able. In this paper, we develop a new deep learning approach to address these settings. Our image-image regression network is trained with a combination of content and adversarial losses to generate crisp photorealistic images, and it contains an integrated spatial transformer network to deal with non-rigid alignment between the domains. We evaluate face synthesis on classic constrained, as well as unviewed, benchmarks namely CUHK, MGDB, and FSMD. The results qualitatively and quantitatively outperform existing approaches.
Original languageEnglish
Title of host publicationThe British Machine Vision Conference (BMVC 2017)
Number of pages11
ISBN (Electronic)1-901725-60-X
Publication statusE-pub ahead of print - 7 Sep 2017
EventThe 28th British Machine Vision Conference - Imperial College London, London, United Kingdom
Duration: 4 Sep 20177 Sep 2017


ConferenceThe 28th British Machine Vision Conference
Abbreviated titleBMVC 2017
Country/TerritoryUnited Kingdom
Internet address


Dive into the research topics of 'Now You See Me: Deep Face Hallucination for Unviewed Sketches'. Together they form a unique fingerprint.

Cite this