Deep Factorised Inverse-Sketching

Kaiyue Pang, Da Li, Jifei Song, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Modelling human free-hand sketches has become topical recently, driven by practical applications such as fine-grained sketch based image retrieval (FG-SBIR). Sketches are clearly related to photo edgemaps, but a human free-hand sketch of a photo is not simply a clean rendering of that photo’s edge map. Instead there is a fundamental process of abstraction and iconic rendering, where overall geometry is warped and salient details are selectively included. In this paper we study this sketching process and attempt to invert it. We model this inversion by translating iconic free-hand sketches to contours that resemble more geometrically realistic projections of object boundaries, and separately factorise out the salient added details. This factorised re-representation makes it easier to match a free-hand sketch to a photo instance of an object. Specifically, we propose a novel unsupervised image style transfer model based on enforcing a cyclic embedding consistency constraint. A deep FG-SBIR model is then formulated to accommodate complementary discriminative detail from each factorised sketch for better matching with the corresponding photo. Our method is evaluated both qualitatively and quantitatively to demonstrate its superiority over a number of state-of-the-art alternatives for style transfer and FG-SBIR.
Original languageEnglish
Title of host publicationEuropean Conference on Computer Vision (ECCV 2018)
Place of PublicationMunich, Germany
PublisherSpringer, Cham
Pages37-54
Number of pages18
ISBN (Electronic)978-3-030-01267-0
ISBN (Print)978-3-030-01266-3
DOIs
Publication statusPublished - 7 Oct 2018
EventEuropean Conference on Computer Vision 2018 - Munich, Germany
Duration: 8 Sep 201814 Sep 2018
https://eccv2018.org/

Publication series

Name Lecture Notes in Computer Science
PublisherSpringer, Cham
Volume11219
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349
NameImage Processing, Computer Vision, Pattern Recognition, and Graphics
Volume11219

Conference

ConferenceEuropean Conference on Computer Vision 2018
Abbreviated titleECCV 2018
CountryGermany
CityMunich
Period8/09/1814/09/18
Internet address

Fingerprint Dive into the research topics of 'Deep Factorised Inverse-Sketching'. Together they form a unique fingerprint.

Cite this