Learning Deep Sketch Abstraction

Umar Riaz Muhammad, Yongxin Yang, Yi-Zhe Song, Tao Xiang, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Human free-hand sketches have been studied in various contexts including sketch recognition, synthesis and finegrained sketch-based image retrieval (FG-SBIR). A fundamental challenge for sketch analysis is to deal with drastically different human drawing styles, particularly in terms of abstraction level. In this work, we propose the first stroke-level sketch abstraction model based on the insight of sketch abstraction as a process of trading off between the recognizability of a sketch and the number of strokes used to draw it. Concretely, we train a model for abstract sketch generation through reinforcement learning of a stroke removal policy that learns to predict which strokes can be safely removed without affecting recognizability. We show that our abstraction model can be used for various sketch analysis tasks including: (1) modeling stroke saliency and understanding the decision of sketch recognition models, (2) synthesizing sketches of variable abstraction for a given category, or reference object instance in a photo, and (3) training a FG-SBIR model with photos only, bypassing the expensive photo-sketch pair collection step.
Original languageEnglish
Title of host publication2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages8014-8023
Number of pages10
ISBN (Electronic)978-1-5386-6420-9
DOIs
Publication statusPublished - 17 Dec 2018
EventComputer Vision and Pattern Recognition 2018 - Salt Lake City, United States
Duration: 18 Jun 201822 Jun 2018
http://cvpr2018.thecvf.com/
http://cvpr2018.thecvf.com/
http://cvpr2018.thecvf.com/

Publication series

Name
ISSN (Electronic)2575-7075

Conference

ConferenceComputer Vision and Pattern Recognition 2018
Abbreviated titleCVPR 2018
Country/TerritoryUnited States
CitySalt Lake City
Period18/06/1822/06/18
Internet address

Fingerprint

Dive into the research topics of 'Learning Deep Sketch Abstraction'. Together they form a unique fingerprint.

Cite this