Intra-category sketch-based image retrieval by matching deformable part models

Yi Li, Timothy M. Hospedales, Yi-Zhe Song, Shaogang Gong

Research output: Chapter in Book/Report/Conference proceedingConference contribution


An important characteristic of sketches, compared with text, rests with their ability of intrinsically capturing structure and appearance detail of objects. Nonetheless, akin to traditional text-based image retrieval, conventional sketch-based image retrieval (SBIR) principally focuses on retrieving photos of the same category, neglecting the fine-grained characteristics of sketches. In this paper, we further advocate the expressiveness of sketches and examine their efficacy under a novel intra-category SBIR framework. In particular, we study how sketches can be adopted to permit pose-specific retrieval within object categories. Key challenge to this problem is introducing a mid-level sketch representation that not only captures object pose, but also possess the ability to traverse sketch and photo domains. More specifically, we learn deformable part-based model (DPM) as a mid-level representation to discover and encode the various poses and parts in sketch and image domains independently, after which graph matching is utilized to establish component and part-level correspondences across the two domains. We further propose an SBIR dataset that covers the unique aspects of fine-grained SBIR. Through in-depth experiments, we demonstrate the superior performance of our proposed SBIR framework, and showcase its unique ability in pose-specific retrieval.
Original languageEnglish
Title of host publicationBritish Machine Vision Conference, BMVC 2014, Nottingham, UK, September 1-5, 2014
PublisherBMVA Press
Number of pages12
Publication statusPublished - 2014


Dive into the research topics of 'Intra-category sketch-based image retrieval by matching deformable part models'. Together they form a unique fingerprint.

Cite this