Edinburgh Research Explorer

Generalising Fine-Grained Sketch-Based Image Retrieval

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Related Edinburgh Organisations

Open Access permissions



Original languageEnglish
Title of host publication2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Number of pages10
Publication statusAccepted/In press - 11 Mar 2019
Event2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Long Beach, United States
Duration: 16 Jun 201920 Jun 2019


Conference2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2019
CountryUnited States
CityLong Beach
Internet address


Fine-grained sketch-based image retrieval (FG-SBIR) addresses matching specific photo instance using free-handsketch as a query modality. Existing models aim to learnan embedding space in which sketch and photo can be directly compared. While successful, they require instance-level pairing within each coarse-grained category as annotated training data. Since the learned embedding space is domain-specific, these models do not generalise well across categories. This limits the practical applicability of FGSBIR. In this paper, we identify cross-category generalisation for FG-SBIR as a domain generalisation problem, and propose the first solution. Our key contribution is a novel unsupervised learning approach to model a universal manifold of prototypical visual sketch traits. This manifold can then be used to paramaterise the
learning of a sketch/photo representation. Model adaptation to novel categories then becomes automatic via embedding the novel sketch in the manifold and updating the representation and retrieval function accordingly. Experiments on the two largest FG-SBIR datasets, Sketchy and QMUL-Shoe-V2, demonstrate the efficacy of our approach in enabling crosscategory generalisation of FG-SBIR.


2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition


Long Beach, United States

Event: Conference

Download statistics

No data available

ID: 82435838