Edinburgh Research Explorer

Generalising Fine-Grained Sketch-Based Image Retrieval

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Related Edinburgh Organisations

Open Access permissions

Open

Documents

https://ieeexplore.ieee.org/document/8953777
Original languageEnglish
Title of host publication2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages677-686
Number of pages10
ISBN (Electronic)978-1-7281-3293-8
ISBN (Print)978-1-7281-3294-5
DOIs
Publication statusPublished - 9 Jan 2020
Event2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Long Beach, United States
Duration: 16 Jun 201920 Jun 2019
http://cvpr2019.thecvf.com/

Publication series

Name
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

Conference2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2019
CountryUnited States
CityLong Beach
Period16/06/1920/06/19
Internet address

Abstract

Fine-grained sketch-based image retrieval (FG-SBIR) addresses matching specific photo instance using free-handsketch as a query modality. Existing models aim to learnan embedding space in which sketch and photo can be directly compared. While successful, they require instance-level pairing within each coarse-grained category as annotated training data. Since the learned embedding space is domain-specific, these models do not generalise well across categories. This limits the practical applicability of FGSBIR. In this paper, we identify cross-category generalisation for FG-SBIR as a domain generalisation problem, and propose the first solution. Our key contribution is a novel unsupervised learning approach to model a universal manifold of prototypical visual sketch traits. This manifold can then be used to paramaterise the learning of a sketch/photo representation. Model adaptation to novel categories then becomes automatic via embedding the novel sketch in the manifold and updating the representation and retrieval function accordingly. Experiments on the two largest FG-SBIR datasets, Sketchy and QMUL-Shoe-V2, demonstrate the efficacy of our approach in enabling crosscategory generalisation of FG-SBIR.

Event

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition

16/06/1920/06/19

Long Beach, United States

Event: Conference

Download statistics

No data available

ID: 82435838