An Erudite Fine-Grained Visual Classification Model

Dongliang Chang, Yujun Tong, Ruoyi Du, Timothy M Hospedales, Yi-Zhe Song, Zhanyu Ma*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Current fine-grained visual classification (FGVC) models are isolated. In practice, we first need to identify the coarse-grained label of an object, then select the corresponding FGVC model for recognition. This hinders the application of FGVC algorithms in real-life scenarios. In this paper, we propose an erudite FGVC model jointly trained by several different datasets1 , which can efficiently and accurately predict an object’s fine-grained label across the combined label space. We found through a pilot study that positive and negative transfers co-occur when different datasets are mixed for training, i.e., the knowledge from other datasets is not always useful. Therefore, we first propose a feature disentanglement module and a feature re-fusion module to reduce negative transfer and boost positive transfer between different datasets. In detail, we reduce negative transfer by decoupling the deep features through many dataset-specific feature extractors. Subsequently, these are channel-wise re-fused to facilitate positive transfer. Finally, we propose a meta-learning based dataset-agnostic spatial attention layer to take full advantage of the multi-dataset training data, given that localisation is dataset-agnostic between different datasets. Experimental results across 11 different mixed-datasets built on four different FGVC datasets demonstrate the effectiveness of the proposed method. Furthermore, the proposed method can be easily combined with existing FGVC methods to obtain state-of-the-art results. Our code is available at
Original languageEnglish
Title of host publication2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Number of pages10
ISBN (Electronic)9798350301298
ISBN (Print)9798350301304
Publication statusPublished - 22 Aug 2023
EventThe IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 - Vancouver Convention Center, Vancouver, Canada
Duration: 18 Jun 202322 Jun 2023

Publication series

NameIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075


ConferenceThe IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023
Abbreviated titleCVPR 2023
Internet address


Dive into the research topics of 'An Erudite Fine-Grained Visual Classification Model'. Together they form a unique fingerprint.

Cite this