Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Xueting Zhang; Debin Meng; Henry Gouk; Timothy Hospedales

doi:10.1109/ICCV48922.2021.00069

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Xueting Zhang, Debin Meng, Henry Gouk, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Many state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple (e.g., nearest centroid) classifiers. We take an approach that is agnostic to the features used, and focus exclusively on meta-learning the final classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalisation of the classic quadratic discriminant analysis. This approach has several benefits of interest to practitioners: meta-learning is fast and memory efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen, and thus will continue to benefit from future advances in feature representations. Empirically, it leads to excellent performance in cross-domain few-shot learning, class-incremental few-shot learning, and crucially for real-world applications, the Bayesian formulation leads to state-of-the-art uncertainty calibration in predictions.

Original language	English
Title of host publication	Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021
Publisher	Institute of Electrical and Electronics Engineers
Pages	631-640
Number of pages	10
ISBN (Electronic)	978-1-6654-2812-5
ISBN (Print)	978-1-6654-2813-2
DOIs	https://doi.org/10.1109/ICCV48922.2021.00069
Publication status	Published - 28 Feb 2022
Event	International Conference on Computer Vision 2021 - Online Duration: 11 Oct 2021 → 17 Oct 2021 https://iccv2021.thecvf.com/home

Publication series

Name	2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher	IEEE
ISSN (Print)	1550-5499
ISSN (Electronic)	2380-7504

Conference

Conference	International Conference on Computer Vision 2021
Abbreviated title	ICCV 2021
Period	11/10/21 → 17/10/21
Internet address	https://iccv2021.thecvf.com/home

Access to Document

10.1109/ICCV48922.2021.00069Licence: All Rights Reserved

Shallow Bayesian Meta_ZHANG_DOA22072021_AFVAccepted author manuscript, 520 KBLicence: Creative Commons: Attribution (CC-BY)

https://ieeexplore.ieee.org/document/9710819Licence: All Rights Reserved

Signal Processing in the Information Age
Davies, M., Hopgood, J., Hospedales, T., Mulgrew, B., Thompson, J., Tsaftaris, S. & Yaghoobi Vaighan, M.
EPSRC
1/07/18 → 31/03/24
Project: Research
UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management (ORCA)
Vijayakumar, S., Mistry, M., Ramamoorthy, R. & Williams, C.
EPSRC
1/10/17 → 31/03/22
Project: Research

Cite this

Zhang, X., Meng, D., Gouk, H., & Hospedales, T. (2022). Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition. In Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021 (pp. 631-640). (2021 IEEE/CVF International Conference on Computer Vision (ICCV)). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICCV48922.2021.00069

@inproceedings{04c169767da941cb90f36be765794139,

title = "Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition",

abstract = "Many state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple (e.g., nearest centroid) classifiers. We take an approach that is agnostic to the features used, and focus exclusively on meta-learning the final classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalisation of the classic quadratic discriminant analysis. This approach has several benefits of interest to practitioners: meta-learning is fast and memory efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen, and thus will continue to benefit from future advances in feature representations. Empirically, it leads to excellent performance in cross-domain few-shot learning, class-incremental few-shot learning, and crucially for real-world applications, the Bayesian formulation leads to state-of-the-art uncertainty calibration in predictions.",

author = "Xueting Zhang and Debin Meng and Henry Gouk and Timothy Hospedales",

note = "Funding Information: We propose an efficient shallow meta-learner for few-shot learning. MetaQDA provides a fast exact inference strategy for amortized Bayesian meta-learning through conjugacy, and highlights a distinct avenue of meta-learning research in contrast to meta representation learning. The empirical performance of our model exceeds that of others that rely on off-the-shelf feature extractors, and often outperforms those that train extractors specialised for few-shot learning. In particular it excels in a number of challenging but highly practically important metrics including cross-domain few-shot learning, class incremental few-shot learning, and providing accurate probability calibration—a vital property for many applications where safety or reliability is of paramount concern. Acknowledgements This work was supported by the Engineering and Physical Sciences Research Council of the UK (EPSRC) Grant number EP/S000631/1 and the UK MOD University Defence Research Collaboration (UDRC) in Signal Processing, and EPSRC Grant EP/R026173/1. Publisher Copyright: {\textcopyright} 2021 IEEE; International Conference on Computer Vision 2021, ICCV 2021 ; Conference date: 11-10-2021 Through 17-10-2021",

year = "2022",

month = feb,

day = "28",

doi = "10.1109/ICCV48922.2021.00069",

language = "English",

isbn = "978-1-6654-2813-2",

series = "2021 IEEE/CVF International Conference on Computer Vision (ICCV)",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "631--640",

booktitle = "Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021",

address = "United States",

url = "https://iccv2021.thecvf.com/home",

}

Zhang, X, Meng, D, Gouk, H & Hospedales, T 2022, Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition. in Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021 . 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Institute of Electrical and Electronics Engineers, pp. 631-640, International Conference on Computer Vision 2021, 11/10/21. https://doi.org/10.1109/ICCV48922.2021.00069

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition. / Zhang, Xueting; Meng, Debin; Gouk, Henry et al.
Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021 . Institute of Electrical and Electronics Engineers, 2022. p. 631-640 (2021 IEEE/CVF International Conference on Computer Vision (ICCV)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

AU - Zhang, Xueting

AU - Meng, Debin

AU - Gouk, Henry

AU - Hospedales, Timothy

N1 - Funding Information: We propose an efficient shallow meta-learner for few-shot learning. MetaQDA provides a fast exact inference strategy for amortized Bayesian meta-learning through conjugacy, and highlights a distinct avenue of meta-learning research in contrast to meta representation learning. The empirical performance of our model exceeds that of others that rely on off-the-shelf feature extractors, and often outperforms those that train extractors specialised for few-shot learning. In particular it excels in a number of challenging but highly practically important metrics including cross-domain few-shot learning, class incremental few-shot learning, and providing accurate probability calibration—a vital property for many applications where safety or reliability is of paramount concern. Acknowledgements This work was supported by the Engineering and Physical Sciences Research Council of the UK (EPSRC) Grant number EP/S000631/1 and the UK MOD University Defence Research Collaboration (UDRC) in Signal Processing, and EPSRC Grant EP/R026173/1. Publisher Copyright: © 2021 IEEE

PY - 2022/2/28

Y1 - 2022/2/28

N2 - Many state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple (e.g., nearest centroid) classifiers. We take an approach that is agnostic to the features used, and focus exclusively on meta-learning the final classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalisation of the classic quadratic discriminant analysis. This approach has several benefits of interest to practitioners: meta-learning is fast and memory efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen, and thus will continue to benefit from future advances in feature representations. Empirically, it leads to excellent performance in cross-domain few-shot learning, class-incremental few-shot learning, and crucially for real-world applications, the Bayesian formulation leads to state-of-the-art uncertainty calibration in predictions.

AB - Many state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple (e.g., nearest centroid) classifiers. We take an approach that is agnostic to the features used, and focus exclusively on meta-learning the final classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalisation of the classic quadratic discriminant analysis. This approach has several benefits of interest to practitioners: meta-learning is fast and memory efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen, and thus will continue to benefit from future advances in feature representations. Empirically, it leads to excellent performance in cross-domain few-shot learning, class-incremental few-shot learning, and crucially for real-world applications, the Bayesian formulation leads to state-of-the-art uncertainty calibration in predictions.

U2 - 10.1109/ICCV48922.2021.00069

DO - 10.1109/ICCV48922.2021.00069

M3 - Conference contribution

SN - 978-1-6654-2813-2

T3 - 2021 IEEE/CVF International Conference on Computer Vision (ICCV)

SP - 631

EP - 640

BT - Proceedings of 2021 IEEE/CVF International Conference on Computer Vision ICCV 2021

PB - Institute of Electrical and Electronics Engineers

T2 - International Conference on Computer Vision 2021

Y2 - 11 October 2021 through 17 October 2021

ER -

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Abstract / Description of output

Publication series

Conference

Access to Document

Fingerprint

Projects

Signal Processing in the Information Age

UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management (ORCA)

Cite this