Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn

Ondrej Bohdal; Yinbing Tian; Yongshuo Zong; Ruchika Chavhan; Da Li; Henry Gouk; Li Guo; Timothy Hospedales

doi:10.1109/CVPR52729.2023.00743

Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn

Ondrej Bohdal, Yinbing Tian, Yongshuo Zong, Ruchika Chavhan, Da Li, Henry Gouk, Li Guo, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Meta-learning and other approaches to few-shot learning are widely studied for image recognition, and are increasingly applied to other vision tasks such as pose estimation and dense prediction. This naturally raises the question of whether there is any few-shot meta-learning algorithm capable of generalizing across these diverse task types? To support the community in answering this question, we introduce Meta Omnium, a dataset-of-datasets spanning multiple vision tasks including recognition, keypoint localization, semantic segmentation and regression. We experiment with popular few-shot meta-learning baselines and analyze their ability to generalize across tasks and to transfer knowledge between them. Meta Omnium enables meta-learning researchers to evaluate model generalization to a much wider array of tasks than previously possible, and provides a single framework for evaluating meta-learners across a wide suite of vision applications in a consistent manner.

Original language	English
Title of host publication	2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher	Institute of Electrical and Electronics Engineers
Pages	7693-7703
Number of pages	11
ISBN (Electronic)	9798350301298
ISBN (Print)	9798350301304
DOIs	https://doi.org/10.1109/CVPR52729.2023.00743
Publication status	Published - 22 Aug 2023
Event	The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 - Vancouver Convention Center, Vancouver, Canada Duration: 18 Jun 2023 → 22 Jun 2023 https://cvpr2023.thecvf.com/

Publication series

Name	Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher	IEEE
ISSN (Print)	1063-6919
ISSN (Electronic)	2575-7075

Conference

Conference	The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023
Abbreviated title	CVPR 2023
Country/Territory	Canada
City	Vancouver
Period	18/06/23 → 22/06/23
Internet address	https://cvpr2023.thecvf.com/

Access to Document

10.1109/CVPR52729.2023.00743Licence: All Rights Reserved

Meta Omnium_BOHDAL_DOA27022023_AFV_CC-BYAccepted author manuscript, 616 KBLicence: Creative Commons: Attribution (CC-BY)

https://ieeexplore.ieee.org/document/10205467Licence: All Rights Reserved

Signal Processing in the Information Age
Davies, M. (Principal Investigator), Hopgood, J. (Co-investigator), Hospedales, T. (Co-investigator), Mulgrew, B. (Co-investigator), Thompson, J. (Co-investigator), Tsaftaris, S. (Co-investigator) & Yaghoobi Vaighan, M. (Co-investigator)
EPSRC
1/07/18 → 31/03/24
Project: Research

Cite this

Bohdal, O., Tian, Y., Zong, Y., Chavhan, R., Li, D., Gouk, H., Guo, L., & Hospedales, T. (2023). Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 7693-7703). (Conference on Computer Vision and Pattern Recognition (CVPR)). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CVPR52729.2023.00743

@inproceedings{7a27f66c522b4ea298353a91dcb06b70,

title = "Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn",

abstract = "Meta-learning and other approaches to few-shot learning are widely studied for image recognition, and are increasingly applied to other vision tasks such as pose estimation and dense prediction. This naturally raises the question of whether there is any few-shot meta-learning algorithm capable of generalizing across these diverse task types? To support the community in answering this question, we introduce Meta Omnium, a dataset-of-datasets spanning multiple vision tasks including recognition, keypoint localization, semantic segmentation and regression. We experiment with popular few-shot meta-learning baselines and analyze their ability to generalize across tasks and to transfer knowledge between them. Meta Omnium enables meta-learning researchers to evaluate model generalization to a much wider array of tasks than previously possible, and provides a single framework for evaluating meta-learners across a wide suite of vision applications in a consistent manner.",

author = "Ondrej Bohdal and Yinbing Tian and Yongshuo Zong and Ruchika Chavhan and Da Li and Henry Gouk and Li Guo and Timothy Hospedales",

note = "This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) grant number EP/S000631/1; the MOD University Defence Research Collaboration (UDRC) in Signal Processing; EPSRC Centre for Doctoral Training in Data Science, funded by the UK Engineering and Physical Sciences Research Council (grant EP/L016427/1) and the University of Edinburgh; the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics, and Samsung AI Center, Cambridge. This project was supported by the Royal Academy of Engineering under the Research Fellowship programme. This work was also supported by Key Project Plan of Blockchain in Ministry of Education of the People{\textquoteright}s Republic of China (Grant No. 2020KJ010802), Innovation and Transformation Fund of Peking University Third Hospital (Grant No. BYSYZHKC2021115) and China Scholarship Council. Yongshuo Zong is supported by the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics. For the purpose of open access, the author has applied a creative commons attribution (CC BY) licence to any author accepted manuscript version arising.; The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023, CVPR 2023 ; Conference date: 18-06-2023 Through 22-06-2023",

year = "2023",

month = aug,

day = "22",

doi = "10.1109/CVPR52729.2023.00743",

language = "English",

isbn = "9798350301304",

series = "Conference on Computer Vision and Pattern Recognition (CVPR)",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "7693--7703",

booktitle = "2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",

address = "United States",

url = "https://cvpr2023.thecvf.com/",

}

Bohdal, O, Tian, Y, Zong, Y, Chavhan, R, Li, D, Gouk, H, Guo, L & Hospedales, T 2023, Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn. in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Conference on Computer Vision and Pattern Recognition (CVPR), Institute of Electrical and Electronics Engineers, pp. 7693-7703, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023, Vancouver, Canada, 18/06/23. https://doi.org/10.1109/CVPR52729.2023.00743

Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn. / Bohdal, Ondrej; Tian, Yinbing; Zong, Yongshuo et al.
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Institute of Electrical and Electronics Engineers, 2023. p. 7693-7703 (Conference on Computer Vision and Pattern Recognition (CVPR)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn

AU - Bohdal, Ondrej

AU - Tian, Yinbing

AU - Zong, Yongshuo

AU - Chavhan, Ruchika

AU - Li, Da

AU - Gouk, Henry

AU - Guo, Li

AU - Hospedales, Timothy

N1 - This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) grant number EP/S000631/1; the MOD University Defence Research Collaboration (UDRC) in Signal Processing; EPSRC Centre for Doctoral Training in Data Science, funded by the UK Engineering and Physical Sciences Research Council (grant EP/L016427/1) and the University of Edinburgh; the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics, and Samsung AI Center, Cambridge. This project was supported by the Royal Academy of Engineering under the Research Fellowship programme. This work was also supported by Key Project Plan of Blockchain in Ministry of Education of the People’s Republic of China (Grant No. 2020KJ010802), Innovation and Transformation Fund of Peking University Third Hospital (Grant No. BYSYZHKC2021115) and China Scholarship Council. Yongshuo Zong is supported by the United Kingdom Research and Innovation (grant EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the University of Edinburgh, School of Informatics. For the purpose of open access, the author has applied a creative commons attribution (CC BY) licence to any author accepted manuscript version arising.

PY - 2023/8/22

Y1 - 2023/8/22

N2 - Meta-learning and other approaches to few-shot learning are widely studied for image recognition, and are increasingly applied to other vision tasks such as pose estimation and dense prediction. This naturally raises the question of whether there is any few-shot meta-learning algorithm capable of generalizing across these diverse task types? To support the community in answering this question, we introduce Meta Omnium, a dataset-of-datasets spanning multiple vision tasks including recognition, keypoint localization, semantic segmentation and regression. We experiment with popular few-shot meta-learning baselines and analyze their ability to generalize across tasks and to transfer knowledge between them. Meta Omnium enables meta-learning researchers to evaluate model generalization to a much wider array of tasks than previously possible, and provides a single framework for evaluating meta-learners across a wide suite of vision applications in a consistent manner.

AB - Meta-learning and other approaches to few-shot learning are widely studied for image recognition, and are increasingly applied to other vision tasks such as pose estimation and dense prediction. This naturally raises the question of whether there is any few-shot meta-learning algorithm capable of generalizing across these diverse task types? To support the community in answering this question, we introduce Meta Omnium, a dataset-of-datasets spanning multiple vision tasks including recognition, keypoint localization, semantic segmentation and regression. We experiment with popular few-shot meta-learning baselines and analyze their ability to generalize across tasks and to transfer knowledge between them. Meta Omnium enables meta-learning researchers to evaluate model generalization to a much wider array of tasks than previously possible, and provides a single framework for evaluating meta-learners across a wide suite of vision applications in a consistent manner.

U2 - 10.1109/CVPR52729.2023.00743

DO - 10.1109/CVPR52729.2023.00743

M3 - Conference contribution

SN - 9798350301304

T3 - Conference on Computer Vision and Pattern Recognition (CVPR)

SP - 7693

EP - 7703

BT - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

PB - Institute of Electrical and Electronics Engineers

T2 - The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023

Y2 - 18 June 2023 through 22 June 2023

ER -

Meta Omnium: A Benchmark for General-Purpose Learning-to-Learn

Abstract

Publication series

Conference

Access to Document

Fingerprint

Projects

Signal Processing in the Information Age

Cite this