Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels

Massimiliano Patacchiola, Jack Turner, Elliot J Crowley, Michael F P O'Boyle, Amos J Storkey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 33
PublisherCurran Associates Inc
Pages16108-16118
Number of pages11
Volume33
Publication statusPublished - 7 Dec 2020
EventThirty-Fourth Conference on Neural Information Processing Systems - Virtual Conference
Duration: 6 Dec 202012 Dec 2020
https://nips.cc/Conferences/2020

Conference

ConferenceThirty-Fourth Conference on Neural Information Processing Systems
Abbreviated titleNeurIPS 2020
CityVirtual Conference
Period6/12/2012/12/20
Internet address

Fingerprint

Dive into the research topics of 'Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels'. Together they form a unique fingerprint.

Cite this