A Channel Coding Benchmark for Meta-Learning

Rui Li, Ondrej Bohdal, Rajesh K Mishra, Hyeji Kim, Da Li, Nicholas Lane, Timothy Hospedales

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Meta-learning provides a popular and effective family of methods for data-efficient learning of new tasks. However, several important issues in meta-learning have proven hard to study thus far. For example, performance degrades in real-world settings where meta-learners must learn from a wide and potentially multi-modal distribution of training tasks; and when distribution shift exists between meta-train and meta-test task distributions. These issues are typically hard to study since the shape of task distributions, and shift between them are not straightforward to measure or control in standard benchmarks. We propose the channel coding problem as a benchmark for meta-learning. Channel coding is an important practical application where task distributions naturally arise, and fast adaptation to new tasks is practically valuable. We use our MetaCC benchmark to study several aspects of meta-learning, including the impact of task distribution breadth and shift, which can be controlled in the coding problem. Going forward, MetaCC provides a tool for the community to study the capabilities and limitations of meta-learning, and to drive research on practically robust and effective meta-learners.
Original languageEnglish
Title of host publicationProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks
EditorsJ. Vanschoren, S. Yeung
PublisherCurran Associates Inc
Number of pages12
ISBN (Electronic)9781713871095
Publication statusPublished - 14 Dec 2021
Event35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online
Duration: 6 Dec 202114 Dec 2021

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258


Conference35th Conference on Neural Information Processing Systems, NeurIPS 2021
CityVirtual, Online


Dive into the research topics of 'A Channel Coding Benchmark for Meta-Learning'. Together they form a unique fingerprint.

Cite this