Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We describe the joint submission of the University of Edinburgh and Charles University, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact student models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech ↔ English test sets, they achieve translation speeds of over 700 whitespace-delimited source words per second on a single CPU thread, thus making neural translation feasible on consumer hardware without a GPU.
Original languageEnglish
Title of host publicationProceedings of the Fifth Conference on Machine Translation
Place of PublicationOnline
PublisherAssociation for Computational Linguistics
Pages191-196
Number of pages6
ISBN (Electronic)978-1-948087-81-0
Publication statusPublished - 19 Nov 2020

Fingerprint Dive into the research topics of 'Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task'. Together they form a unique fingerprint.

Cite this