Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

Ulrich Germann; Roman Grundkiewicz; Martin Popel; Radina Dobreva; Nikolay Bogoychev; Kenneth Heafield

Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

Ulrich Germann, Roman Grundkiewicz, Martin Popel, Radina Dobreva, Nikolay Bogoychev, Kenneth Heafield

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We describe the joint submission of the University of Edinburgh and Charles University, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact student models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech â†” English test sets, they achieve translation speeds of over 700 whitespace-delimited source words per second on a single CPU thread, thus making neural translation feasible on consumer hardware without a GPU.

Original language	English
Title of host publication	Proceedings of the Fifth Conference on Machine Translation
Place of Publication	Online
Publisher	Association for Computational Linguistics
Pages	191-196
Number of pages	6
ISBN (Electronic)	978-1-948087-81-0
Publication status	Published - 19 Nov 2020

Access to Document

Speed-optimized_GERMANN_DOA29092020_VOR_CC-BYFinal published version, 364 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/2020.wmt-1.17Licence: Creative Commons: Attribution (CC-BY)

Fingerprint Dive into the research topics of 'Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task'. Together they form a unique fingerprint.

2 Active

ELG- European Language Grid
Germann, U. & Renals, S.
EU government bodies
1/01/19 → 30/06/22
Project: Research
Browser-based Multilingual Translation
Heafield, K.
EU government bodies
1/01/19 → 31/03/22
Project: Research

Cite this

Germann, U., Grundkiewicz, R., Popel, M., Dobreva, R., Bogoychev, N., & Heafield, K. (2020). Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task. In Proceedings of the Fifth Conference on Machine Translation (pp. 191-196). Association for Computational Linguistics. https://www.aclweb.org/anthology/2020.wmt-1.17

Germann, Ulrich ; Grundkiewicz, Roman ; Popel, Martin ; Dobreva, Radina ; Bogoychev, Nikolay ; Heafield, Kenneth. / Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task. Proceedings of the Fifth Conference on Machine Translation. Online : Association for Computational Linguistics, 2020. pp. 191-196

@inproceedings{d73867d0a0e441db9ca86325dfe5036e,

title = "Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task",

abstract = "We describe the joint submission of the University of Edinburgh and Charles University, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact student models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech {\^a}†” English test sets, they achieve translation speeds of over 700 whitespace-delimited source words per second on a single CPU thread, thus making neural translation feasible on consumer hardware without a GPU.",

author = "Ulrich Germann and Roman Grundkiewicz and Martin Popel and Radina Dobreva and Nikolay Bogoychev and Kenneth Heafield",

year = "2020",

month = nov,

day = "19",

language = "English",

pages = "191--196",

booktitle = "Proceedings of the Fifth Conference on Machine Translation",

publisher = "Association for Computational Linguistics",

}

Germann, U , Grundkiewicz, R, Popel, M, Dobreva, R, Bogoychev, N & Heafield, K 2020, Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task. in Proceedings of the Fifth Conference on Machine Translation. Association for Computational Linguistics, Online, pp. 191-196. <https://www.aclweb.org/anthology/2020.wmt-1.17>

Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task. / Germann, Ulrich ; Grundkiewicz, Roman; Popel, Martin; Dobreva, Radina; Bogoychev, Nikolay ; Heafield, Kenneth.

Proceedings of the Fifth Conference on Machine Translation. Online : Association for Computational Linguistics, 2020. p. 191-196.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

AU - Germann, Ulrich

AU - Grundkiewicz, Roman

AU - Popel, Martin

AU - Dobreva, Radina

AU - Bogoychev, Nikolay

AU - Heafield, Kenneth

PY - 2020/11/19

Y1 - 2020/11/19

N2 - We describe the joint submission of the University of Edinburgh and Charles University, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact student models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech â†” English test sets, they achieve translation speeds of over 700 whitespace-delimited source words per second on a single CPU thread, thus making neural translation feasible on consumer hardware without a GPU.

AB - We describe the joint submission of the University of Edinburgh and Charles University, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact student models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech â†” English test sets, they achieve translation speeds of over 700 whitespace-delimited source words per second on a single CPU thread, thus making neural translation feasible on consumer hardware without a GPU.

M3 - Conference contribution

SP - 191

EP - 196

BT - Proceedings of the Fifth Conference on Machine Translation

PB - Association for Computational Linguistics

CY - Online

ER -

Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

Abstract

Access to Document

ELG- European Language Grid

Browser-based Multilingual Translation

Cite this