End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

Jindřich Libovický; Jindřich Helcl

doi:10.18653/v1/D18-1336

End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

Jindřich Libovický, Jindřich Helcl

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.

Original language	English
Title of host publication	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Place of Publication	Brussels, Belgium
Publisher	Association for Computational Linguistics
Pages	3016-3021
Number of pages	6
ISBN (Electronic)	978-1-948087-84-1
DOIs	https://doi.org/10.18653/v1/D18-1336
Publication status	Published - 1 Oct 2018
Event	2018 Conference on Empirical Methods in Natural Language Processing - Square Meeting Center, Brussels, Belgium Duration: 31 Oct 2018 → 4 Nov 2018 http://emnlp2018.org/

Conference

Conference	2018 Conference on Empirical Methods in Natural Language Processing
Abbreviated title	EMNLP 2018
Country/Territory	Belgium
City	Brussels
Period	31/10/18 → 4/11/18
Internet address	http://emnlp2018.org/

Access to Document

10.18653/v1/D18-1336Licence: Creative Commons: Attribution (CC-BY)

End-to-End_LIBOVICKY_DOA10082018_VOR_CC-BYFinal published version, 425 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/D18-1336Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{f97950ff5ae7499ca2f878479360b02b,

title = "End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification",

abstract = "Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.",

author = "Jind{\v r}ich Libovick{\'y} and Jind{\v r}ich Helcl",

year = "2018",

month = oct,

day = "1",

doi = "10.18653/v1/D18-1336",

language = "English",

pages = "3016--3021",

booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",

publisher = "Association for Computational Linguistics",

note = "2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 ; Conference date: 31-10-2018 Through 04-11-2018",

url = "http://emnlp2018.org/",

}

Libovický, J & Helcl, J 2018, End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification. in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, pp. 3016-3021, 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31/10/18. https://doi.org/10.18653/v1/D18-1336

End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification. / Libovický, Jindřich; Helcl, Jindřich.
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, 2018. p. 3016-3021.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

AU - Libovický, Jindřich

AU - Helcl, Jindřich

PY - 2018/10/1

Y1 - 2018/10/1

N2 - Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.

AB - Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.

U2 - 10.18653/v1/D18-1336

DO - 10.18653/v1/D18-1336

M3 - Conference contribution

SP - 3016

EP - 3021

BT - Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

PB - Association for Computational Linguistics

CY - Brussels, Belgium

T2 - 2018 Conference on Empirical Methods in Natural Language Processing

Y2 - 31 October 2018 through 4 November 2018

ER -

End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification

Abstract

Conference

Access to Document

Fingerprint

Cite this