Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data

Anne Beyer; Vivien Macketanz; Aljoscha Burchardt; Philip Williams

Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data

Anne Beyer, Vivien Macketanz, Aljoscha Burchardt, Philip Williams

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In the last year, we have seen a lot of evidence about the superiority of neural
machine translation approaches (NMT) over phrase-based statistical approaches
(PBMT). This trend has shown for the general domain at public competitions such as the WMT challenges as well as in the obvious quality increase in online translation services that have changed their technology. In this paper, we take the perspective of an LSP. The questions we want to answer with this study is if now is already the time to invest in the new technology. To answer this question, we have collected evidence as to whether an existing state of-the-art NMT system for the general domain can already compete with a domain trained and optimised Moses (PBMT) system or if it is maybe already better. As it is well known that automatic quality measures are not reliable for comparing the performance of different system types, we have performed a detailed manual evaluation based on a test suite of domain segments.

Original language	English
Title of host publication	EAMT 2017: The 20th Annual Conference of the European Association for Machine Translation
Number of pages	6
Publication status	Published - 31 May 2017
Event	20th Annual Conference of the European Association for Machine Translation - Prague, Czech Republic Duration: 29 May 2017 → 31 May 2017 https://ufal.mff.cuni.cz/eamt2017/

Conference

Conference	20th Annual Conference of the European Association for Machine Translation
Abbreviated title	EAMT 2017
Country/Territory	Czech Republic
City	Prague
Period	29/05/17 → 31/05/17
Internet address	https://ufal.mff.cuni.cz/eamt2017/

Access to Document

EAMT2017_paper_32Final published version, 116 KBLicence: Creative Commons: Attribution No Derivatives (CC-BY-ND)

https://ufal.mff.cuni.cz/eamt2017/user-project-product-papers/papers/user/EAMT2017_paper_32.pdfLicence: Creative Commons: Attribution No Derivatives (CC-BY-ND)

1 Finished

QT21: Quality Translation 21
Koehn, P., Haddow, B. & Huck, M.
EU government bodies
1/02/15 → 31/01/18
Project: Research

Cite this

@inproceedings{dd17d7ab4e3342f4be8a0468bbbdad39,

title = "Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data",

abstract = "In the last year, we have seen a lot of evidence about the superiority of neuralmachine translation approaches (NMT) over phrase-based statistical approaches(PBMT). This trend has shown for the general domain at public competitions such as the WMT challenges as well as in the obvious quality increase in online translation services that have changed their technology. In this paper, we take the perspective of an LSP. The questions we want to answer with this study is if now is already the time to invest in the new technology. To answer this question, we have collected evidence as to whether an existing state of-the-art NMT system for the general domain can already compete with a domain trained and optimised Moses (PBMT) system or if it is maybe already better. As it is well known that automatic quality measures are not reliable for comparing the performance of different system types, we have performed a detailed manual evaluation based on a test suite of domain segments. ",

author = "Anne Beyer and Vivien Macketanz and Aljoscha Burchardt and Philip Williams",

year = "2017",

month = may,

day = "31",

language = "English",

booktitle = "EAMT 2017: The 20th Annual Conference of the European Association for Machine Translation",

note = "20th Annual Conference of the European Association for Machine Translation, EAMT 2017 ; Conference date: 29-05-2017 Through 31-05-2017",

url = "https://ufal.mff.cuni.cz/eamt2017/",

}

Beyer, A, Macketanz, V, Burchardt, A & Williams, P 2017, Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data. in EAMT 2017: The 20th Annual Conference of the European Association for Machine Translation. 20th Annual Conference of the European Association for Machine Translation, Prague, Czech Republic, 29/05/17. <https://ufal.mff.cuni.cz/eamt2017/user-project-product-papers/papers/user/EAMT2017_paper_32.pdf>

TY - GEN

T1 - Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data

AU - Beyer, Anne

AU - Macketanz, Vivien

AU - Burchardt, Aljoscha

AU - Williams, Philip

PY - 2017/5/31

Y1 - 2017/5/31

N2 - In the last year, we have seen a lot of evidence about the superiority of neuralmachine translation approaches (NMT) over phrase-based statistical approaches(PBMT). This trend has shown for the general domain at public competitions such as the WMT challenges as well as in the obvious quality increase in online translation services that have changed their technology. In this paper, we take the perspective of an LSP. The questions we want to answer with this study is if now is already the time to invest in the new technology. To answer this question, we have collected evidence as to whether an existing state of-the-art NMT system for the general domain can already compete with a domain trained and optimised Moses (PBMT) system or if it is maybe already better. As it is well known that automatic quality measures are not reliable for comparing the performance of different system types, we have performed a detailed manual evaluation based on a test suite of domain segments.

AB - In the last year, we have seen a lot of evidence about the superiority of neuralmachine translation approaches (NMT) over phrase-based statistical approaches(PBMT). This trend has shown for the general domain at public competitions such as the WMT challenges as well as in the obvious quality increase in online translation services that have changed their technology. In this paper, we take the perspective of an LSP. The questions we want to answer with this study is if now is already the time to invest in the new technology. To answer this question, we have collected evidence as to whether an existing state of-the-art NMT system for the general domain can already compete with a domain trained and optimised Moses (PBMT) system or if it is maybe already better. As it is well known that automatic quality measures are not reliable for comparing the performance of different system types, we have performed a detailed manual evaluation based on a test suite of domain segments.

M3 - Conference contribution

BT - EAMT 2017: The 20th Annual Conference of the European Association for Machine Translation

T2 - 20th Annual Conference of the European Association for Machine Translation

Y2 - 29 May 2017 through 31 May 2017

ER -

Can Out-of-the-box NMT Beat a Domain-trained Moses on Technical Data

Abstract

Conference

Access to Document

Projects

QT21: Quality Translation 21

Cite this