Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Mathias Müller; Rico Sennrich

doi:10.18653/v1/2021.acl-long.22

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Mathias Müller, Rico Sennrich

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search – the de facto standard inference algorithm in NMT – and Eikema and Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead.

In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.

Original language	English
Title of host publication	Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Place of Publication	Online
Publisher	Association for Computational Linguistics
Pages	259-272
Number of pages	14
ISBN (Electronic)	978-1-954085-52-7
DOIs	https://doi.org/10.18653/v1/2021.acl-long.22
Publication status	Published - 1 Aug 2021
Event	The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing - Bangkok, Thailand Duration: 1 Aug 2021 → 6 Aug 2021 https://2021.aclweb.org/

Conference

Conference	The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing
Abbreviated title	ACL-IJCNLP 2021
Country/Territory	Thailand
City	Bangkok
Period	1/08/21 → 6/08/21
Internet address	https://2021.aclweb.org/

Access to Document

10.18653/v1/2021.acl-long.22Licence: Creative Commons: Attribution (CC-BY)

Understanding the Properties_MULLER_DOA06052021_VOR_CC-BYFinal published version, 8.8 MBLicence: Creative Commons: Attribution (CC-BY)

https://aclanthology.org/2021.acl-long.22Licence: Creative Commons: Attribution (CC-BY)

Cite this

Müller, M., & Sennrich, R. (2021). Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 259-272). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.acl-long.22

Müller, Mathias ; Sennrich, Rico. / Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online : Association for Computational Linguistics, 2021. pp. 259-272

@inproceedings{f2192b1dd6e44e2f95aa7250a24da8fb,

title = "Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation",

abstract = "Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search – the de facto standard inference algorithm in NMT – and Eikema and Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead.In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.",

author = "Mathias M{\"u}ller and Rico Sennrich",

year = "2021",

month = aug,

day = "1",

doi = "10.18653/v1/2021.acl-long.22",

language = "English",

pages = "259--272",

booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",

publisher = "Association for Computational Linguistics",

note = "The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021 ; Conference date: 01-08-2021 Through 06-08-2021",

url = "https://2021.aclweb.org/",

}

Müller, M & Sennrich, R 2021, Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, pp. 259-272, The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Bangkok, Thailand, 1/08/21. https://doi.org/10.18653/v1/2021.acl-long.22

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. / Müller, Mathias; Sennrich, Rico.
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics, 2021. p. 259-272.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

AU - Müller, Mathias

AU - Sennrich, Rico

PY - 2021/8/1

Y1 - 2021/8/1

N2 - Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search – the de facto standard inference algorithm in NMT – and Eikema and Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead.In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.

AB - Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words, and shows poor robustness to copy noise in training data or domain shift. Recent work has tied these shortcomings to beam search – the de facto standard inference algorithm in NMT – and Eikema and Aziz (2020) propose to use Minimum Bayes Risk (MBR) decoding on unbiased samples instead.In this paper, we empirically investigate the properties of MBR decoding on a number of previously reported biases and failure cases of beam search. We find that MBR still exhibits a length and token frequency bias, owing to the MT metrics used as utility functions, but that MBR also increases robustness against copy noise in the training data and domain shift.

U2 - 10.18653/v1/2021.acl-long.22

DO - 10.18653/v1/2021.acl-long.22

M3 - Conference contribution

SP - 259

EP - 272

BT - Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

PB - Association for Computational Linguistics

CY - Online

T2 - The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing

Y2 - 1 August 2021 through 6 August 2021

ER -

Müller M, Sennrich R. Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics. 2021. p. 259-272 doi: 10.18653/v1/2021.acl-long.22

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Abstract / Description of output

Conference

Access to Document

Fingerprint

Cite this