Contrastive decoding reduces hallucinations in large multilingual machine translation models

Jonas Waldendorf, Barry Haddow, Alexandra Birch

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In Neural Machine Translation (NMT), models will sometimes generate repetitive or fluent output that is not grounded in the source sentence. This phenomenon is known as hallucination and is a problem even in large-scale multilingual translation models. We propose to use Contrastive Decoding, an algorithm developed to improve generation from unconditional language models, to mitigate hallucinations in NMT. Specifically, we maximise the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. Additionally, we propose an alternative implementation of Contrastive Decoding that dynamically weights the difference based on the maximum probability in the output distribution to reduce the effect of CD when the model is confident of its prediction. We evaluate our methods using the Small (418M) and Medium (1.2B) M2M models across 21 low and medium-resource language pairs. Our results show a 14.6 ± 0.5 and 11.0 ± 0.6 maximal increase in the mean COMET scores for the Small and Medium models on those sentences for which the M2M models initially generate a hallucination., respectively.
Original languageEnglish
Title of host publicationProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics
EditorsYvette Graham, Matthew Purver
PublisherAssociation for Computational Linguistics
Pages2526–2539
Number of pages14
Volume1
ISBN (Electronic)9798891760882
Publication statusPublished - 22 Mar 2024
EventThe 18th Conference of the European Chapter of the Association for Computational Linguistics - St. Julian’s, Malta
Duration: 17 Mar 202422 Mar 2024
Conference number: 18
https://2024.eacl.org/

Publication series

NameProceedings of the EACL Conference
PublisherAssociation for Computational Linguistics
ISSN (Print)1525-2450

Conference

ConferenceThe 18th Conference of the European Chapter of the Association for Computational Linguistics
Abbreviated titleEACL 2024
Country/TerritoryMalta
CitySt. Julian’s
Period17/03/2422/03/24
Internet address

Fingerprint

Dive into the research topics of 'Contrastive decoding reduces hallucinations in large multilingual machine translation models'. Together they form a unique fingerprint.

Cite this