Parallel Sentence Mining by Constrained Decoding

Pinzhen Chen; Nikolay Bogoychev; Kenneth Heafield; Faheem Kirefu

doi:10.18653/v1/2020.acl-main.152

Parallel Sentence Mining by Constrained Decoding

Pinzhen Chen, Nikolay Bogoychev, Kenneth Heafield, Faheem Kirefu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation. Our method relies on translating sentences in one corpus, but constraining the decoding by a prefix tree built on the other corpus. We argue that a neural machine translation system by itself can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search. When benchmarked on the BUCC shared task, our method achieves results comparable to other submissions.

Original language	English
Title of host publication	Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Publisher	Association for Computational Linguistics (ACL)
Pages	1672–1678
Number of pages	7
ISBN (Electronic)	978-1-952148-25-5
DOIs	https://doi.org/10.18653/v1/2020.acl-main.152
Publication status	Published - 10 Jul 2020
Event	2020 Annual Conference of the Association for Computational Linguistics - Hyatt Regency Seattle, Virtual conference, United States Duration: 5 Jul 2020 → 10 Jul 2020 Conference number: 58 https://acl2020.org/

Conference

Conference	2020 Annual Conference of the Association for Computational Linguistics
Abbreviated title	ACL 2020
Country/Territory	United States
City	Virtual conference
Period	5/07/20 → 10/07/20
Internet address	https://acl2020.org/

Access to Document

10.18653/v1/2020.acl-main.152Licence: Creative Commons: Attribution (CC-BY)

Parallel Sentence Mining_CHEN_DOA03042020_AFVAccepted author manuscript, 176 KBLicence: Creative Commons: Attribution (CC-BY)
Parallel Sentence_CHEN_DOA03042020_VOR_CC-BYFinal published version, 255 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/2020.acl-main.152Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{dac6d5f2fd0f40648a6cb4bb07647b49,

title = "Parallel Sentence Mining by Constrained Decoding",

abstract = "We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation. Our method relies on translating sentences in one corpus, but constraining the decoding by a prefix tree built on the other corpus. We argue that a neural machine translation system by itself can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search. When benchmarked on the BUCC shared task, our method achieves results comparable to other submissions.",

author = "Pinzhen Chen and Nikolay Bogoychev and Kenneth Heafield and Faheem Kirefu",

year = "2020",

month = jul,

day = "10",

doi = "10.18653/v1/2020.acl-main.152",

language = "English",

pages = "1672–1678",

booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

note = "2020 Annual Conference of the Association for Computational Linguistics, ACL 2020 ; Conference date: 05-07-2020 Through 10-07-2020",

url = "https://acl2020.org/",

}

Chen, P, Bogoychev, N, Heafield, K & Kirefu, F 2020, Parallel Sentence Mining by Constrained Decoding. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics (ACL), pp. 1672–1678, 2020 Annual Conference of the Association for Computational Linguistics, Virtual conference, Washington, United States, 5/07/20. https://doi.org/10.18653/v1/2020.acl-main.152

TY - GEN

T1 - Parallel Sentence Mining by Constrained Decoding

AU - Chen, Pinzhen

AU - Bogoychev, Nikolay

AU - Heafield, Kenneth

AU - Kirefu, Faheem

N1 - Conference code: 58

PY - 2020/7/10

Y1 - 2020/7/10

N2 - We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation. Our method relies on translating sentences in one corpus, but constraining the decoding by a prefix tree built on the other corpus. We argue that a neural machine translation system by itself can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search. When benchmarked on the BUCC shared task, our method achieves results comparable to other submissions.

AB - We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation. Our method relies on translating sentences in one corpus, but constraining the decoding by a prefix tree built on the other corpus. We argue that a neural machine translation system by itself can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search. When benchmarked on the BUCC shared task, our method achieves results comparable to other submissions.

U2 - 10.18653/v1/2020.acl-main.152

DO - 10.18653/v1/2020.acl-main.152

M3 - Conference contribution

SP - 1672

EP - 1678

BT - Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - 2020 Annual Conference of the Association for Computational Linguistics

Y2 - 5 July 2020 through 10 July 2020

ER -

Parallel Sentence Mining by Constrained Decoding

Abstract

Conference

Access to Document

Fingerprint

Cite this