Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System

Anne H. Schneider; Ielka van der Sluis; Saturnino Luz

Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System

Anne H. Schneider^*, Ielka van der Sluis, Saturnino Luz

^*Corresponding author for this work

Research output: Contribution to conference › Paper › peer-review

Abstract / Description of output

We present an exploratory study to assess machine translation output for application in a dialogue system using an intrinsic and an extrinsic evaluation method. For the intrinsic evaluation we developed an annotation scheme to determine the quality of the translated utterances in isolation. For the extrinsic evaluation we employed the Wizard of Oz technique to assess the quality of the translations in the context of a dialogue application. Results differ and we discuss the possible reasons for this outcome.

Original language	English
Pages	329-336
Number of pages	8
Publication status	Published - 2 Dec 2010
Event	7th International Workshop on Spoken Language Translation, IWSLT 2010 - Paris, France Duration: 2 Dec 2010 → 3 Dec 2010

Conference

Conference	7th International Workshop on Spoken Language Translation, IWSLT 2010
Country/Territory	France
City	Paris
Period	2/12/10 → 3/12/10

Cite this

@conference{88abf6fbafe4448da0c34bf0b37d0e9b,

title = "Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System",

abstract = "We present an exploratory study to assess machine translation output for application in a dialogue system using an intrinsic and an extrinsic evaluation method. For the intrinsic evaluation we developed an annotation scheme to determine the quality of the translated utterances in isolation. For the extrinsic evaluation we employed the Wizard of Oz technique to assess the quality of the translations in the context of a dialogue application. Results differ and we discuss the possible reasons for this outcome.",

author = "Schneider, {Anne H.} and {van der Sluis}, Ielka and Saturnino Luz",

note = "Funding Information: This research is supported by the Science Foundation Ireland (Grant 07/CE/I1142) as part of the Centre for Next Generation Localisation (www.cngl.ie) at Trinity College Dublin. Publisher Copyright: {\textcopyright} IWSLT 2010. All rights reserved.; 7th International Workshop on Spoken Language Translation, IWSLT 2010 ; Conference date: 02-12-2010 Through 03-12-2010",

year = "2010",

month = dec,

day = "2",

language = "English",

pages = "329--336",

}

TY - CONF

T1 - Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System

AU - Schneider, Anne H.

AU - van der Sluis, Ielka

AU - Luz, Saturnino

N1 - Funding Information: This research is supported by the Science Foundation Ireland (Grant 07/CE/I1142) as part of the Centre for Next Generation Localisation (www.cngl.ie) at Trinity College Dublin. Publisher Copyright: © IWSLT 2010. All rights reserved.

PY - 2010/12/2

Y1 - 2010/12/2

N2 - We present an exploratory study to assess machine translation output for application in a dialogue system using an intrinsic and an extrinsic evaluation method. For the intrinsic evaluation we developed an annotation scheme to determine the quality of the translated utterances in isolation. For the extrinsic evaluation we employed the Wizard of Oz technique to assess the quality of the translations in the context of a dialogue application. Results differ and we discuss the possible reasons for this outcome.

AB - We present an exploratory study to assess machine translation output for application in a dialogue system using an intrinsic and an extrinsic evaluation method. For the intrinsic evaluation we developed an annotation scheme to determine the quality of the translated utterances in isolation. For the extrinsic evaluation we employed the Wizard of Oz technique to assess the quality of the translations in the context of a dialogue application. Results differ and we discuss the possible reasons for this outcome.

UR - http://www.scopus.com/inward/record.url?scp=84987856936&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:84987856936

SP - 329

EP - 336

T2 - 7th International Workshop on Spoken Language Translation, IWSLT 2010

Y2 - 2 December 2010 through 3 December 2010

ER -

Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System

Abstract / Description of output

Conference

Other files and links

Fingerprint

Cite this