A Comparative Evaluation Methodology for NLG in Interactive Systems

Helen Hastie, Anja Belz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Interactive systems have become an increasingly important type of application for deployment of NLG technology over recent years. At present, we do not yet have commonly agreed terminology or methodology for evaluating NLG within interactive systems. In this paper, we take steps towards addressing this gap by presenting a set of principles for designing new evaluations in our comparative evaluation methodology. We start with presenting a categorisation framework, giving an overview of different categories of evaluation measures, in order to provide standard terminology for categorising existing and new evaluation techniques. Background on existing evaluation methodologies for NLG and interactive systems is presented. The comparative evaluation methodology is presented. Finally, a methodology for comparative evaluation of NLG components embedded within interactive systems is presented in terms of the comparative evaluation methodology, using a specific task for illustrative purposes.
Original languageEnglish
Title of host publicationProceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
EditorsNicoletta Calzolari
PublisherEuropean Language Resources Association (ELRA)
Pages4004-4011
Number of pages7
ISBN (Print)9782951740884
Publication statusPublished - May 2014
Event9th International Conference on Language Resources and Evaluation, LREC 2014 - Reykjavik, Iceland
Duration: 26 May 201431 May 2014
Conference number: 9
http://lrec2014.lrec-conf.org/en/

Conference

Conference9th International Conference on Language Resources and Evaluation, LREC 2014
Abbreviated titleLREC'14
Country/TerritoryIceland
CityReykjavik
Period26/05/1431/05/14
Internet address

Fingerprint

Dive into the research topics of 'A Comparative Evaluation Methodology for NLG in Interactive Systems'. Together they form a unique fingerprint.

Cite this