Abstract / Description of output
Interactive systems have become an increasingly important type of application for deployment of NLG technology over recent years. At present, we do not yet have commonly agreed terminology or methodology for evaluating NLG within interactive systems. In this paper, we take steps towards addressing this gap by presenting a set of principles for designing new evaluations in our comparative evaluation methodology. We start with presenting a categorisation framework, giving an overview of different categories of evaluation measures, in order to provide standard terminology for categorising existing and new evaluation techniques. Background on existing evaluation methodologies for NLG and interactive systems is presented. The comparative evaluation methodology is presented. Finally, a methodology for comparative evaluation of NLG components embedded within interactive systems is presented in terms of the comparative evaluation methodology, using a specific task for illustrative purposes.
Original language | English |
---|---|
Title of host publication | Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) |
Editors | Nicoletta Calzolari |
Publisher | European Language Resources Association (ELRA) |
Pages | 4004-4011 |
Number of pages | 7 |
ISBN (Print) | 9782951740884 |
Publication status | Published - May 2014 |
Event | 9th International Conference on Language Resources and Evaluation, LREC 2014 - Reykjavik, Iceland Duration: 26 May 2014 → 31 May 2014 Conference number: 9 http://lrec2014.lrec-conf.org/en/ |
Conference
Conference | 9th International Conference on Language Resources and Evaluation, LREC 2014 |
---|---|
Abbreviated title | LREC'14 |
Country/Territory | Iceland |
City | Reykjavik |
Period | 26/05/14 → 31/05/14 |
Internet address |