Edinburgh Research Explorer

Extrinsic Summarization Evaluation: A Decision Audit Task

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Original languageEnglish
Title of host publicationMachine Learning for Multimodal Interaction
Subtitle of host publication5th International Workshop, MLMI 2008, Utrecht, The Netherlands, September 8-10, 2008. Proceedings
EditorsAndrei Popescu-Belis, Rainer Stiefelhagen
PublisherSpringer Berlin Heidelberg
Pages349-361
Number of pages13
ISBN (Electronic)978-3-540-85853-9
ISBN (Print)978-3-540-85852-2
DOIs
Publication statusPublished - 2008

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Berlin Heidelberg
Volume5237
ISSN (Print)0302-9743

Abstract

In this work we describe a large-scale extrinsic evaluation of automatic speech summarization technologies for meeting speech. The particular task is a decision audit, wherein a user must satisfy a complex information need, navigating several meetings in order to gain an understanding of how and why a given decision was made. We compare the usefulness of extractive and abstractive technologies in satisfying this information need, and assess the impact of automatic speech recognition (ASR) errors on user performance. We employ several evaluation methods for participant performance, including post-questionnaire data, human subjective and objective judgments, and an analysis of participant browsing behaviour.

ID: 14858244