Edinburgh Research Explorer

Evaluation of Extractive Voicemail Summarization

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Original languageEnglish
Title of host publication2003 ISCA Workshop on Multilingual Spoken Document Retrieval, Proceedings of the
Subtitle of host publicationMSDR 2003
PublisherISCA
Pages19-24
Publication statusPublished - 2003
Event2003 ISCA Workshop on Multilingual Spoken Document Retrieval (MSDR2003) - Hong Kong, Hong Kong
Duration: 4 Apr 20035 Apr 2003

Workshop

Workshop2003 ISCA Workshop on Multilingual Spoken Document Retrieval (MSDR2003)
CountryHong Kong
CityHong Kong
Period4/04/035/04/03

Abstract

This paper is about the evaluation of a system that generates short text summaries of voicemail messages, suitable for transmission as text messages. Our approach to summarization is based on a speech-recognized transcript of the voicemail message, from which a set of summary words is extracted. The system uses a classifier to identify the summary words, with each word being identified by a vector of lexical and prosodic features. The features are selected using Parcel, an ROC-based algorithm. Our evaluations of the system, using a slot error rate metric, have compared manual and automatic summarization, and manual and automatic recognition (using two different recognizers). We also report on two subjective evaluations using mean opinion score of summaries, and a set of comprehension tests. The main results from these experiments were that the perceived difference in quality of summarization was affected more by errors resulting from automatic transcription, than by the automatic summarization process.

Event

2003 ISCA Workshop on Multilingual Spoken Document Retrieval (MSDR2003)

4/04/035/04/03

Hong Kong, Hong Kong

Event: Workshop

ID: 27411869