Evaluating Automatic Summaries of Meeting Recordings

Gabriel Murray; Steve Renals; Jean Carletta; Johanna Moore

Evaluating Automatic Summaries of Meeting Recordings

Gabriel Murray, Steve Renals, Jean Carletta, Johanna Moore

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The research below explores schemes for evaluating automatic summaries of business meetings, using the ICSI Meeting Corpus (Janin et al., 2003). Both automatic and subjective evaluations were carried out, with a central interest being whether or not the two types of evaluations correlate with each other. The evaluation metrics were used to compare and contrast differing approaches to automatic summarization, the deterioration of summary quality on ASR output versus manual transcripts, and to determine whether manual extracts are rated significantly higher than automatic extracts.

Original language	English
Title of host publication	Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization
Subtitle of host publication	MTSE
Publisher	Association for Computational Linguistics
Pages	33-40
Number of pages	8
Publication status	Published - 2005
Event	Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL-05 Workshop) - University of Michigan, Ann Arbor, United States Duration: 29 Jun 2005 → 29 Jun 2005

Workshop

Workshop	Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL-05 Workshop)
Country/Territory	United States
City	Ann Arbor
Period	29/06/05 → 29/06/05

Access to Document

Murray Renals ET AL 2005 Evaluating Automatic Summaries of Meeting RecordingsAccepted author manuscript, 70.4 KB

http://aclweb.org/anthology/W05-0905

Cite this

@inproceedings{f914e3547bf7416bb1c203de5cc04e88,

title = "Evaluating Automatic Summaries of Meeting Recordings",

abstract = "The research below explores schemes for evaluating automatic summaries of business meetings, using the ICSI Meeting Corpus (Janin et al., 2003). Both automatic and subjective evaluations were carried out, with a central interest being whether or not the two types of evaluations correlate with each other. The evaluation metrics were used to compare and contrast differing approaches to automatic summarization, the deterioration of summary quality on ASR output versus manual transcripts, and to determine whether manual extracts are rated significantly higher than automatic extracts.",

author = "Gabriel Murray and Steve Renals and Jean Carletta and Johanna Moore",

year = "2005",

language = "English",

pages = "33--40",

booktitle = "Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization",

publisher = "Association for Computational Linguistics",

note = "Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL-05 Workshop) ; Conference date: 29-06-2005 Through 29-06-2005",

}

Murray, G, Renals, S, Carletta, J & Moore, J 2005, Evaluating Automatic Summaries of Meeting Recordings. in Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization: MTSE. Association for Computational Linguistics, pp. 33-40, Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL-05 Workshop), Ann Arbor, United States, 29/06/05. <http://aclweb.org/anthology/W05-0905>

Evaluating Automatic Summaries of Meeting Recordings. / Murray, Gabriel; Renals, Steve; Carletta, Jean et al.

Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization: MTSE. Association for Computational Linguistics, 2005. p. 33-40.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Evaluating Automatic Summaries of Meeting Recordings

AU - Murray, Gabriel

AU - Renals, Steve

AU - Carletta, Jean

AU - Moore, Johanna

PY - 2005

Y1 - 2005

N2 - The research below explores schemes for evaluating automatic summaries of business meetings, using the ICSI Meeting Corpus (Janin et al., 2003). Both automatic and subjective evaluations were carried out, with a central interest being whether or not the two types of evaluations correlate with each other. The evaluation metrics were used to compare and contrast differing approaches to automatic summarization, the deterioration of summary quality on ASR output versus manual transcripts, and to determine whether manual extracts are rated significantly higher than automatic extracts.

AB - The research below explores schemes for evaluating automatic summaries of business meetings, using the ICSI Meeting Corpus (Janin et al., 2003). Both automatic and subjective evaluations were carried out, with a central interest being whether or not the two types of evaluations correlate with each other. The evaluation metrics were used to compare and contrast differing approaches to automatic summarization, the deterioration of summary quality on ASR output versus manual transcripts, and to determine whether manual extracts are rated significantly higher than automatic extracts.

M3 - Conference contribution

SP - 33

EP - 40

BT - Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

PB - Association for Computational Linguistics

T2 - Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (ACL-05 Workshop)

Y2 - 29 June 2005 through 29 June 2005

ER -

Evaluating Automatic Summaries of Meeting Recordings

Abstract

Workshop

Access to Document

Fingerprint

Cite this