DARPA Communicator: Cross System Results for the 2001 Evaluation

Marilyn Walker; Alex Rudnicky; John Aberdeen; E Bratt; J Garofolo; Helen Wright Hastie; A Le; B Pellom; A Potamianos; Rebecca Passonneau; Rashmi Prasad; S Roukos; G Sanders; Stephanie Seneff; D Stallard

doi:10.21437/ICSLP.2002-132

DARPA Communicator: Cross System Results for the 2001 Evaluation

Marilyn Walker, Alex Rudnicky, John Aberdeen, E Bratt, J Garofolo, Helen Wright Hastie, A Le, B Pellom, A Potamianos, Rebecca Passonneau, Rashmi Prasad, S Roukos, G Sanders, Stephanie Seneff, D Stallard

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

Original language	English
Title of host publication	Proc. 7th International Conference on Spoken Language Processing (ICSLP 2002)
Publisher	ISCA
Pages	269-272
DOIs	https://doi.org/10.21437/ICSLP.2002-132
Publication status	Published - 2002
Event	7th International Conference on Spoken Language Processing (Interspeech 2002) - Denver, CO, United States Duration: 16 Sept 2002 → 20 Sept 2002

Conference

Conference	7th International Conference on Spoken Language Processing (Interspeech 2002)
Country/Territory	United States
City	Denver, CO
Period	16/09/02 → 20/09/02

Access to Document

10.21437/ICSLP.2002-132Licence: All Rights Reserved

https://www.isca-speech.org/archive/icslp_2002/walker02_icslp.htmlLicence: All Rights Reserved

Cite this

Walker, M., Rudnicky, A., Aberdeen, J., Bratt, E., Garofolo, J., Wright Hastie, H., Le, A., Pellom, B., Potamianos, A., Passonneau, R., Prasad, R., Roukos, S., Sanders, G., Seneff, S., & Stallard, D. (2002). DARPA Communicator: Cross System Results for the 2001 Evaluation. In Proc. 7th International Conference on Spoken Language Processing (ICSLP 2002) (pp. 269-272). ISCA. https://doi.org/10.21437/ICSLP.2002-132

@inproceedings{d1d8e94af6e541dd8b829b072e83a9d8,

title = "DARPA Communicator: Cross System Results for the 2001 Evaluation",

abstract = "This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.",

author = "Marilyn Walker and Alex Rudnicky and John Aberdeen and E Bratt and J Garofolo and {Wright Hastie}, Helen and A Le and B Pellom and A Potamianos and Rebecca Passonneau and Rashmi Prasad and S Roukos and G Sanders and Stephanie Seneff and D Stallard",

year = "2002",

doi = "10.21437/ICSLP.2002-132",

language = "English",

pages = "269--272",

booktitle = "Proc. 7th International Conference on Spoken Language Processing (ICSLP 2002)",

publisher = "ISCA",

note = "7th International Conference on Spoken Language Processing (Interspeech 2002) ; Conference date: 16-09-2002 Through 20-09-2002",

}

Walker, M, Rudnicky, A, Aberdeen, J, Bratt, E, Garofolo, J, Wright Hastie, H, Le, A, Pellom, B, Potamianos, A, Passonneau, R, Prasad, R, Roukos, S, Sanders, G, Seneff, S & Stallard, D 2002, DARPA Communicator: Cross System Results for the 2001 Evaluation. in Proc. 7th International Conference on Spoken Language Processing (ICSLP 2002). ISCA, pp. 269-272, 7th International Conference on Spoken Language Processing (Interspeech 2002), Denver, CO, United States, 16/09/02. https://doi.org/10.21437/ICSLP.2002-132

TY - GEN

T1 - DARPA Communicator: Cross System Results for the 2001 Evaluation

AU - Walker, Marilyn

AU - Rudnicky, Alex

AU - Aberdeen, John

AU - Bratt, E

AU - Garofolo, J

AU - Wright Hastie, Helen

AU - Le, A

AU - Pellom, B

AU - Potamianos, A

AU - Passonneau, Rebecca

AU - Prasad, Rashmi

AU - Roukos, S

AU - Sanders, G

AU - Seneff, Stephanie

AU - Stallard, D

PY - 2002

Y1 - 2002

N2 - This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

AB - This paper describes the evaluation methodology and results of the 2001 DARPA Communicator evaluation. The experiment spanned 6 months of 2001 and involved eight DARPA Communicator systems in the travel planning domain. It resulted in a corpus of 1242 dialogs which include many more dialogues for complex tasks than the 2000 evaluation. We describe the experimental design, the approach to data collection, and the results. We compare the results by the type of travel plan and by system. The results demonstrate some large differences across sites and show that the complex trips are clearly more difficult.

U2 - 10.21437/ICSLP.2002-132

DO - 10.21437/ICSLP.2002-132

M3 - Conference contribution

SP - 269

EP - 272

BT - Proc. 7th International Conference on Spoken Language Processing (ICSLP 2002)

PB - ISCA

T2 - 7th International Conference on Spoken Language Processing (Interspeech 2002)

Y2 - 16 September 2002 through 20 September 2002

ER -

DARPA Communicator: Cross System Results for the 2001 Evaluation

Abstract

Conference

Access to Document

Fingerprint

Cite this