Abstract / Description of output
Dialectal Arabic has no standard orthographic representation. This creates a
challenge when evaluating an Automatic Speech Recognition (ASR) system for dialect. Since the reference transcription text can vary widely from one user to another, we propose an innovative approach for evaluating dialectal speech recognition using Multi-References. For each recognized speech segments, we ask five different users to transcribe the speech. We combine the alignment for the multiple references, and use the combined alignment to report a modified version of Word Error Rate (WER). This approach is in favor of accepting a recognized word if any of the references typed it in the same form. Our method proved to be more effective in capturing many correctly recognized
words that have multiple acceptable spellings. The initial WER according
to each of the five references individually ranged between 76.4% to 80.9%. When
considering all references combined, the Multi-References MR-WER was found to
be 53%
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the Second Workshop on Arabic Natural Language Processing |
| Publisher | Association for Computational Linguistics |
| Pages | 118-126 |
| Number of pages | 9 |
| ISBN (Print) | 978-1-941643-58-7 |
| Publication status | Published - 1 Aug 2015 |