Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates

Sharon Goldwater, Dan Jurafsky, Christopher D. Manning

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Despite years of speech recognition research, little is known about which words tend to be misrecognized and why. Previous work has shown that errors increase for infrequent words, short words, and very loud or fast speech, but many other presumed causes of error (e.g., nearby disfluencies, turn-initial words, phonetic neighborhood density) have never been carefully tested. The reasons for the huge differences found in error rates between speakers also remain largely mysterious. Using a mixed-effects regression model, we investigate these and other factors by analyzing the errors of two state-of-the-art recognizers on conversational speech. Words with higher error rates include those with extreme prosodic characteristics, those occurring turn-initially or as discourse markers, and doubly confusable pairs: acoustically similar words that also have similar language model probabilities. Words preceding disfluent interruption points (first repetition tokens and words before fragments) also have higher error rates. Finally, even after accounting for other factors, speaker differences cause enormous variance in error rates, suggesting that speaker error rate variance is not fully explained by differences in word choice, fluency, or prosodic characteristics. We also propose that doubly confusable pairs, rather than high neighborhood density, may better explain phonetic neighborhood errors in human speech processing.
Original languageEnglish
Pages (from-to)181-200
Number of pages20
JournalSpeech Communication
Volume52
Issue number3
DOIs
Publication statusPublished - Mar 2010

Keywords / Materials (for Non-textual outputs)

  • Mixed-effects model

Fingerprint

Dive into the research topics of 'Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates'. Together they form a unique fingerprint.

Cite this