Power Law Discounting for N-Gram Language Models

Songfang Huang, Steve Renals

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We present an approximation to the Bayesian hierarchical Pitman-Yor process language model which maintains the power law distribution over word tokens, while not requiring a computationally expensive approximate inference process. This approximation, which we term power law discounting, has a similar computational complexity to interpolated and modified Kneser-Ney smoothing. We performed experiments on meeting transcription using the NIST RT06s evaluation data and the AMI corpus, with a vocabulary of 50,000 words and a language model training set of up to 211 million words. Our results indicate that power law discounting results in statistically significant reductions in perplexity and word error rate compared to both interpolated and modified Kneser-Ney smoothing, while producing similar results to the hierarchical Pitman-Yor process language model.
Original languageEnglish
Title of host publicationProc. IEEE ICASSP--10
Publication statusPublished - 2010


Dive into the research topics of 'Power Law Discounting for N-Gram Language Models'. Together they form a unique fingerprint.

Cite this