Interpolating between types and tokens by estimating power-law generators

Sharon Goldwater; Tom Griffiths; Mark Johnson

Interpolating between types and tokens by estimating power-law generators

Sharon Goldwater, Tom Griffiths, Mark Johnson

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process – the Pitman-Yor process – as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 18
Editors	Y. Weiss, B. Schölkopf, J. Platt
Place of Publication	Cambridge, MA
Publisher	MIT Press
Pages	459-466
Number of pages	8
Publication status	Published - 2006

Access to Document

NIPS2005_0333Final published version, 0.98 MB

http://books.nips.cc/papers/files/nips18/NIPS2005_0333.pdf

Cite this

@inproceedings{f454297dbd0741fdba3b9a514f5bf710,

title = "Interpolating between types and tokens by estimating power-law generators",

abstract = "Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process – the Pitman-Yor process – as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.",

author = "Sharon Goldwater and Tom Griffiths and Mark Johnson",

year = "2006",

language = "English",

pages = "459--466",

editor = "Y. Weiss and B. Sch{\"o}lkopf and J. Platt",

booktitle = "Advances in Neural Information Processing Systems 18",

publisher = "MIT Press",

}

TY - GEN

T1 - Interpolating between types and tokens by estimating power-law generators

AU - Goldwater, Sharon

AU - Griffiths, Tom

AU - Johnson, Mark

PY - 2006

Y1 - 2006

N2 - Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process – the Pitman-Yor process – as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

AB - Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process – the Pitman-Yor process – as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

M3 - Conference contribution

SP - 459

EP - 466

BT - Advances in Neural Information Processing Systems 18

A2 - Weiss, Y.

A2 - Schölkopf, B.

A2 - Platt, J.

PB - MIT Press

CY - Cambridge, MA

ER -

Interpolating between types and tokens by estimating power-law generators

Abstract

Access to Document

Fingerprint

Cite this