Bayesian Inference for PCFGs via Markov Chain Monte Carlo

Mark Johnson; Thomas Griffiths; Sharon Goldwater

Bayesian Inference for PCFGs via Markov Chain Monte Carlo

Mark Johnson, Thomas Griffiths, Sharon Goldwater

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as Inside-Outside algorithm only produce a trivial grammar.

Original language	English
Title of host publication	Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference
Place of Publication	Rochester, New York
Publisher	Association for Computational Linguistics
Pages	139-146
Number of pages	8
Publication status	Published - 1 Apr 2007

Access to Document

N07 1018Final published version, 201 KB

http://www.aclweb.org/anthology/N/N07/N07-1018.pdf

Cite this

Johnson, M., Griffiths, T., & Goldwater, S. (2007). Bayesian Inference for PCFGs via Markov Chain Monte Carlo. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference (pp. 139-146). Association for Computational Linguistics. http://www.aclweb.org/anthology/N/N07/N07-1018.pdf

@inproceedings{a446f3fb73d744d3b4a408ecacb13b05,

title = "Bayesian Inference for PCFGs via Markov Chain Monte Carlo",

abstract = "This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as Inside-Outside algorithm only produce a trivial grammar.",

author = "Mark Johnson and Thomas Griffiths and Sharon Goldwater",

year = "2007",

month = apr,

day = "1",

language = "English",

pages = "139--146",

booktitle = "Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference",

publisher = "Association for Computational Linguistics",

}

Johnson, M, Griffiths, T & Goldwater, S 2007, Bayesian Inference for PCFGs via Markov Chain Monte Carlo. in Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference. Association for Computational Linguistics, Rochester, New York, pp. 139-146. <http://www.aclweb.org/anthology/N/N07/N07-1018.pdf>

Bayesian Inference for PCFGs via Markov Chain Monte Carlo. / Johnson, Mark; Griffiths, Thomas; Goldwater, Sharon.
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference. Rochester, New York: Association for Computational Linguistics, 2007. p. 139-146.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Bayesian Inference for PCFGs via Markov Chain Monte Carlo

AU - Johnson, Mark

AU - Griffiths, Thomas

AU - Goldwater, Sharon

PY - 2007/4/1

Y1 - 2007/4/1

N2 - This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as Inside-Outside algorithm only produce a trivial grammar.

AB - This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as Inside-Outside algorithm only produce a trivial grammar.

M3 - Conference contribution

SP - 139

EP - 146

BT - Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference

PB - Association for Computational Linguistics

CY - Rochester, New York

ER -

Bayesian Inference for PCFGs via Markov Chain Monte Carlo

Abstract / Description of output

Access to Document

Fingerprint

Cite this