Representational Bias in Unsupervised Learning of Syllable Structure

Sharon Goldwater, Mark Johnson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Unsupervised learning algorithms based on Expectation Maximization (EM) are often straightforward to implement and provably converge on a local likelihood maximum. However, these algorithms often do not perform well in practice. Common wisdom holds that they yield poor results because they are overly sensitive to initial parameter values and easily get stuck in local (but not global) maxima. We present a series of experiments indicating that for the task of learning syllable structure, the initial parameter weights are not crucial. Rather, it is the choice of model class itself that makes the difference between successful and unsuccessful learning. We use a language-universal rule-based algorithm to find a good set of parameters, and then train the parameter weights using EM. We achieve word accuracy of 95.9% on German and 97.1% on English, as compared to 97.4% and 98.1% respectively for supervised training.
Original languageEnglish
Title of host publicationProceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)
Place of PublicationAnn Arbor, Michigan
PublisherAssociation for Computational Linguistics
Pages112-119
Number of pages8
Publication statusPublished - 1 Jun 2005

Fingerprint

Dive into the research topics of 'Representational Bias in Unsupervised Learning of Syllable Structure'. Together they form a unique fingerprint.

Cite this