Improving Sparse Word Representations with Distributional Inference for Semantic Composition

Thomas Kober, Julie Weeds, Jeremy Reffin, David Weir

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Distributional models are derived from cooccurrences in a corpus, where only a small proportion of all possible plausible cooccurrences will be observed. This results in a very sparse vector space, requiring a mechanism for inferring missing knowledge. Most methods face this challenge in ways that render the resulting word representations uninterpretable, with the consequence that semantic composition becomes hard to model. In this paper we explore an alternative which involves explicitly inferring unobserved co-occurrences using the distributional neighbourhood. We show that distributional inference improves sparse word representations on several word similarity benchmarks and demonstrate that our model is competitive with the state-of-the-art for adjective-noun, noun-noun and verb-object compositions while being fully interpretable.
Original languageEnglish
Title of host publicationProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
PublisherAssociation for Computational Linguistics
Pages1691-1702
Number of pages12
DOIs
Publication statusPublished - 5 Nov 2016
Event2016 Conference on Empirical Methods in Natural Language Processing - Austin, United States
Duration: 1 Nov 20165 Nov 2016
https://www.aclweb.org/mirror/emnlp2016/

Conference

Conference2016 Conference on Empirical Methods in Natural Language Processing
Abbreviated titleEMNLP 2016
Country/TerritoryUnited States
CityAustin
Period1/11/165/11/16
Internet address

Fingerprint

Dive into the research topics of 'Improving Sparse Word Representations with Distributional Inference for Semantic Composition'. Together they form a unique fingerprint.

Cite this