Using the Web to Overcome Data Sparseness

Frank Keller; Maria Lapata; Olga Ourioupina

Using the Web to Overcome Data Sparseness

Frank Keller, Maria Lapata, Olga Ourioupina

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

This paper shows that the web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the web by querying a search engine. We evaluate this method by demonstrating that web frequencies and correlate with frequencies obtained from a carefully edited, balanced corpus. We also perform a task-based evaluation, showing that web frequencies can reliably predict human plausibility judgments.

Original language	English
Title of host publication	EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing
Place of Publication	Stroudsburg, PA
Publisher	Association for Computational Linguistics
Pages	230-237
Number of pages	8
Volume	10
Publication status	Published - 2002
Event	7th Conference on Empirical Methods in Natural Language Processing (EMNLP 2002) - University of Pennsylvania, Philadelphia, PA, United States Duration: 6 Jul 2002 → 7 Jul 2002

Conference

Conference	7th Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)
Country/Territory	United States
City	Philadelphia, PA
Period	6/07/02 → 7/07/02

Access to Document

W02-1030Final published version, 45.6 KB

http://aclweb.org/anthology/W02-1030

Cite this

@inbook{c38241b32c404d0985caf8c4a8f49f66,

title = "Using the Web to Overcome Data Sparseness",

abstract = "This paper shows that the web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the web by querying a search engine. We evaluate this method by demonstrating that web frequencies and correlate with frequencies obtained from a carefully edited, balanced corpus. We also perform a task-based evaluation, showing that web frequencies can reliably predict human plausibility judgments.",

author = "Frank Keller and Maria Lapata and Olga Ourioupina",

note = "EMNLP Best Paper Award; 7th Conference on Empirical Methods in Natural Language Processing (EMNLP 2002) ; Conference date: 06-07-2002 Through 07-07-2002",

year = "2002",

language = "English",

volume = "10",

pages = "230--237",

booktitle = "EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing",

publisher = "Association for Computational Linguistics",

}

Keller, F , Lapata, M & Ourioupina, O 2002, Using the Web to Overcome Data Sparseness. in EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing. vol. 10, Association for Computational Linguistics, Stroudsburg, PA, pp. 230-237, 7th Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), Philadelphia, PA, United States, 6/07/02. <http://aclweb.org/anthology/W02-1030>

TY - CHAP

T1 - Using the Web to Overcome Data Sparseness

AU - Keller, Frank

AU - Lapata, Maria

AU - Ourioupina, Olga

N1 - EMNLP Best Paper Award

PY - 2002

Y1 - 2002

N2 - This paper shows that the web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the web by querying a search engine. We evaluate this method by demonstrating that web frequencies and correlate with frequencies obtained from a carefully edited, balanced corpus. We also perform a task-based evaluation, showing that web frequencies can reliably predict human plausibility judgments.

AB - This paper shows that the web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the web by querying a search engine. We evaluate this method by demonstrating that web frequencies and correlate with frequencies obtained from a carefully edited, balanced corpus. We also perform a task-based evaluation, showing that web frequencies can reliably predict human plausibility judgments.

M3 - Chapter

VL - 10

SP - 230

EP - 237

BT - EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing

PB - Association for Computational Linguistics

CY - Stroudsburg, PA

T2 - 7th Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)

Y2 - 6 July 2002 through 7 July 2002

ER -

Using the Web to Overcome Data Sparseness

Abstract

Conference

Access to Document

Fingerprint

Cite this