A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text

Anirban Chakraborty.; Kripabandhu Ghosh.; Utpal Roy.

doi:10.5220/0005157304500456

A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text

Anirban Chakraborty., Kripabandhu Ghosh., Utpal Roy.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

OCR errors hurt retrieval performance to a great extent. Research has been done on modelling and correction of OCR errors. However, most of the existing systems use language dependent resources or training texts for studying the nature of errors. Not much research has been reported on improving retrieval performance from erroneous text when no training data is available. We propose an algorithm of detecting OCR errors and improving retrieval performance from the erroneous corpus. We present two versions of the algorithm: one based on word cooccurrence and the other based on Pointwise Mutual Information. Our algorithm does not use any training data or any language specific resources like thesaurus. It also does not use any knowledge about the language except that the word delimiter is a blank space. We have tested our algorithm on erroneous Bangla FIRE collection and obtained significant improvements.

Original language	English
Title of host publication	Proceedings of the International Conference on Knowledge Discovery and Information Retrieval
Editors	Ana Fred, Joaquim Filipe
Publisher	SCITEPRESS
Pages	450-456
Number of pages	17
ISBN (Electronic)	978-989-758-048-2
DOIs	https://doi.org/10.5220/0005157304500456
Publication status	Published - 24 Oct 2014
Event	6th International Conference on Knowledge Discovery and Information Retrieval (KDIR) - Rome, Italy Duration: 21 Oct 2014 → 24 Oct 2014 Conference number: 6 https://kdir.scitevents.org/Home.aspx?y=2014

Publication series

Name	Proceedings of the International Conference on Knowledge Discovery and Information Retrieval
Publisher	Scitepress
ISSN (Electronic)	2184-3228

Conference

Conference	6th International Conference on Knowledge Discovery and Information Retrieval (KDIR)
Abbreviated title	KDIR 2014
Country/Territory	Italy
City	Rome
Period	21/10/14 → 24/10/14
Internet address	https://kdir.scitevents.org/Home.aspx?y=2014

Keywords

Erroneous Text
Cooccurrence
Pointwise Mutual Information

Access to Document

10.5220/0005157304500456Licence: All Rights Reserved

https://www.scitepress.org/Link.aspx?doi=10.5220/0005157304500456Licence: All Rights Reserved

Cite this

Chakraborty., A., Ghosh., K., & Roy., U. (2014). A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text. In A. Fred, & J. Filipe (Eds.), Proceedings of the International Conference on Knowledge Discovery and Information Retrieval (pp. 450-456). (Proceedings of the International Conference on Knowledge Discovery and Information Retrieval). SCITEPRESS. https://doi.org/10.5220/0005157304500456

Chakraborty., Anirban ; Ghosh., Kripabandhu ; Roy., Utpal. / A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text. Proceedings of the International Conference on Knowledge Discovery and Information Retrieval. editor / Ana Fred ; Joaquim Filipe. SCITEPRESS, 2014. pp. 450-456 (Proceedings of the International Conference on Knowledge Discovery and Information Retrieval).

@inproceedings{81471e989f804532a8a5e33821d13a33,

title = "A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text",

abstract = "OCR errors hurt retrieval performance to a great extent. Research has been done on modelling and correction of OCR errors. However, most of the existing systems use language dependent resources or training texts for studying the nature of errors. Not much research has been reported on improving retrieval performance from erroneous text when no training data is available. We propose an algorithm of detecting OCR errors and improving retrieval performance from the erroneous corpus. We present two versions of the algorithm: one based on word cooccurrence and the other based on Pointwise Mutual Information. Our algorithm does not use any training data or any language specific resources like thesaurus. It also does not use any knowledge about the language except that the word delimiter is a blank space. We have tested our algorithm on erroneous Bangla FIRE collection and obtained significant improvements.",

keywords = "Erroneous Text, Cooccurrence, Pointwise Mutual Information",

author = "Anirban Chakraborty. and Kripabandhu Ghosh. and Utpal Roy.",

year = "2014",

month = oct,

day = "24",

doi = "10.5220/0005157304500456",

language = "English",

series = "Proceedings of the International Conference on Knowledge Discovery and Information Retrieval",

publisher = "SCITEPRESS",

pages = "450--456",

editor = "Ana Fred and Joaquim Filipe",

booktitle = "Proceedings of the International Conference on Knowledge Discovery and Information Retrieval",

note = "6th International Conference on Knowledge Discovery and Information Retrieval (KDIR), KDIR 2014 ; Conference date: 21-10-2014 Through 24-10-2014",

url = "https://kdir.scitevents.org/Home.aspx?y=2014",

}

Chakraborty., A, Ghosh., K & Roy., U 2014, A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text. in A Fred & J Filipe (eds), Proceedings of the International Conference on Knowledge Discovery and Information Retrieval. Proceedings of the International Conference on Knowledge Discovery and Information Retrieval, SCITEPRESS, pp. 450-456, 6th International Conference on Knowledge Discovery and Information Retrieval (KDIR), Rome, Italy, 21/10/14. https://doi.org/10.5220/0005157304500456

A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text. / Chakraborty., Anirban; Ghosh., Kripabandhu; Roy., Utpal.

Proceedings of the International Conference on Knowledge Discovery and Information Retrieval. ed. / Ana Fred; Joaquim Filipe. SCITEPRESS, 2014. p. 450-456 (Proceedings of the International Conference on Knowledge Discovery and Information Retrieval).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text

AU - Chakraborty., Anirban

AU - Ghosh., Kripabandhu

AU - Roy., Utpal

N1 - Conference code: 6

PY - 2014/10/24

Y1 - 2014/10/24

N2 - OCR errors hurt retrieval performance to a great extent. Research has been done on modelling and correction of OCR errors. However, most of the existing systems use language dependent resources or training texts for studying the nature of errors. Not much research has been reported on improving retrieval performance from erroneous text when no training data is available. We propose an algorithm of detecting OCR errors and improving retrieval performance from the erroneous corpus. We present two versions of the algorithm: one based on word cooccurrence and the other based on Pointwise Mutual Information. Our algorithm does not use any training data or any language specific resources like thesaurus. It also does not use any knowledge about the language except that the word delimiter is a blank space. We have tested our algorithm on erroneous Bangla FIRE collection and obtained significant improvements.

AB - OCR errors hurt retrieval performance to a great extent. Research has been done on modelling and correction of OCR errors. However, most of the existing systems use language dependent resources or training texts for studying the nature of errors. Not much research has been reported on improving retrieval performance from erroneous text when no training data is available. We propose an algorithm of detecting OCR errors and improving retrieval performance from the erroneous corpus. We present two versions of the algorithm: one based on word cooccurrence and the other based on Pointwise Mutual Information. Our algorithm does not use any training data or any language specific resources like thesaurus. It also does not use any knowledge about the language except that the word delimiter is a blank space. We have tested our algorithm on erroneous Bangla FIRE collection and obtained significant improvements.

KW - Erroneous Text

KW - Cooccurrence

KW - Pointwise Mutual Information

U2 - 10.5220/0005157304500456

DO - 10.5220/0005157304500456

M3 - Conference contribution

T3 - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval

SP - 450

EP - 456

BT - Proceedings of the International Conference on Knowledge Discovery and Information Retrieval

A2 - Fred, Ana

A2 - Filipe, Joaquim

PB - SCITEPRESS

T2 - 6th International Conference on Knowledge Discovery and Information Retrieval (KDIR)

Y2 - 21 October 2014 through 24 October 2014

ER -

Chakraborty. A, Ghosh. K, Roy. U. A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text. In Fred A, Filipe J, editors, Proceedings of the International Conference on Knowledge Discovery and Information Retrieval. SCITEPRESS. 2014. p. 450-456. (Proceedings of the International Conference on Knowledge Discovery and Information Retrieval). https://doi.org/10.5220/0005157304500456

A Word Association Based Approach for Improving Retrieval Performance from Noisy OCRed Text

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this