Improving Information Retrieval Performance on OCRed Text in the Absence of Clean Text Ground Truth

Kripabandhu Ghosh, Anirban Chakraborty, Swapan Kumar Parui, Prasenjit Majumder

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

OCR errors in text harm information retrieval performance. Much research has been reported on modelling and correction of Optical Character Recognition (OCR) errors. Most of the prior work employ language dependent resources or training texts in studying the nature of errors. However, not much research has been reported that focuses on improving retrieval performance from erroneous text in the absence of training data. We propose a novel approach for detecting OCR errors and improving retrieval performance from the erroneous corpus in a situation where training samples are not available to model errors. In this paper we propose a method that automatically identifies erroneous term variants in the noisy corpus, which are used for query expansion, in the absence of clean text. We employ an effective combination of contextual information and string matching techniques. Our proposed approach automatically identifies the erroneous variants of query terms and consequently leads to improvement in retrieval performance through query expansion. Our proposed approach does not use any training data or any language specific resources like thesaurus for identification of error variants. It also does not expend any knowledge about the language except that the word delimiter is blank space. We have tested our approach on erroneous Bangla (Bengali in English) and Hindi FIRE collections, and also on TREC Legal IIT CDIP and TREC 5 Confusion track English corpora. Our proposed approach has achieved statistically significant improvements over the state-of-the-art baselines on most of the datasets.
Original languageEnglish
Pages (from-to)873-884
Number of pages12
JournalInformation Processing and Management
Volume52
Issue number5
Early online date27 May 2016
DOIs
Publication statusPublished - 22 Jul 2016

Keywords / Materials (for Non-textual outputs)

  • Information Retrieval
  • OCR error
  • Word co-occurrence

Fingerprint

Dive into the research topics of 'Improving Information Retrieval Performance on OCRed Text in the Absence of Clean Text Ground Truth'. Together they form a unique fingerprint.

Cite this