Edinburgh Research Explorer

Repeatable and reliable semantic search evaluation

Research output: Contribution to journalArticle

Related Edinburgh Organisations

Open Access permissions


Original languageEnglish
Pages (from-to)14-29
JournalWeb Semantics: Science, Services and Agents on the World Wide Web
Publication statusPublished - 1 Aug 2013


An increasing amount of structured data on the Web has attracted industry attention and renewed research interest in what is collectively referred to as semantic search. These solutions exploit the explicit semantics captured in structured data such as RDF for enhancing document representation and retrieval, or for finding answers by directly searching over the data. These data have been used for different tasks and a wide range of corresponding semantic search solutions have been proposed in the past. However, it has been widely recognized that a standardized setting to evaluate and analyze the current state-of-the-art in semantic search is needed to monitor and stimulate further progress in the field. In this paper, we present an evaluation framework for semantic search, analyze the framework with regard to repeatability and reliability, and report on our experiences on applying it in the Semantic Search Challenge 2010 and 2011.

ID: 24995495