TY - JOUR
T1 - Repeatable and reliable semantic search evaluation
AU - Blanco, Roi
AU - Halpin, Harry
AU - Herzig, Daniel M.
AU - Mika, Peter
AU - Pound, Jeffrey
AU - Thompson, Henry S.
AU - Tran, Thanh
PY - 2013/8/1
Y1 - 2013/8/1
N2 - An increasing amount of structured data on the Web has attracted industry attention and renewed research interest in what is collectively referred to as semantic search. These solutions exploit the explicit semantics captured in structured data such as RDF for enhancing document representation and retrieval, or for finding answers by directly searching over the data. These data have been used for different tasks and a wide range of corresponding semantic search solutions have been proposed in the past. However, it has been widely recognized that a standardized setting to evaluate and analyze the current state-of-the-art in semantic search is needed to monitor and stimulate further progress in the field. In this paper, we present an evaluation framework for semantic search, analyze the framework with regard to repeatability and reliability, and report on our experiences on applying it in the Semantic Search Challenge 2010 and 2011.
AB - An increasing amount of structured data on the Web has attracted industry attention and renewed research interest in what is collectively referred to as semantic search. These solutions exploit the explicit semantics captured in structured data such as RDF for enhancing document representation and retrieval, or for finding answers by directly searching over the data. These data have been used for different tasks and a wide range of corresponding semantic search solutions have been proposed in the past. However, it has been widely recognized that a standardized setting to evaluate and analyze the current state-of-the-art in semantic search is needed to monitor and stimulate further progress in the field. In this paper, we present an evaluation framework for semantic search, analyze the framework with regard to repeatability and reliability, and report on our experiences on applying it in the Semantic Search Challenge 2010 and 2011.
U2 - 10.1016/j.websem.2013.05.005
DO - 10.1016/j.websem.2013.05.005
M3 - Article
SN - 1570-8268
VL - 21
SP - 14
EP - 29
JO - Journal of Web Semantics
JF - Journal of Web Semantics
ER -