Assessing the Quality of a Knowledge Graph via Link Prediction Tasks

Ricky Zhu; Alan Bundy; Fangrong Wang; Xue Li; Kuwabena Nuamah; Lei Xu; Stefano Mauceri; Jeff Z Pan

doi:10.1145/3639233.3639357

Assessing the Quality of a Knowledge Graph via Link Prediction Tasks

Ricky Zhu, Alan Bundy, Fangrong Wang, Xue Li, Kuwabena Nuamah, Lei Xu, Stefano Mauceri, Jeff Z Pan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Knowledge Graph (KG) Construction is the prerequisite for all other KG research and applications. Researchers and engineers have proposed various approaches to build KGs for their use cases. However, how can we know whether our constructed KG is good or bad? Is it correct and complete? Is it consistent and robust? In this paper, we propose a method called LP-Measure to assess the quality of a KG via a link prediction tasks, without using a gold standard or other human labour. Though theoretically, the LP-Measure can only assess consistency and redundancy, instead of the more desirable correctness and completeness, empirical evidence shows that this measurement method can quantitatively distinguish the good KGs from the bad ones, even in terms of incorrectness and incompleteness. Compared with the most commonly used manual assessment, our LP-Measure is an automated evaluation, which saves time and human labour.

Original language	English
Title of host publication	NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval
Publisher	Association for Computing Machinery
Pages	124-129
Number of pages	6
ISBN (Electronic)	9798400709227
DOIs	https://doi.org/10.1145/3639233.3639357
Publication status	Published - 5 Mar 2024
Event	7th International Conference on Natural Language Processing and Information Retrieval - Seoul, Korea, Republic of Duration: 15 Dec 2023 → 17 Dec 2023 Conference number: 7 http://www.nlpir.net/index.html

Conference

Conference	7th International Conference on Natural Language Processing and Information Retrieval
Abbreviated title	NLPIR 2023
Country/Territory	Korea, Republic of
City	Seoul
Period	15/12/23 → 17/12/23
Internet address	http://www.nlpir.net/index.html

Keywords / Materials (for Non-textual outputs)

knowledge graph
link prediction
quality assessment

Access to Document

10.1145/3639233.3639357Licence: All Rights Reserved

Assessing_the_Quality_of_a_Knowledge_Graph_via_Link_Prediction_TasksAccepted author manuscript, 652 KBLicence: Creative Commons: Attribution (CC-BY)

https://dl.acm.org/doi/10.1145/3639233.3639357Licence: All Rights Reserved

1 Finished
1 Active

TEAMER : Teaching Machines to Reason Like Humans
Lapata, M.
UK central government bodies/local authorities, health and hospital authorities
1/10/21 → 30/09/26
Project: Research
UKRI Trustworthy Autonomous Systems Node in Governance and Regulation
Ramamoorthy, R., Belle, V., Bundy, A., Jackson, P., Lascarides, A. & Rajan, A.
EPSRC
1/11/20 → 30/04/24
Project: Research

Cite this

Zhu, R., Bundy, A., Wang, F., Li, X., Nuamah, K., Xu, L., Mauceri, S., & Pan, J. Z. (2024). Assessing the Quality of a Knowledge Graph via Link Prediction Tasks. In NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval (pp. 124-129). Association for Computing Machinery. https://doi.org/10.1145/3639233.3639357

@inproceedings{e843434412874f9ab4aae55d49b232c7,

title = "Assessing the Quality of a Knowledge Graph via Link Prediction Tasks",

abstract = "Knowledge Graph (KG) Construction is the prerequisite for all other KG research and applications. Researchers and engineers have proposed various approaches to build KGs for their use cases. However, how can we know whether our constructed KG is good or bad? Is it correct and complete? Is it consistent and robust? In this paper, we propose a method called LP-Measure to assess the quality of a KG via a link prediction tasks, without using a gold standard or other human labour. Though theoretically, the LP-Measure can only assess consistency and redundancy, instead of the more desirable correctness and completeness, empirical evidence shows that this measurement method can quantitatively distinguish the good KGs from the bad ones, even in terms of incorrectness and incompleteness. Compared with the most commonly used manual assessment, our LP-Measure is an automated evaluation, which saves time and human labour.",

keywords = "knowledge graph, link prediction, quality assessment",

author = "Ricky Zhu and Alan Bundy and Fangrong Wang and Xue Li and Kuwabena Nuamah and Lei Xu and Stefano Mauceri and Pan, {Jeff Z}",

year = "2024",

month = mar,

day = "5",

doi = "10.1145/3639233.3639357",

language = "English",

pages = "124--129",

booktitle = "NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval",

publisher = "Association for Computing Machinery",

note = "7th International Conference on Natural Language Processing and Information Retrieval, NLPIR 2023 ; Conference date: 15-12-2023 Through 17-12-2023",

url = "http://www.nlpir.net/index.html",

}

Zhu, R, Bundy, A, Wang, F, Li, X , Nuamah, K, Xu, L, Mauceri, S & Pan, JZ 2024, Assessing the Quality of a Knowledge Graph via Link Prediction Tasks. in NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval. Association for Computing Machinery, pp. 124-129, 7th International Conference on Natural Language Processing and Information Retrieval, Seoul, Korea, Republic of, 15/12/23. https://doi.org/10.1145/3639233.3639357

Assessing the Quality of a Knowledge Graph via Link Prediction Tasks. / Zhu, Ricky; Bundy, Alan; Wang, Fangrong et al.
NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval. Association for Computing Machinery, 2024. p. 124-129.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Assessing the Quality of a Knowledge Graph via Link Prediction Tasks

AU - Zhu, Ricky

AU - Bundy, Alan

AU - Wang, Fangrong

AU - Li, Xue

AU - Nuamah, Kuwabena

AU - Xu, Lei

AU - Mauceri, Stefano

AU - Pan, Jeff Z

N1 - Conference code: 7

PY - 2024/3/5

Y1 - 2024/3/5

N2 - Knowledge Graph (KG) Construction is the prerequisite for all other KG research and applications. Researchers and engineers have proposed various approaches to build KGs for their use cases. However, how can we know whether our constructed KG is good or bad? Is it correct and complete? Is it consistent and robust? In this paper, we propose a method called LP-Measure to assess the quality of a KG via a link prediction tasks, without using a gold standard or other human labour. Though theoretically, the LP-Measure can only assess consistency and redundancy, instead of the more desirable correctness and completeness, empirical evidence shows that this measurement method can quantitatively distinguish the good KGs from the bad ones, even in terms of incorrectness and incompleteness. Compared with the most commonly used manual assessment, our LP-Measure is an automated evaluation, which saves time and human labour.

AB - Knowledge Graph (KG) Construction is the prerequisite for all other KG research and applications. Researchers and engineers have proposed various approaches to build KGs for their use cases. However, how can we know whether our constructed KG is good or bad? Is it correct and complete? Is it consistent and robust? In this paper, we propose a method called LP-Measure to assess the quality of a KG via a link prediction tasks, without using a gold standard or other human labour. Though theoretically, the LP-Measure can only assess consistency and redundancy, instead of the more desirable correctness and completeness, empirical evidence shows that this measurement method can quantitatively distinguish the good KGs from the bad ones, even in terms of incorrectness and incompleteness. Compared with the most commonly used manual assessment, our LP-Measure is an automated evaluation, which saves time and human labour.

KW - knowledge graph

KW - link prediction

KW - quality assessment

U2 - 10.1145/3639233.3639357

DO - 10.1145/3639233.3639357

M3 - Conference contribution

SP - 124

EP - 129

BT - NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval

PB - Association for Computing Machinery

T2 - 7th International Conference on Natural Language Processing and Information Retrieval

Y2 - 15 December 2023 through 17 December 2023

ER -

Assessing the Quality of a Knowledge Graph via Link Prediction Tasks

Abstract / Description of output

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Projects

TEAMER : Teaching Machines to Reason Like Humans

UKRI Trustworthy Autonomous Systems Node in Governance and Regulation

Cite this