A Comprehensive Approach to Assess Trustworthiness and Completeness of Knowledge Graphs

Abstract—Knowledge Graphs (KGs) have been widely used for many tasks such as question-answering, recommendation, natural language processing, and so on. The quality of KGs is a crucial factor in determining whether a specific KG is appropriate for usage in a certain application. Completeness and trustworthiness are two dimensions that are used to assess the quality of KGs. Estimation of the completeness and trustworthiness of a largescale knowledge graph often requires humans to annotate samples from the graph. How to obtain statistically meaningful estimates for quality evaluation while keeping humans out of the loop to reduce cost is a critical problem. Nowadays, to reduce the costs of the manual construction of knowledge graphs, many KGs have been constructed automatically from sources with varying degrees of trustworthiness. Therefore, possible noises and conflicts are inevitably introduced in the process of construction, which severely interferes with the quality of constructed KGs. Many works have been done to detect noisy triples; however, how to estimate the quality of the entire KG has largely been ignored in prior research. To fill this gap, we propose a new approach to automatically evaluate and assess existing KGs in terms of completeness and trustworthiness. In this paper, we conduct several experiments on three standard datasets, namely, FB15K, WN18, and NELL995 to estimate the quality of KGs and assign a specific score based on the completeness and trustworthiness of the KG. The experimental findings demonstrate the reliability of the proposed scores.