Deep Learning for Knowledge Graph Completion with XLNET

Knowledge Graph is a graph knowledge base composed of fact entities and relations. Recently, the adoption of Knowledge Graph in Natural Language Processing tasks has proved the efficiency and convenience of KG. Therefore, the plausibility of Knowledge Graph become an import subject, which is also named as KG Completion or Link Prediction. The plausibility of Knowledge Graph reflects in the validness of triples which is structured representation of the entities and relations of Knowledge Graph. Some research work has devoted to KG Completion tasks. The typical methods include semantic matching models like TransE or TransH and Pre-trained models like KG-BERT. In this article, we propose a novel method based on the pre-trained model XLNET and the classification model to verify whether the triples of Knowledge Graph are valid or not. This method takes description of entities or relations as the input sentence text for fine-tuning. Meanwhile contextualized representations with rich semantic information can be obtained by XLNET, avoiding limitations and shortcomings of other typical neural network models. Then these representations are fed into a classifier for classification. Experimental results show that there is an improvement in KG Completion Tasks that the proposed method has achieved.