A Knowledge-Enriched Ensemble Method for Word Embedding and Multi-Sense Embedding

Lanting Fang*, Yong Luo, Kaiyu Feng*, Kaiqi Zhao, Aiqun Hu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)

Abstract

Representing words as embeddings has been proven to be successful in improving the performance in many natural language processing tasks. Different from the traditional methods that learn the embeddings from large text corpora, ensemble methods have been proposed to leverage the merits of pre-trained word embeddings as well as external semantic sources. In this paper, we propose a knowledge-enriched ensemble method to combine information from both knowledge graphs and pre-trained word embeddings. Specifically, we propose an attention network to retrofit the semantic information in the lexical knowledge graph into the pre-trained word embeddings. In addition, we further extend our method to contextual word embeddings and multi-sense embeddings. Extensive experiments demonstrate that the proposed word embeddings outperform the state-of-the-art models in word analogy, word similarity and several downstream tasks. The proposed word sense embeddings outperform the state-of-the-art models in word similarity and word sense induction tasks.

Original languageEnglish
Pages (from-to)5534-5549
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume35
Issue number6
DOIs
Publication statusPublished - 1 Jun 2023

Keywords

  • Word embedding
  • ensemble model
  • knowledge graph
  • multi-sense embedding

Fingerprint

Dive into the research topics of 'A Knowledge-Enriched Ensemble Method for Word Embedding and Multi-Sense Embedding'. Together they form a unique fingerprint.

Cite this