Image captioning with relational knowledge

Huan Yang, Dandan Song*, Lejian Liao

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)
Plum Print visual indicator of research metrics
  • Citations
    • Citation Indexes: 2
  • Captures
    • Readers: 2
see details

Abstract

People have learned extensive relational knowledge from daily life. This is one of the facts that enables human to describe the information from images easily. In this paper, we propose a novel framework called Image Captioning with Relational Knowledge (ICRK) that combines relational knowledge with image captioning model and utilizes relational knowledge to strengthen the learning process of representing words. As more precise syntactic and semantic word relationships were learned, the image captioning model acquires more semantic features that help to generate more accurate image descriptions. Experiments on several benchmark datasets, using automatic evaluation metrics, have all demonstrated that our model can significantly improve the quality of image captioning.

Original languageEnglish
Title of host publicationPRICAI 2018
Subtitle of host publicationTrends in Artificial Intelligence - 15th Pacific Rim International Conference on Artificial Intelligence, Proceedings
EditorsXin Geng, Byeong-Ho Kang
PublisherSpringer Verlag
Pages378-386
Number of pages9
ISBN (Print)9783319973098
DOIs
Publication statusPublished - 2018
Event15th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2018 - Nanjing, China
Duration: 28 Aug 201831 Aug 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11013 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2018
Country/TerritoryChina
CityNanjing
Period28/08/1831/08/18

Keywords

  • Image captioning
  • Relational knowledge
  • Word embedding

Fingerprint

Dive into the research topics of 'Image captioning with relational knowledge'. Together they form a unique fingerprint.

Cite this

Yang, H., Song, D., & Liao, L. (2018). Image captioning with relational knowledge. In X. Geng, & B.-H. Kang (Eds.), PRICAI 2018: Trends in Artificial Intelligence - 15th Pacific Rim International Conference on Artificial Intelligence, Proceedings (pp. 378-386). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11013 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-97310-4_43