Semantic description of image based on I-NiC model

Chaoying Zhang; Yaping Dai; Hao Wang; Zhiyang Jia; Kaoru Hirota

Semantic description of image based on I-NiC model

Chaoying Zhang, Yaping Dai, Hao Wang, Zhiyang Jia^*, Kaoru Hirota

^*Corresponding author for this work

School of Automation

Beijing Institute of Technology

Research output: Contribution to conference › Paper › peer-review

Abstract

In order to address the problems of misprediction and object missing in semantic description of image, an improved Neural Image Caption (I-NIC) model is proposed. It primarily consists of the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The model uses Inception-v4 model developed by Google to extract image features and iteratively optimizes the training process parameters through word-based loss function. Therefore, the I-NIC model can generate more relevant descriptions and improve the accuracy and efficiency of the system. Compared with NIC model, the experiment results show that the accuracy of I-NIC model is improved by 2.5% with BLEU-4 metrics, 1.2% with METEOR metrics and 7.5% with CIDEr metrics on the Microsoft COCO Caption dataset.

Original language	English
Publication status	Published - 2018
Event	8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018 - Tengzhou, Shandong, China Duration: 2 Nov 2018 → 6 Nov 2018

Conference

Conference	8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018
Country/Territory	China
City	Tengzhou, Shandong
Period	2/11/18 → 6/11/18

Keywords

Convolutional Neural Network
Long Short-Term Memory
Neural Networks
Semantic Description of Image

Cite this

@conference{58c9bf0b57f6408baea905f8a242a618,

title = "Semantic description of image based on I-NiC model",

abstract = "In order to address the problems of misprediction and object missing in semantic description of image, an improved Neural Image Caption (I-NIC) model is proposed. It primarily consists of the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The model uses Inception-v4 model developed by Google to extract image features and iteratively optimizes the training process parameters through word-based loss function. Therefore, the I-NIC model can generate more relevant descriptions and improve the accuracy and efficiency of the system. Compared with NIC model, the experiment results show that the accuracy of I-NIC model is improved by 2.5% with BLEU-4 metrics, 1.2% with METEOR metrics and 7.5% with CIDEr metrics on the Microsoft COCO Caption dataset.",

keywords = "Convolutional Neural Network, Long Short-Term Memory, Neural Networks, Semantic Description of Image",

author = "Chaoying Zhang and Yaping Dai and Hao Wang and Zhiyang Jia and Kaoru Hirota",

note = "Publisher Copyright: {\textcopyright} ISCIIA and ITCA 2018 - 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Application.; 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018 ; Conference date: 02-11-2018 Through 06-11-2018",

year = "2018",

language = "English",

}

Zhang, C, Dai, Y, Wang, H, Jia, Z & Hirota, K 2018, 'Semantic description of image based on I-NiC model', Paper presented at 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018, Tengzhou, Shandong, China, 2/11/18 - 6/11/18.

Semantic description of image based on I-NiC model. / Zhang, Chaoying; Dai, Yaping; Wang, Hao et al.
2018. Paper presented at 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018, Tengzhou, Shandong, China.

Research output: Contribution to conference › Paper › peer-review

TY - CONF

T1 - Semantic description of image based on I-NiC model

AU - Zhang, Chaoying

AU - Dai, Yaping

AU - Wang, Hao

AU - Jia, Zhiyang

AU - Hirota, Kaoru

N1 - Publisher Copyright: © ISCIIA and ITCA 2018 - 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Application.

PY - 2018

Y1 - 2018

N2 - In order to address the problems of misprediction and object missing in semantic description of image, an improved Neural Image Caption (I-NIC) model is proposed. It primarily consists of the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The model uses Inception-v4 model developed by Google to extract image features and iteratively optimizes the training process parameters through word-based loss function. Therefore, the I-NIC model can generate more relevant descriptions and improve the accuracy and efficiency of the system. Compared with NIC model, the experiment results show that the accuracy of I-NIC model is improved by 2.5% with BLEU-4 metrics, 1.2% with METEOR metrics and 7.5% with CIDEr metrics on the Microsoft COCO Caption dataset.

AB - In order to address the problems of misprediction and object missing in semantic description of image, an improved Neural Image Caption (I-NIC) model is proposed. It primarily consists of the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The model uses Inception-v4 model developed by Google to extract image features and iteratively optimizes the training process parameters through word-based loss function. Therefore, the I-NIC model can generate more relevant descriptions and improve the accuracy and efficiency of the system. Compared with NIC model, the experiment results show that the accuracy of I-NIC model is improved by 2.5% with BLEU-4 metrics, 1.2% with METEOR metrics and 7.5% with CIDEr metrics on the Microsoft COCO Caption dataset.

KW - Convolutional Neural Network

KW - Long Short-Term Memory

KW - Neural Networks

KW - Semantic Description of Image

UR - http://www.scopus.com/inward/record.url?scp=85057977797&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85057977797

T2 - 8th International Symposium on Computational Intelligence and Industrial Applications and 12th China-Japan International Workshop on Information Technology and Control Applications, ISCIIA and ITCA 2018

Y2 - 2 November 2018 through 6 November 2018

ER -

Semantic description of image based on I-NiC model

Abstract

Conference

Keywords

Other files and links

Fingerprint

Cite this