Attribute driven zero-shot classification and segmentation

Shu Yang, Yemin Shi, Yaowei Wang*, Jing Wang, Zesong Fei

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Citations (Scopus)

Abstract

Zero-shot classification and segmentation aims to recognize and segment objects of unseen classes. The attribute information, such as color, shape, part and material, is usually used for zero-shot classification. Moreover, we observe that this kind of attribute information could also be helpful in the segmentation task. On this basis, we propose an Attribute-Segmentation-Attribute (ASA) framework to address the zero-shot classification and segmentation problem. In the framework, a multi-task model is pre-trained to capture category and attribute features simultaneously. Then, a two-branch fully convolutional structure is built on the pre-trained model and fine-tuned for segmentation task. Finally, the extracted class-unseen object is recognized with the segmentation-assisted attribute prediction and a class-attribute matrix. Experimental results on the public bench-mark datasets indicate that the proposed ASA framework out-performs the state-of-the-art methods for both classification and segmentation tasks.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538641958
DOIs
Publication statusPublished - 28 Nov 2018
Event2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018 - San Diego, United States
Duration: 23 Jul 201827 Jul 2018

Publication series

Name2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018

Conference

Conference2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018
Country/TerritoryUnited States
CitySan Diego
Period23/07/1827/07/18

Keywords

  • ASA framework
  • attribute driven
  • zero-shot classification and segmentation

Fingerprint

Dive into the research topics of 'Attribute driven zero-shot classification and segmentation'. Together they form a unique fingerprint.

Cite this

Yang, S., Shi, Y., Wang, Y., Wang, J., & Fei, Z. (2018). Attribute driven zero-shot classification and segmentation. In 2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018 Article 8551489 (2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMEW.2018.8551489