ASA-Net: Deep representation learning between object silhouette and attributes

Object silhouette and semantic attributes are respectively verified for their effectiveness as auxiliary supervision in image recognition. It encourages us to propose a novel zero-shot recognition model, Attribute-Segmentation-Attribute network (ASA-Net), which jointly conducts object segmentation, attribute prediction and recognition in a multi-task learning manner. Firstly, a feature extraction module is pre-trained based on smooth attribute and category annotations. This module is then adopted to initialize the feature encoding module of a multi-scale segmentation CNN to generate coarse-to-fine object silhouettes. Finally, the segments are multiplied with the original image to obtain regions of interest, and semantic features of these regions are extracted and combined to predict attributes. The obtained attribute prediction is further projected into the category space to accomplish the zero-shot recognition task. Experimental results on two public benchmarks indicate that our ASA-Net performs better than baseline and existing methods in attribute prediction and segmentation tasks, as well as the unseen object recognition. The source code is publicly available online (https://github.com/YsSue/ASA-Net.git).