TY - JOUR
T1 - Fashion clothes matching scheme based on Siamese Network and AutoEncoder
AU - Gao, Guangyu
AU - Liu, Liling
AU - Wang, Li
AU - Zhang, Yihang
N1 - Publisher Copyright:
© 2019, Springer-Verlag GmbH Germany, part of Springer Nature.
PY - 2019/12/1
Y1 - 2019/12/1
N2 - Owing to the rise of living standard, people attach greater importance to personal appearance, especially clothes matching. With image processing and machine learning technology, we can analyze the pattern of clothes matching for recommendation on clothes images. However, we still face great challenges. To be more specific, there exist excessive complicated factors influencing relation among clothes items, such as color or material, and we also struggle against the problem about how to extract efficient and accurate features. Thus, with the purpose of dealing with such challenges, this paper proposes an efficient clothes matching scheme with Siamese Network and AutoEncoder based on both labeled data from dataset FashionVC and unlabeled data from MicroBlog. More specifically, at first, except for clothes suiting with text from FashionVC, the gallery data also include matching clothes outfits recommended by fashionista in MicroBlog (MbFashion). Meanwhile, a semi-supervised clustering based on assembling was also proposed to generate negative samples to form a comprehensive dataset. Secondly, with consideration of matching patterns from MbFashion, we promoted the Siamese Network properly to more efficiently extract vision features on the constructed training dataset. After that, the traditional features are also extracted, while the Triple AutoEncoder and Bayesian Personalized Ranking are used to map the three kinds of features into the same latent space to learn the compatibility between tops and bottoms. Finally, we conducted a series of experiments and evaluated our results to demonstrate the usefulness and effectiveness of the whole scheme on FashionVC and MbFashion.
AB - Owing to the rise of living standard, people attach greater importance to personal appearance, especially clothes matching. With image processing and machine learning technology, we can analyze the pattern of clothes matching for recommendation on clothes images. However, we still face great challenges. To be more specific, there exist excessive complicated factors influencing relation among clothes items, such as color or material, and we also struggle against the problem about how to extract efficient and accurate features. Thus, with the purpose of dealing with such challenges, this paper proposes an efficient clothes matching scheme with Siamese Network and AutoEncoder based on both labeled data from dataset FashionVC and unlabeled data from MicroBlog. More specifically, at first, except for clothes suiting with text from FashionVC, the gallery data also include matching clothes outfits recommended by fashionista in MicroBlog (MbFashion). Meanwhile, a semi-supervised clustering based on assembling was also proposed to generate negative samples to form a comprehensive dataset. Secondly, with consideration of matching patterns from MbFashion, we promoted the Siamese Network properly to more efficiently extract vision features on the constructed training dataset. After that, the traditional features are also extracted, while the Triple AutoEncoder and Bayesian Personalized Ranking are used to map the three kinds of features into the same latent space to learn the compatibility between tops and bottoms. Finally, we conducted a series of experiments and evaluated our results to demonstrate the usefulness and effectiveness of the whole scheme on FashionVC and MbFashion.
KW - AutoEncoder
KW - Clothes matching
KW - Fashion analysis
KW - Siamese Network
UR - http://www.scopus.com/inward/record.url?scp=85065815332&partnerID=8YFLogxK
U2 - 10.1007/s00530-019-00617-9
DO - 10.1007/s00530-019-00617-9
M3 - Article
AN - SCOPUS:85065815332
SN - 0942-4962
VL - 25
SP - 593
EP - 602
JO - Multimedia Systems
JF - Multimedia Systems
IS - 6
ER -