Orthogonal Vector-Decomposed Disentanglement Network of Interactive Image Retrieval for Fashion Outfit Recommendation

Chen Chen, Jie Guo, Bin Song*, Tong Zhang

*此作品的通讯作者

科研成果: 书/报告/会议事项章节会议稿件同行评审

摘要

Interactive image retrieval for fashion outfit recommendation is a challenging task, which aims to search for the target desired image according to a multi-modal query (a reference image and a modification text). Previous studies focus on exploring effective feature composing methods to achieve similarity matching between different modalities. However, the existence of feature redundancy and the semantic inconsistency between modalities introduces many task-irrelevant information. It is intractable to correctly identify the particular information to be modified and will inevitably introduce noise disturbances which lead to suboptimal performance. To this end, we present a novel Orthogonal Vector-Decomposed Disentanglement Network (OVDDN) for image retrieval. It proposes to leverage the disentangled parts to learn a controllable denoising embedding space. First, we design an orthogonal disentanglement module. It is applied to both image and text features to decouple them into two independent components (invariant and specific) through orthogonal constraints. A similarity metric loss ensures semantic consistency of paired images. Then, an attention network generates composition of the reference image invariant part and text task-related part to match the target one. Finally, a differential feature alignment module maintain the cross-modal semantic consistency. Extensive experiments conducted on three benchmark datasets denote the OVDDN achieving the consistently superior performance. Ablation analyses further verify the effectiveness of our proposed model.

源语言英语
主期刊名MCFR 2022 - Proceedings of the 1st Workshop on Multimedia Computing towards Fashion Recommendation
出版商Association for Computing Machinery, Inc
21-29
页数9
ISBN(电子版)9781450394987
DOI
出版状态已出版 - 14 10月 2022
已对外发布
活动1st Workshop on Multimedia Computing towards Fashion Recommendation, MCFR 2022 - Lisboa, 葡萄牙
期限: 14 10月 2022 → …

出版系列

姓名MCFR 2022 - Proceedings of the 1st Workshop on Multimedia Computing towards Fashion Recommendation

会议

会议1st Workshop on Multimedia Computing towards Fashion Recommendation, MCFR 2022
国家/地区葡萄牙
Lisboa
时期14/10/22 → …

指纹

探究 'Orthogonal Vector-Decomposed Disentanglement Network of Interactive Image Retrieval for Fashion Outfit Recommendation' 的科研主题。它们共同构成独一无二的指纹。

引用此