Investigating the dynamic decision mechanisms of users’ Relevance judgment for information retrieval via log analysis

Yi Su, Jingfei Li*, Dawei Song, Pengqing Zhang, Yazhou Zhang

*此作品的通讯作者

科研成果: 书/报告/会议事项章节会议稿件同行评审

2 引用 (Scopus)
Plum Print visual indicator of research metrics
  • Citations
    • Citation Indexes: 2
  • Captures
    • Readers: 7
see details

摘要

Measuring relevance of documents with respect to a user’s query is at the heart of information retrieval (IR), where the user’s relevance judgment criteria have been recognized as multi-dimensional. A set of relevance dimensions that are considered as critical factors in document relevance judgment have been investigated, such as topicality, novelty, and reliability. However, most existing work focuses on individual relevance dimensions, yet neglecting how different dimensions would interact with each other to influence the overall relevance judgment in real-world search scenarios. This paper aims at an initial step to fill the gap. Specifically, we divide 7 relevance dimensions in an enriched Multidimensional User Relevance Model (MURM) into three categories according to three main requirements for document relevance, i.e., document content requirement, document quality requirement and personalization requirement. We then exploit the Learning to Rank framework to conduct document ranking experiments on a query log dataset from a prominent search engine. The experimental results indicate the existence of an order effect between different dimensions, and suggest that considering different dimensions across categories in different orders for document relevance judgment could lead to distinct search results. Our findings provide valuable insights to build more intelligent and user-centric information retrieval systems, and potentially benefit other natural language processing tasks that involve decision making from multiple perspectives.

源语言英语
主期刊名PRICAI 2018
主期刊副标题Trends in Artificial Intelligence - 15th Pacific Rim International Conference on Artificial Intelligence, Proceedings
编辑Byeong-Ho Kang, Xin Geng
出版商Springer Verlag
968-979
页数12
ISBN(印刷版)9783319973036
DOI
出版状态已出版 - 2018
活动15th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2018 - Nanjing, 中国
期限: 28 8月 201831 8月 2018

出版系列

姓名Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
11012 LNAI
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议15th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2018
国家/地区中国
Nanjing
时期28/08/1831/08/18

指纹

探究 'Investigating the dynamic decision mechanisms of users’ Relevance judgment for information retrieval via log analysis' 的科研主题。它们共同构成独一无二的指纹。

引用此

Su, Y., Li, J., Song, D., Zhang, P., & Zhang, Y. (2018). Investigating the dynamic decision mechanisms of users’ Relevance judgment for information retrieval via log analysis. 在 B.-H. Kang, & X. Geng (编辑), PRICAI 2018: Trends in Artificial Intelligence - 15th Pacific Rim International Conference on Artificial Intelligence, Proceedings (页码 968-979). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 卷 11012 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-97304-3_74