Investigating Reading Behavior in Fine-grained Relevance Judgment

Zhijing Wu, Jiaxin Mao, Yiqun Liu, Min Zhang, Shaoping Ma

科研成果: 书/报告/会议事项章节会议稿件同行评审

5 引用 (Scopus)

摘要

A better understanding of users' reading behavior helps improve many information retrieval (IR) tasks, such as relevance estimation and document ranking. Existing research has already leveraged eye movement information to investigate user's reading process during document-level relevance judgments and the findings were adopted to build more effective ranking models. Recently, fine-grained (e.g., passage or sentence level) relevance judgments have been paid much attention to with the requirements in conversational search and QA systems. However, there is still a lack of thorough investigation on user's reading behavior during these kinds of interaction processes. To shed light on this research question, we investigate how users allocate their attention to passages of a document during the relevance judgment process. With the eye-tracking data collected in a laboratory study, we show that users pay more attention to the "key" passages which contain key useful information. Users tend to revisit these key passages several times to accumulate and verify the gathered information. With both content and user behavior features, we find that key passages can be predicted with supervised learning. We believe that this work contributes to better understanding users' reading behavior and may provide more explainability for relevance estimation.

源语言英语
主期刊名SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
出版商Association for Computing Machinery, Inc
1889-1892
页数4
ISBN(电子版)9781450380164
DOI
出版状态已出版 - 25 7月 2020
已对外发布
活动43rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020 - Virtual, Online, 中国
期限: 25 7月 202030 7月 2020

出版系列

姓名SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

会议

会议43rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020
国家/地区中国
Virtual, Online
时期25/07/2030/07/20

指纹

探究 'Investigating Reading Behavior in Fine-grained Relevance Judgment' 的科研主题。它们共同构成独一无二的指纹。

引用此

Wu, Z., Mao, J., Liu, Y., Zhang, M., & Ma, S. (2020). Investigating Reading Behavior in Fine-grained Relevance Judgment. 在 SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (页码 1889-1892). (SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval). Association for Computing Machinery, Inc. https://doi.org/10.1145/3397271.3401305