TY - GEN
T1 - A Flexible User Study Platform for Generative Information Retrieval
AU - Liang, Yidong
AU - Wu, Zhijing
AU - He, Yuchen
AU - Liang, Fengming
AU - Liu, Kexin
AU - Mao, Jiaxin
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/7/13
Y1 - 2025/7/13
N2 - User behavior and experience are important for improving information retrieval (IR) systems. While much research has focused on traditional IR systems, few studies have systematically examined user behavior and search experience with emerging generative IR systems. A key reason for this gap is the lack of publicly available toolkits to record user behavior and feedback in generative IR systems. We developed a comprehensive platform to collect user behavior and feedback on the generative IR system. This platform consists of: 1) a generative IR system that supports both API-based and customized retrieval-augmented generation (RAG) methods, 2) a user interface that logs various user behavior, including prompts, clicks, mouse movements, and scrolling, and 3) an annotation website that allows users to provide feedback. We believe the proposed platform has the potential to streamline data collection for user studies on generative IR systems, paving the way for future research on how users engage with and interact with these systems.
AB - User behavior and experience are important for improving information retrieval (IR) systems. While much research has focused on traditional IR systems, few studies have systematically examined user behavior and search experience with emerging generative IR systems. A key reason for this gap is the lack of publicly available toolkits to record user behavior and feedback in generative IR systems. We developed a comprehensive platform to collect user behavior and feedback on the generative IR system. This platform consists of: 1) a generative IR system that supports both API-based and customized retrieval-augmented generation (RAG) methods, 2) a user interface that logs various user behavior, including prompts, clicks, mouse movements, and scrolling, and 3) an annotation website that allows users to provide feedback. We believe the proposed platform has the potential to streamline data collection for user studies on generative IR systems, paving the way for future research on how users engage with and interact with these systems.
KW - Generative Information Retrieval Systems
KW - User Behavior
KW - User Study Platform
UR - https://www.scopus.com/pages/publications/105011817872
U2 - 10.1145/3726302.3730140
DO - 10.1145/3726302.3730140
M3 - Conference contribution
AN - SCOPUS:105011817872
T3 - SIGIR 2025 - Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval
SP - 4066
EP - 4070
BT - SIGIR 2025 - Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval
PB - Association for Computing Machinery, Inc
T2 - 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2025
Y2 - 13 July 2025 through 18 July 2025
ER -