A Flexible User Study Platform for Generative Information Retrieval

User behavior and experience are important for improving information retrieval (IR) systems. While much research has focused on traditional IR systems, few studies have systematically examined user behavior and search experience with emerging generative IR systems. A key reason for this gap is the lack of publicly available toolkits to record user behavior and feedback in generative IR systems. We developed a comprehensive platform to collect user behavior and feedback on the generative IR system. This platform consists of: 1) a generative IR system that supports both API-based and customized retrieval-augmented generation (RAG) methods, 2) a user interface that logs various user behavior, including prompts, clicks, mouse movements, and scrolling, and 3) an annotation website that allows users to provide feedback. We believe the proposed platform has the potential to streamline data collection for user studies on generative IR systems, paving the way for future research on how users engage with and interact with these systems.