TY - GEN
T1 - G-CoS
T2 - 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2026
AU - Shi, Jia Ling
AU - Wu, Zhijing
AU - Liang, Yidong
AU - Mao, Xian Ling
N1 - Publisher Copyright:
© 2026 Owner/Author.
PY - 2026/7/19
Y1 - 2026/7/19
N2 - User satisfaction serves as a key indicator of search experience in Generative Information Retrieval (GenIR) systems, and its accurate estimation is essential for system optimization. Extensive research in traditional IR has established interaction signals (e.g., dwell time, clicks, query reformulations) as reliable indicators of user satisfaction. However, prevailing approaches for satisfaction estimation in GenIR (e.g., LLM-as-a-judge) primarily rely on textual content and fail to account for user interactions during the search process. In this work, empirical analysis of real-world data shows that user satisfaction correlates negatively with interaction signals reflecting interaction cost, and positively with response quality. Building on these findings, we propose the Gain-aware Cost-sensitive Satisfaction estimator (G-CoS), an interpretable gain-cost framework for user satisfaction estimation in GenIR. G-CoS models user satisfaction as a dynamic trade-off between Response Quality and multidimensional Interaction Cost. Experimental results demonstrate that G-CoS outperforms LLM-as-a-judge methods, interaction sequence models, as well as machine learning models using the same gain and cost features. Moreover, the learned parameters reveal interpretable associations between gain-cost dynamics and user satisfaction. This work contributes an interpretable framework for user satisfaction estimation and offers insights for GenIR system optimization.
AB - User satisfaction serves as a key indicator of search experience in Generative Information Retrieval (GenIR) systems, and its accurate estimation is essential for system optimization. Extensive research in traditional IR has established interaction signals (e.g., dwell time, clicks, query reformulations) as reliable indicators of user satisfaction. However, prevailing approaches for satisfaction estimation in GenIR (e.g., LLM-as-a-judge) primarily rely on textual content and fail to account for user interactions during the search process. In this work, empirical analysis of real-world data shows that user satisfaction correlates negatively with interaction signals reflecting interaction cost, and positively with response quality. Building on these findings, we propose the Gain-aware Cost-sensitive Satisfaction estimator (G-CoS), an interpretable gain-cost framework for user satisfaction estimation in GenIR. G-CoS models user satisfaction as a dynamic trade-off between Response Quality and multidimensional Interaction Cost. Experimental results demonstrate that G-CoS outperforms LLM-as-a-judge methods, interaction sequence models, as well as machine learning models using the same gain and cost features. Moreover, the learned parameters reveal interpretable associations between gain-cost dynamics and user satisfaction. This work contributes an interpretable framework for user satisfaction estimation and offers insights for GenIR system optimization.
KW - generative information retrieval
KW - interaction signals
KW - interpretable modeling
KW - user satisfaction estimation
UR - https://www.scopus.com/pages/publications/105047256590
U2 - 10.1145/3805712.3809934
DO - 10.1145/3805712.3809934
M3 - Conference contribution
AN - SCOPUS:105047256590
T3 - SIGIR 2026 - Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
SP - 4110
EP - 4115
BT - SIGIR 2026 - Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
PB - Association for Computing Machinery, Inc
Y2 - 20 July 2026 through 24 July 2026
ER -