Exploration of quantum interference in document relevance judgement discrepancy

Quantum theory has been applied in a number of fields outside physics, e.g., cognitive science and information retrieval (IR). Recently, it has been shown that quantum theory can subsume various key IR models into a single mathematical formalism of Hilbert vector spaces. While a series of quantum-inspired IR models has been proposed, limited effort has been devoted to verify the existence of the quantum-like phenomenon in real users' information retrieval processes, from a real user study perspective. In this paper, we aim to explore and model the quantum interference in users' relevance judgement about documents, caused by the presentation order of documents. A user study in the context of IR tasks have been carried out. The existence of the quantum interference is tested by the violation of the law of total probability and the validity of the order effect. Our main findings are: (1) there is an apparent judging discrepancy across different users and document presentation orders, and empirical data have violated the law of total probability; (2) most search trials recorded in the user study show the existence of the order effect, and the incompatible decision perspectives in the quantum question (QQ) model are valid in some trials. We further explain the judgement discrepancy in more depth, in terms of four effects (comparison, unfamiliarity, attraction and repulsion) and also analyse the dynamics of document relevance judgement in terms of the evolution of the information need subspace.