Towards Relevance and Diversity in Crowdsourcing Worker Recruitment With Insufficient Information

Nowadays, crowdsourcing has become an increasingly popular paradigm in real-world applications. It provides a fundamental mechanism for inviting thoughts and perspectives from a wide swath of people. One of the critical issues in crowdsourcing lies in worker recruitment. However, the existing literature mainly focuses on maximizing the relevance between workers and tasks, ignoring the diversity among workers. Such neglect may lead to poor quality of task completion. In this article, we present an online worker recruitment mechanism with relevance and diversity. To comprehensively evaluate the capability of a worker group, we propose an enhanced metric called utility. We also consider a practically motivated setting in which the observed side-information of workers is insufficient due to privacy concerns. Then, we utilize contextual combinatorial multi-armed bandit to model the online worker selection process, and propose a novel Diversified Hidden Upper Confidence Bound (DH-UCB) algorithm to address the problem. Most importantly, we rigorously prove that our DH-UCB algorithm achieves significant sublinear upper regret bound with high probability. In other words, it can effectively recruit workers under insufficient information while maintaining sustainable relevance and diversity. Extensive experiments on both synthetic and large-scale real-world datasets empirically validate the advantages of our proposed mechanism.