Online Crowd Learning Through Strategic Worker Reports

When it is difficult to verify contributed solutions in mobile crowdsourcing, the majority voting mechanism is widely utilized to incentivize distributed workers to provide high-quality and truthful solutions. In the majority voting mechanism, a worker is rewarded based on whether his solution is consistent with the majority. However, most prior related work relies on a strong assumption that workers' solution accuracy levels are public knowledge, which may not hold in many practical scenarios. We relax such an assumption and propose an online mechanism, which allows the platform to learn the distribution of the workers' solution accuracy levels via asking workers to report their private accuracy levels (which do not need to be the true values), in addition to deciding their effort levels and solution reporting strategies. The mechanism design is challenging, as neither the workers' task solutions nor their accuracy reports can be verified. We devise a randomized reward mechanism that computes the workers' rewards based on their reported accuracy levels, under which the workers obtain rewards if their reported solutions match the majority. We show that our mechanism induces workers to truthfully report their solution accuracy levels in the long run, in which the empirical accuracy distribution (collected from workers' accuracy reports) converges to the actual accuracy distribution. Moreover, we show that our online mechanism converges faster when the workers are more capable of solving the tasks.