An online statistical quality control framework for performance management in crowdsourcing
File version
Author(s)
Hussain, Omar K
Chang, Elizabeth
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
Leipzig, Germany
License
Abstract
The big data research topic has grown rapidly for the past decade due to the advent of the "data deluge". Recent advancements in the literature leverage human computing power known as crowdsourcing to manage and harness big data for various applications. However, human involvement in the completion of crowdsourcing tasks is an error-prone process that affects the overall performance of the crowd. Thus, controlling the quality of workers is an essential step for crowdsourcing systems, which due to unavailability of ground-truth data for any task at hand becomes increasingly challenging. To propose a solution to this problem, in this study, we propose OSQC (Online Statistical Quality Control Framework) for managing the performance of workers in crowdsourcing. OSQC ascertains the worker's performance by using a statistical model and then leverages the traditional statistical control techniques to decide whether to retain a worker for crowdsourcing or to evict him. We evaluate our proposed framework on a real dataset and demonstrate how OSQC assists crowdsourcing to maintain its accuracy.
Journal Title
Conference Title
WI '17: Proceedings of the International Conference on Web Intelligence
Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject
Science & Technology
Technology
Computer Science, Artificial Intelligence
Computer Science, Information Systems
Computer Science
Persistent link to this record
Citation
Saberi, M; Hussain, OK; Chang, E, An online statistical quality control framework for performance management in crowdsourcing, WI '17: Proceedings of the International Conference on Web Intelligence, 2017, pp. 476-482