dc.contributor.author | Nguyen, Quoc Viet Hung | |
dc.contributor.author | Huynh, Huu Viet | |
dc.contributor.author | Nguyen, Thanh Tam | |
dc.contributor.author | Weidlich, Matthias | |
dc.contributor.author | Yin, Hongzhi | |
dc.contributor.author | Zhou, Xiaofang | |
dc.date.accessioned | 2018-08-08T01:30:32Z | |
dc.date.available | 2018-08-08T01:30:32Z | |
dc.date.issued | 2018 | |
dc.identifier.issn | 1041-4347 | |
dc.identifier.doi | 10.1109/TKDE.2017.2750683 | |
dc.identifier.uri | http://hdl.handle.net/10072/347582 | |
dc.description.abstract | Crowdsourcing has been widely established as a means to enable human computation at large-scale, in particular for tasks that require manual labelling of large sets of data items. Answers obtained from heterogeneous crowd workers are aggregated to obtain a robust result.However, existing methods for answer aggregation are designed for discrete tasks, where answers are given as a single label per item. In this paper, we consider partial-agreement tasks that are common in many applications such as image tagging and document annotation, where items are assigned sets of labels. Common approaches for the aggregation of partial-agreement answers either (i) reduce the problem to several instances of an aggregation problem for discrete tasks or (ii) consider each label independently. Going beyond the state-of-the-art, we propose a novel Bayesian nonparametric model to aggregate the partial-agreement answers in a generic way. This model enables us to compute the consensus of partially-sound and partially-complete worker answers. We also show how this model is instantiated for incremental learning, incorporating new answers from crowd workers as they arrive. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of precision, recall, and robustness against faulty workers and data sparsity. | |
dc.description.peerreviewed | Yes | |
dc.language | English | |
dc.language.iso | eng | |
dc.publisher | Institute of Electrical and Electronics Engineers (IEEE) | |
dc.relation.ispartofpagefrom | 1 | |
dc.relation.ispartofpageto | 14 | |
dc.relation.ispartofjournal | IEEE Transactions on Knowledge and Data Engineering | |
dc.subject.fieldofresearch | Information and computing sciences | |
dc.subject.fieldofresearch | Database systems | |
dc.subject.fieldofresearchcode | 46 | |
dc.subject.fieldofresearchcode | 460505 | |
dc.title | Computing Crowd Consensus with Partial Agreement | |
dc.type | Journal article | |
dc.type.description | C1 - Articles | |
dc.type.code | C - Journal Articles | |
dc.description.version | Accepted Manuscript (AM) | |
gro.faculty | Griffith Sciences, School of Information and Communication Technology | |
gro.description.notepublic | This publication has been entered into Griffith Research Online as an Advanced Online Version. | |
gro.rights.copyright | © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |
gro.hasfulltext | Full Text | |
gro.griffith.author | Nguyen, Henry | |
gro.griffith.author | Nguyen, Thanh Tam | |