Show simple item record

dc.contributor.authorNguyen, Quoc Viet Hung
dc.contributor.authorHuynh, Huu Viet
dc.contributor.authorNguyen, Thanh Tam
dc.contributor.authorWeidlich, Matthias
dc.contributor.authorYin, Hongzhi
dc.contributor.authorZhou, Xiaofang
dc.date.accessioned2018-08-08T01:30:32Z
dc.date.available2018-08-08T01:30:32Z
dc.date.issued2018
dc.identifier.issn1041-4347
dc.identifier.doi10.1109/TKDE.2017.2750683
dc.identifier.urihttp://hdl.handle.net/10072/347582
dc.description.abstractCrowdsourcing has been widely established as a means to enable human computation at large-scale, in particular for tasks that require manual labelling of large sets of data items. Answers obtained from heterogeneous crowd workers are aggregated to obtain a robust result.However, existing methods for answer aggregation are designed for discrete tasks, where answers are given as a single label per item. In this paper, we consider partial-agreement tasks that are common in many applications such as image tagging and document annotation, where items are assigned sets of labels. Common approaches for the aggregation of partial-agreement answers either (i) reduce the problem to several instances of an aggregation problem for discrete tasks or (ii) consider each label independently. Going beyond the state-of-the-art, we propose a novel Bayesian nonparametric model to aggregate the partial-agreement answers in a generic way. This model enables us to compute the consensus of partially-sound and partially-complete worker answers. We also show how this model is instantiated for incremental learning, incorporating new answers from crowd workers as they arrive. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of precision, recall, and robustness against faulty workers and data sparsity.
dc.description.peerreviewedYes
dc.languageEnglish
dc.language.isoeng
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartofpagefrom1
dc.relation.ispartofpageto14
dc.relation.ispartofjournalIEEE Transactions on Knowledge and Data Engineering
dc.subject.fieldofresearchInformation and computing sciences
dc.subject.fieldofresearchDatabase systems
dc.subject.fieldofresearchcode46
dc.subject.fieldofresearchcode460505
dc.titleComputing Crowd Consensus with Partial Agreement
dc.typeJournal article
dc.type.descriptionC1 - Articles
dc.type.codeC - Journal Articles
dc.description.versionAccepted Manuscript (AM)
gro.facultyGriffith Sciences, School of Information and Communication Technology
gro.description.notepublicThis publication has been entered into Griffith Research Online as an Advanced Online Version.
gro.rights.copyright© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
gro.hasfulltextFull Text
gro.griffith.authorNguyen, Henry
gro.griffith.authorNguyen, Thanh Tam


Files in this item

This item appears in the following Collection(s)

  • Journal articles
    Contains articles published by Griffith authors in scholarly journals.

Show simple item record