Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches

Loading...
Thumbnail Image
File version

Accepted Manuscript (AM)

Author(s)
Chen, Z
Wang, S
Li, J
Huang, Z
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2020
Size
File type(s)
Location

Seattle, WA, United States / Virtual

License
Abstract

Zero-shot learning (ZSL) is commonly used to address the very pervasive problem of predicting unseen classes in fine-grained image classification and other tasks. One family of solutions is to learn synthesised unseen visual samples produced by generative models from auxiliary semantic information, such as natural language descriptions. However, for most of these models, performance suffers from noise in the form of irrelevant image backgrounds. Further, most methods do not allocate a calculated weight to each semantic patch. Yet, in the real world, the discriminative power of features can be quantified and directly leveraged to improve accuracy and reduce computational complexity. To address these issues, we propose a novel framework called multi-patch generative adversarial nets (MPGAN) that synthesises local patch features and labels unseen classes with a novel weighted voting strategy. The process begins by generating discriminative visual features from noisy text descriptions for a set of predefined local patches using multiple specialist generative models. The features synthesised from each patch for unseen classes are then used to construct an ensemble of diverse supervised classifiers, each corresponding to one local patch. A voting strategy averages the probability distributions output from the classifiers and, given that some patches are more discriminative than others, a discrimination-based attention mechanism helps to weight each patch accordingly. Extensive experiments show that MPGAN has significantly greater accuracy than state-of-the-art methods.

Journal Title
Conference Title

MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia

Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement

© ACM, 2021. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in MM '20: Proceedings of the 28th ACM International Conference on Multimedia, ISBN: 978-1-4503-7988-5, https://doi.org/10.1145/3394171.3413813

Item Access Status
Note
Access the data
Related item(s)
Subject

Nanotechnology

Persistent link to this record
Citation

Chen, Z; Wang, S; Li, J; Huang, Z, Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches, MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 3413-3421