Distributed Feature Selection for Big Data Using Fuzzy Rough Sets
File version
Author(s)
Qu, Wenhao
Yu, Jiadi
Zuo, Hua
Chen, Guihai
Xiong, Fei
Pan, Shirui
Lin, Siyu
Qiu, Meikang
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
License
Abstract
Fuzzy rough-set-based feature selection is an important technique for big data analysis. However, the classic fuzzy rough set algorithm takes all the data correlations into account, which leads to the centralized computing mode, requiring high computing and memory space resources. With the increasing amount of data in the big data era, the centralized server cannot afford the computation of fuzzy rough set. To enable the fuzzy rough set for big data analysis, in this article, we propose the novel distributed fuzzy rough set (DFRS)-based feature selection, which separates and assigns the tasks to multiple nodes for parallel computing. The key challenge is to maintain the global information on each distributed node without conserving the entire fuzzy relation matrix. We tackle this challenge by a dynamic data decomposition algorithm and a data summarization process on each distributed node. Extensive experiments based on multiple real datasets demonstrate that DFRS significantly improves the runtime, and its feature selection accuracy is nearly the same as the traditional centralized computing.
Journal Title
IEEE Transactions on Fuzzy Systems
Conference Title
Book Title
Edition
Volume
28
Issue
5
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject
Applied mathematics
Artificial intelligence
Electrical engineering
Science & Technology
Computer Science, Artificial Intelligence
Engineering, Electrical & Electronic
Persistent link to this record
Citation
Kong, L; Qu, W; Yu, J; Zuo, H; Chen, G; Xiong, F; Pan, S; Lin, S; Qiu, M, Distributed Feature Selection for Big Data Using Fuzzy Rough Sets, IEEE Transactions on Fuzzy Systems , 2020, 28 (5), pp. 846-857