Fusion of Hyperspectral and LiDAR Data for Land Cover Classification

Abstract

Land cover classification has become increasingly important for making the plan to overcome the problems of disorganized and uncontrolled development, the disappearance of prime agricultural lands and deteriorating environmental quality by losing forest, wildlife habitat, wetlands etc. Different remote sensing technologies capture different properties e.g., spectral, shape, etc of ground objects. Nowadays, combined use of multiple remote sensing technologies for land cover classification becomes popular. Spectral image e.g., hyperspectral and lidar point cloud data are commonly used in land cover classification. Among the spectral images, the hyperspectral image contains detailed spectral responses of an object. On the contrary, light detection and ranging (LiDAR) data capture structural information of an object. Thus hyperspectral and LiDAR complement each other by accumulating information from land cover. Several state-of-the-art methods were developed for fusing hyperspectral and LiDAR data for land cover classification where the methods included feature extraction, feature fusion and classification. Still, there are undiscovered properties of both modalities which can contribute significantly in this domain. In this thesis, we discover a number of effective ways for feature extraction from both hyperspectral and LiDAR data. Furthermore, we propose two feature fusion techniques which are able to decrease between-class correlation and increase within-class correlation while fusing features from two modalities. Finally, a decision fusion approach e.g., ensemble classification is incorporated for integrating prediction metrics. In this thesis, we propose three different approaches for separating complex land cover classes by fusing hyperspectral and LiDAR data. The effectiveness of these approaches is validated by experimenting on two datasets e.g., Houston and GU datasets. The Houston dataset is a benchmark dataset that contains fifteen landcover classes and distributed in 2013 IEEE GRSS Data Fusion Contest. On the other hand, GU dataset consists of land cover classes and is prepared from the hyperspectral and LiDAR data collected by the Spectral Imaging Lab of Griffith University. We use two state-of-the-art classifiers e.g., random forest (RF) and support vector machine (SVM) for classifying the features derived by our proposed approaches. In our first approach, we derive eight features from hyperspectral and LiDAR data. Among them two are from hyperspectral and six are from LiDAR data. These eight features show perfect complement property to hyperspectral features. In feature fusion, we explore the effectiveness of layer stacking and principal component analysis (PCA) where effective combination of features is investigated specially for PCA fusion. In our second approach, we integrate three key tasks e.g., band-group fusion, multisource fusion and generic feature (GF) extraction. In band-group fusion, we group hyperspectral bands based on their joint entropy and structural similarity. We apply PCA on each group and retain a few principal components and apply differential attribute profiles (DAP) for extracting spatial features. The spatial and spectral features from individual groups are fused using discriminant correlation analysis (DCA). In multisource fusion, spatial features from hyperspectral and LiDAR are also fused by DCA.We derive eight pixel-wise GF from hyperspectral and LiDAR data which are then arranged sequentially to form an additional feature vector. Finally, we concatenate the features generated by band-group fusion, multi-source fusion and generic feature extraction steps to get a final signature. In our third approach, we propose a novel feature extraction technique named inverse coeffcient of variation (ICV) which explores the Gaussian probability of neighbourhood between every pair of bands in hyperspectral data. We calculate ICV for each band with respect to every other band and form an ICV cube. We derive spatial features (e.g. DAP) from the first few principal components of both hyperspectral and ICV cube. In addition, we derive GF from both hyperspectral and LiDAR data and then spatial features from GF. Secondly, we propose a two-stream fusion approach where canonical correlation analysis (CCA) is used as a basic fusion unit. In one stream pair-wise CCA fusion of spectral features of hyperspectral with spatial features of both hyperspectral and LiDAR takes place. In the other stream, pair-wise CCA fusion of ICV features with spatial features derived from ICV, hyperspectral and LiDAR are performed. Thirdly, an ensemble classification system is designed for decision fusion where features from twostream fusion are distributed into random subsets, and then each subset is transformed for improving feature quality, all are concatenated and classified. This process is executed for several iteration. The final classification results are obtained by weighting and aggregating the prediction metrics given by RF or applying majority voting on the predicted classes given by SVM.