Deep Feature Learning for Spectral-Spatial Classification of Hyperspectral Remote Sensing Images

Abstract

The recent advances in aerial- and satellite-based hyperspectral imaging sensor technologies have led to an increased availability of Earth's images with high spatial and spectral resolution, which opened the door to a large range of important applications. Hyperspectral imaging records detailed spectrum of the received light in each spatial position in the image, in which each pixel contains a highly detailed representation of the reflectance of the materials present on the ground, and a better characterization in terms of geometrical details. Since different substances exhibit different spectral signatures, the abundance of informative content conveyed in the hyperspectral images permits an improved characterization of different land coverage. Therefore, hyperspectral imaging emerged as a well-suited technology for accurate image classi fication in remote sensing. In spite of that, a signi ficantly increased complexity of the analysis introduces a series of challenges that need to be addressed on a serious note. In order to fully exploit the potential offered by these sensors, there is a need to develop accurate and effective models for spectral-spatial analysis of the recorded data. This thesis aims at presenting novel strategies for the analysis and classifi cation of hyperspectral remote sensing images, placing the focal point on the investigation on deep networks for the extraction and integration of spectral and spatial information. Deep learning has demonstrated cutting-edge performances in computer vision, particularly in object recognition and classi cation. It has also been successfully adopted in hyperspectral remote sensing domain as well. However, it is a very challenging task to fully utilize the massive potential of deep models in hyperspectral remote sensing applications since the number of training samples is limited which limits the representation capability of a deep model. Furthermore, the existing architectures of deep models need to be further investigated and modifi ed accordingly to better complement the joint use of spectral and spatial contents of hyperspectral images. In this thesis, we propose three different deep learning-based models to effectively represent spectral-spatial characteristics of hyperspectral data in the interest of classifi cation of remote sensing images. Our first proposed model focuses on integrating CRF and CNN into an end-to-end learning framework for classifying images. Our main contribution in this model is the introduction of a deep CRF in which the CRF parameters are computed using CNN and further optimized by adopting piecewise training. Furthermore, we address the problem of over fitting by employing data augmentation techniques and increased the size of the training samples for training deep networks. Our proposed 3DCNN-CRF model can be trained to fully exploit the usefulness of CRF in the context of classi fication by integrating it completely inside of a deep model. Considering that the separation of constituent materials and their abundances provide detailed analysis of the data, our second algorithm investigates the potential of using unmixing results in deep models to classify images. We extend an existing region based structure preserving non-negative matrix factorization method to estimate groups of spectral bands with the goal to capture subtle spectral-spatial distribution from the image. We subsequently use these important unmixing results as input to generate superpixels, which are further represented by kernel density estimated probability distribution function. Finally, these abundance information-guided superpixels are directly supplied into a deep model in which the inference is implicitly formulated as a recurrent neural network to perform the eventual classifi cation. Finally, we perform a detailed investigation on the possibilities of adopting generative adversarial models into hyperspectral image classifi cation. We present a GAN-based spectral-spatial method that primarily focuses on signifi cantly improving the multiclass classi cation ability of the discriminator of GAN models. In this context, we propose to adopt the triplet constraint property and extend it to build a useful feature embedding for remote sensing images for use in classi cation. Furthermore, our proposed Triplet- 3D-GAN model also includes feedback from discriminator's intermediate features to improve the quality of the generator's sample generation process.