An automatic building detection technique using LIDAR data and multispectral imagery has been proposed. Two masks are obtained from the LIDAR data: a 'primary building mask' and a 'secondary building mask'. The primary building mask indicates the void areas where the laser does not reach below a certain height threshold. The secondary building mask indicates the filled areas, from where the laser reflects, above the same threshold. Line segments are extracted from around the void areas in the primary building mask. Line segments around trees are removed using the normalized difference vegetation index derived from the orthorectified multispectral images. The initial building positions are obtained based on the remaining line segments. The complete buildings are detected from their initial positions using the two masks and multispectral images in the YIQ colour system. It is experimentally shown that the proposed technique can successfully detect buildings, when assessed in terms of 15 indices including completeness, correctness and quality.