This paper presents a novel image descriptor called Derivative Variation Pattern (DVP) and its application to face and palmprint recognition. DVP captures image variations in both the frequency and the spatial domains. The effects of uncontrolled illumination are compensated in the frequency domain by discarding the illumination affected frequencies. Image pixels are encoded as binary patterns based on the higher-order spatial derivatives computed in the spatial domain. The proposed descriptor was evaluated on the Extended Yale-B and FERET face databases, and the PolyU palmprint database. Experimental results demonstrate the effectiveness of the DVP descriptor in both the face and the palmprint recognition tasks under uncontrolled illuminations.