The periocular region has recently emerged as a standalone biometric trait, promising attractive tradeoff between the iris alone and the entire face, especially for cases where neither the iris nor a full facial image can be acquired. This advantage provides another dimension for implementing a robust biometric system performed in non-ideal conditions. Global features [local binary pattern (LBP), Histogram of Gradient (HOG)] and local features have been introduced; however, the performance of these features can deteriorate for images captured in unconstrained and less-cooperative conditions. A particular set of higher order spectral (HOS) features have been proved to be invariant to translation, scale, rotation, brightness level shift, and contrast change. These properties are desirable in the periocular recognition problem to deal with the non-ideal imaging conditions. This paper investigates the HOS features in different configurations for the periocular recognition problem under non-ideal conditions. Specifically, we introduce a new sampling approach for the periocular region based on an elliptical coordinate. This non-linear sampling approach is then combined with the robustness of the HOS features for encoding the periocular region. In addition, we also propose a new technique for combining left and right perioculars. The proposed feature-level fusion approach is based on the state-of-the-art bilinear pooling technique to allow efficient interaction between the features of both perioculars. We show the validity of the proposed approach in encoding discriminant features outperforming or comparing favorably with the state-of-the-art features on the two popular data sets: Face Recognition Grand Challenge and Japanese Female Facial Expression.