Diabetic retinopathy is a microvascular complication of diabetes mellitus that develops over time. Diabetic retinopathy is one of the retinal disorders. Early detection of diabetic retinopathy reduces the chances of permanent vision loss. However, the identification and regular diagnosis of diabetic retinopathy is a time-consuming task and requires expert ophthalmologists and radiologists. In addition, an automatic diabetic retinopathy detection technique is necessary for real-time applications to facilitate and minimize potential human errors. Therefore, we propose an ensemble deep neural network and a novel four-step feature selection technique in this paper. In the first step, the preprocessed entropy images improve the quality of the retinal features. Second, the features are extracted using a deep ensemble model include InceptionV3, ResNet101, and Vgg19 from the retinal fundus images. Then, these features are combined to create an ample feature space. To reduce the feature space, we propose four-step feature selection techniques: minimum redundancy, maximum relevance, Chi-Square, ReliefF, and F test for selecting efficient features. Further, appropriate features are chosen from the majority voting techniques to reduce the computational complexity. Finally, the standard machine learning classifier, support vector machines, is used in diabetic retinopathy classification. The proposed method is tested on Kaggle, MESSIDOR-2, and IDRiD databases, available publicly. The proposed algorithm provided an accuracy of 97.78%, a sensitivity of 97.6%, and a specificity of 99.3%, using top 300 features, which are better than other state-of-the-art methods.