Dataset with imbalanced class distribution used to abate classification performance for most of the standard classifier learning algorithms. Moreover, some application area consists of scarcity of labeled training data where clustering is most prominent way to support classification process. Gaussian Mixture Model (GMM) being able to approximate arbitrary probability distribution, is a dominant tool for classification in such cases by means of clustering. An ensemble approach is presented in this paper considering GMM as a weak learner to boost the GMMs in a semi supervised manner via Adaptive Boosting technique. This paper, firstly investigates how much K-means and GMM suffers from uneven class distribution in data. Later experiment on benchmark imbalanced datasets with different imbalance ratio and over sampled datasets using Synthetic Minority Over-sampling Technique (SMOTE) has been carried out for proposed approach. For each case cluster forest has been used as an attribute selection technique. Efficacy of the proposed Boosted GMM approach compared to standard clustering approaches like K means and GMM is exhibited from empirical analysis.