In heterogeneous ensemble systems, each learning algorithm learns a classifier on a given training set to describe the relationship between a feature vector and its class label. As each classifier outputs different result on an observation, uncertainty is introduced. In this paper, we introduce a heterogeneous ensemble system with a fuzzy IF-THEN rule inference engine as the combiner to capture the uncertainty in the outputs of the base classifiers. In our method, fuzzy rules are generated on the outputs of an ensemble of base classifiers, which can be viewed as the class posterior probability of the observations. The performance of our method was evaluated on thirty datasets and in comparison with nine ensemble methods (AdaBoost, Decision Template, Decision Tree on meta-data, and six fixed combiners) and two single learning algorithms (SVM with L2-loss function and Decision Tree), and was shown to significantly outperforms these algorithms in terms of classification accuracy.