The most challenging issue in constructing a classification system based on ensemble is how to construct an appropriate ensemble of basic classifiers. In this paper, a new approach of constructing ensemble namely, KSBC, K-means Based Classifier Selection is introduced. This approach utilizes Bagging algorithm as the producer of basic classifiers. Type of all basic classifiers, decision tree or multi-layer nervous networks are considered and remained unchanged during construction of ensemble. After constructing a large number of basic classifiers, KSBC partitions them with the help of k-means clustering. Afterward, by choosing one classifier from each partition, final ensemble is constructed. Weight voting method is the assembling function of the ensemble. In addition the approaches for selection of a classifier from each partition were analyzed. The effectiveness of sampling rate on the efficiency of combining classification based on clustering of classifiers was also evaluated. Finally, several tests were carried out on a large number of standard datasets in machine learning database. Experimental results illustrate the effectiveness of the proposed method compared to other approaches.

Ensemble method bagging clustering