Call for Paper - November 2023 Edition
IJCA solicits original research papers for the November 2023 Edition. Last date of manuscript submission is October 20, 2023. Read More

An Ensemble Approach to Classifier Construction based on Bootstrap Aggregation

International Journal of Computer Applications
© 2011 by IJCA Journal
Number 5 - Article 4
Year of Publication: 2011
Dewan Md. Farid
Mohammad Zahidur Rahman
Chowdhury Mofizur Rahman

Dewan Md. Farid, Mohammad Zahidur Rahman and Chowdhury Mofizur Rahman. Article: An Ensemble Approach to Classifier Construction based on Bootstrap Aggregation. International Journal of Computer Applications 25(5):30-34, July 2011. Full text available. BibTeX

	author = {Dewan Md. Farid and Mohammad Zahidur Rahman and Chowdhury Mofizur Rahman},
	title = {Article: An Ensemble Approach to Classifier Construction based on Bootstrap Aggregation},
	journal = {International Journal of Computer Applications},
	year = {2011},
	volume = {25},
	number = {5},
	pages = {30-34},
	month = {July},
	note = {Full text available}


In this paper, we introduce a new approach to the classification of streaming data based on bootstrap aggregation (bagging). The proposed approach creates an ensemble model by using ID3 classifier, naïve Bayesian classifier, and k-Nearest-Neighbor classifier for a learning scheme where each classifier gives the weighted prediction. ID3, naïve Bayesian, and k-Nearest-Neighbor classifiers are very well known data mining methods, which have been already used in many real life classification problems. The proposed approach addresses the practical problems of the classification of streaming data and successfully tested on a number of benchmark problems including large intrusion detection dataset from the UCI machine learning repository to produce a comparison with the established approaches. The experimental results demonstrate that the proposed ensemble classifier achieved high classification rates and generated very low misclassification error.


  • L. Breiman, “Bagging predictors,” Machine Learning, Vol. 24, 1996, pp. 123-140.
  • B. Efron, and R. Tibshirani, “An Introduction to the Bootstrap,” Chapman & Hall, 1993.
  • R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” In Proc. of the 14th Joint International Conference Artificial Intelligence (IJCAI’95), Vol. 2, August 1995, Montreal, Canada, pp. 1137-1143.
  • U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “From data mining to knowledge discovery: An overview,” In Advances in Knowledge Discovery and Data Mining, MIT Press, 1996.
  • S. Pang, S. Ozawa, and N. Kasabov, “Incremental linear discriminant analysis for classification of data streams,” IEEE Trans. Syst. Man Cybern. B, Cybern., Vol. 35, No. 5, Oct. 2005, pp. 905-914.
  • R. Jin, and G. Agrawal, “Efficient decision tree construction on streaming data,” in Proc. ACM SIGKDD, 2003, pp. 571-576.
  • L. Breiman, J. Friedman, C. Stone, and R. Olshen, “Classification and Regression Trees,” Boca Raton, Fl: Chapman & Hall, 1993.
  • K. M. A. Chai, H. T. Ng, and H. L. Chieu, “Bayesian online classifiers for text classification and filtering,” in Proc. SIGIR 2002, Tampere, Finland, Aug. 11-15, pp. 97-104.
  • C. M. Bishop, “Neural Networks for Pattern Recognition,” Oxford, U.K.: Oxford University Press, 1995.
  • The Archive UCI Machine Learning Datasets.
  • J. R. Quinlan, “Induction of Decision Tree,” Machine Learning Vol. 1, pp. 81-106, 1986.
  • P. Langley, “Induction of recursive Bayesian classifier,” In Proc. of the European Conference on Machine Learning, 1993, pp. 153-164.
  • P. Langely, W. Iba, and K. Thomas, “An analysis of Bayesian classifier,” In Proc. of the 10th National Conference on Artificial Intelligence, San Mateo, CA: AAAI Press, 1992, pp. 223-228.
  • Z. Zheng, and I. G. Webb, “Lazy learning of Bayesian roles,” Machine Learning-1, Kluwer Academic Publishers, Boston, 2000, pp. 1-35.
  • B. V. Dasarathy, “Nearest Neighor (NN) Norms: NN Pattern Classification Techniques,” IEEE Computer Society Press, 1991,
  • Duda, R., P.E. Hart, and D.G. Stork, “Pattern classification,” Second edn. John Wiley & Sons, 2001.
  • The KDD Archive. KDD99 cup dataset, 1999.