Notification: Our email services are now fully restored after a brief, temporary outage caused by a denial-of-service (DoS) attack. If you sent an email on Dec 6 and haven't received a response, please resend your email.
CFP last date
20 December 2024
Reseach Article

Keyphrase based Evaluation of Automatic Text Summarization

by Fatma Elghannam, Tarek El-shishtawy
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 117 - Number 7
Year of Publication: 2015
Authors: Fatma Elghannam, Tarek El-shishtawy
10.5120/20564-2953

Fatma Elghannam, Tarek El-shishtawy . Keyphrase based Evaluation of Automatic Text Summarization. International Journal of Computer Applications. 117, 7 ( May 2015), 5-8. DOI=10.5120/20564-2953

@article{ 10.5120/20564-2953,
author = { Fatma Elghannam, Tarek El-shishtawy },
title = { Keyphrase based Evaluation of Automatic Text Summarization },
journal = { International Journal of Computer Applications },
issue_date = { May 2015 },
volume = { 117 },
number = { 7 },
month = { May },
year = { 2015 },
issn = { 0975-8887 },
pages = { 5-8 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume117/number7/20564-2953/ },
doi = { 10.5120/20564-2953 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2024-02-06T22:58:40.804751+05:30
%A Fatma Elghannam
%A Tarek El-shishtawy
%T Keyphrase based Evaluation of Automatic Text Summarization
%J International Journal of Computer Applications
%@ 0975-8887
%V 117
%N 7
%P 5-8
%D 2015
%I Foundation of Computer Science (FCS), NY, USA
Abstract

The development of methods to deal with the informative contents of the text units in the matching process is a major challenge in automatic summary evaluation systems that use fixed n-gram matching. The limitation causes inaccurate matching between units in a peer and reference summaries. The present study introduces a new Keyphrase based Summary Evaluator (KpEval) for evaluating automatic summaries. The KpEval relies on the keyphrases since they convey the most important concepts of a text. In the evaluation process, the keyphrases are used in their lemma form as the matching text unit. The system was applied to evaluate different summaries of Arabic multi-document data set presented at TAC2011. The results showed that the new evaluation technique correlates well with the known evaluation systems: Rouge-1, Rouge-2, Rouge-SU4, and AutoSummENG–MeMoG. KpEval has the strongest correlation with AutoSummENG–MeMoG, Pearson and spearman correlation coefficient measures are 0. 8840, 0. 9667 respectively.

References
  1. El-Shishtawy, T. , & El-Ghannam, F. 2012. An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes. International Journal of Computer Science Issues (IJCSI), 9(1). ?
  2. El-Shishtawy, T. , & El-Ghannam, F. 2012, May. Keyphrase based Arabic summarizer (KPAS). Informatics and Systems (INFOS), 2012 8th International Conference. (pp. NLP-7. IEEE). ?
  3. Giannakopoulos, G. , Karkaletsis, V. , Vouros, G. , & Stamatopoulos, P. 2008. Summarization system evaluation revisited: N-gram graphs. ACM Transactions on Speech and Language Processing (TSLP), 5(3), 5. ?
  4. Hovy, E. , Lin, C. Y. , & Zhou, L. 2005. Evaluating duc 2005 using basic elements. In Proceedings of DUC (Vol. 2005). ?
  5. Hovy, E. , Lin, C. Y. , Zhou, L. , & Fukumoto, J. 2006, May. Automated summarization evaluation with basic elements. In Proceedings of the Fifth Conference on Language Resources and Evaluation (LREC 2006) (pp. 604-611). ?
  6. Jones, K. S. , & Galliers, J. R. (Eds. ). 1996. Evaluating natural language processing systems: An analysis and review (Vol. 1083). Springer. ?
  7. Lin, C. Y. 2004, July. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop (pp. 74-81). ?
  8. Lin, C. Y. , & Hovy, E. 2003, May. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1 (pp. 71-78). Association for Computational Linguistics. ?
  9. Papineni, K. , Roukos, S. , Ward, T. , & Zhu, W. J. 2002, July. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics. ?
  10. Tratz, S. , & Hovy, E. 2008. Summarization evaluation using transformed basic elements. In Proceedings of the 1st Text Analysis Conference. ?
Index Terms

Computer Science
Information Sciences

Keywords

Evaluating automatic text summarization keyphrase-based summary evaluation Summarization keyphrase extraction Arabic summary evaluation.