A new generation of attacks called as polymorphic attacks - where malware repeatedly mutates to deceive regular malware detection - are continuing to drive the growth in complexity of malware. Polymorphic malwares are using far more sophisticated approaches that may include editing its own source code to avoid signature-based detection. There is increasing necessity to handle this level of unprecedented polymorphism. Especially, scripts have been exploited to widespread polymorphic malwares. In this paper, we propose a modified Hybrid detection model based dependency analysis. Every script malware can be represented by a dependency graph and then the detection can be transformed to the problem finding maximum subgraph isomorphism in that polymorphism still maintains the core of logical structures of malwares. We also present threshold selection and priority level management approaches which can be used to improve detection accuracy and reduce computational cost.

1. A Survey on Techniques in Detection and Analyzing Malware Executables , IJARCSSE Volume 3, Issue 4, April 2013
2. J. Aycock. Computer Viruses and Malware. Springer, 2006. 7
3. Mila DallaPreda: Code Obfuscation and Malware Detection by Abstract Interpretation Universit`adegliStudi di Verona, Dipartimento di Informatica, TD-02-07, 2007.
4. Ilsun You and KangbinYim: Malware Obfuscation Techniques: A Brief Survey, International Conference on Broadband, Wireless Computing, Communication and Applications, 2010.
5. AriniBalakrishnan, Chloe Schulze "Code Obfuscation Literature Survey"
6. Keehyung Kim, Byung-Ro Moon "Malware Detection based on Dependency Graph using Hybrid Genetic algorithm"
7. DinabandhuBhandari, C. A. Murthy, Sankar K. Pal "Variance As A Stopping Criterion For Genetic Algorithms With Elitist Model"
8. J. Ferrante, K. J. Ottenstein, and J. D. Warren "The program dependence graph and its use in optimization,"
9. Thomas B¨arecke and MarcinDetyniecki "Combining Exhaustive and Approximate Methods for Improved Sub-Graph Matching"
10. AbdounOtman, AbouchbakaJaafar "A Comparative Study of Adaptive Crossover Operators for Genetic Algorithms to Resolve the Traveling Salesman Problem" IJARCSSE,volume 3 , issue 4 , 2013
11. D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co. , Inc. , 1989.
12. Jonathan A. P. Marpaung, MangalSain and Hoon-Jae Lee: Survey on malware evasion techniques: state of the art and challenges, International Conference of Advanced Communication Technology, pp 19-22, 2012.
13. RizwanRehmani , G. C. Hazarika and GunadeepChetia : Malware Threats and Mitigation Strategies: A Survey, Journalof Theoretical and Applied Information Technology, Vol. 29 No. 2, 2011.
14. I. Oliver, D. Smith, and J. Holland, "A study of permutation crossover operators on the traveling salesman problem," in Proc. of the 2nd Int. Conf. on Genetic Algorithms, Mahwah, NJ, USA, 1987, pp. 224–230. 6
15. JacoboToran´ "on the hardness of graph isomorphism"
16. Thomas B¨arecke and MarcinDetyniecki "Memetic Algorithms for Inexact Graph Matching"

Malware Detection Subgraph Isomorphism Genetic Algorithm Dependency Graph