CFP last date
20 May 2026
Reseach Article

Behavioral Anomaly Detection in Linux Systems using eBPF and LSTM Neural Networks: A Comparative Study with Traditional Machine Learning

by Mustafa Ajanovic
International Journal of Computer Applications
Foundation of Computer Science (FCS), NY, USA
Volume 187 - Number 99
Year of Publication: 2026
Authors: Mustafa Ajanovic
10.5120/ijca502db8d4e336

Mustafa Ajanovic . Behavioral Anomaly Detection in Linux Systems using eBPF and LSTM Neural Networks: A Comparative Study with Traditional Machine Learning. International Journal of Computer Applications. 187, 99 ( Apr 2026), 1-6. DOI=10.5120/ijca502db8d4e336

@article{ 10.5120/ijca502db8d4e336,
author = { Mustafa Ajanovic },
title = { Behavioral Anomaly Detection in Linux Systems using eBPF and LSTM Neural Networks: A Comparative Study with Traditional Machine Learning },
journal = { International Journal of Computer Applications },
issue_date = { Apr 2026 },
volume = { 187 },
number = { 99 },
month = { Apr },
year = { 2026 },
issn = { 0975-8887 },
pages = { 1-6 },
numpages = {9},
url = { https://ijcaonline.org/archives/volume187/number99/behavioral-anomaly-detection-in-linux-systems-using-ebpf-and-lstm-neural-networks-a-comparative-study-with-traditional-machine-learning/ },
doi = { 10.5120/ijca502db8d4e336 },
publisher = {Foundation of Computer Science (FCS), NY, USA},
address = {New York, USA}
}
%0 Journal Article
%1 2026-04-28T21:29:24.410720+05:30
%A Mustafa Ajanovic
%T Behavioral Anomaly Detection in Linux Systems using eBPF and LSTM Neural Networks: A Comparative Study with Traditional Machine Learning
%J International Journal of Computer Applications
%@ 0975-8887
%V 187
%N 99
%P 1-6
%D 2026
%I Foundation of Computer Science (FCS), NY, USA
Abstract

Host-based intrusion detection systems (HIDS) represent a critical layer of defense against cyberattacks that bypass perimeter controls or originate from within a host environment. Despite significant progress, existing solutions continue to suffer from high false-positive rates, susceptibility to adversarial evasion, and an inherent inability to detect zero-day threats. This research investigates artificial intelligence and machine learning techniques to advance host-based intrusion detection through two complementary experiments. First, a Random Forest classifier is trained and evaluated on the ADFA-WD benchmark dataset using a TF-IDF and n-gram preprocessing pipeline, establishing a reproducible performance baseline (ROC-AUC: 0.74, F1-Score: 0.12). Second, a novel eBPF-based collection framework is designed for Linux systems, pairing kernel-level system call telemetry with an LSTM Autoencoder trained exclusively on normal behavioral sequences. Evaluated on a controlled synthetic dataset of 8,417 behavioral sessions simulating realistic Linux web server attack scenarios, the LSTM Autoencoder achieves an F1-Score of 0.66 and ROC-AUC of 0.81, demonstrating the architectural superiority of sequential, context-aware modeling over traditional ensemble approaches.

References
  1. S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A. Longstaff, “A sense of self for Unix processes,” Proc. IEEE Symp. Security and Privacy, pp. 120–128, 1996.
  2. R. Sekar, M. Bendre, D. Dhurjati, and P. Bollineni, “A fast automaton-based method for detecting anomalous program behaviors,” Proc. IEEE Symp. Security and Privacy, pp. 144–155, 2001.
  3. D. Kim, H. Lee, S. Cho, and B. Noh, “A recurrent neural network based approach for intrusion detection,” Proc. IEEE Intl. Conf. Big Data, pp. 2828–2836, 2016.
  4. J. Choi, H. Kim, and C. Choi, “Intrusion detection system combined with CNN and RNN for system call sequences,” Proc. PlatCon, pp. 1–5, 2017.
  5. A. Javaid, Q. Niyaz, W. Sun, and M. Alam, “A deep learning approach for network intrusion detection system,” Proc. 9th EAI BICT Conf., pp. 21–26, 2016.
  6. A. Goyal, X. Han, G. Wang, and A. Bates, “Sometimes, you aren’t what you do: Mimicry attacks against provenance graph host intrusion detection systems,” Proc. NDSS Symp., 2023.
  7. J. Glass-Vanderlan, M. K. Reiter, and A. Bates, “Provenance-based intrusion detection: Opportunities and challenges,” ACM Computing Surveys, vol. 55, no. 1, pp. 1–36, 2023.
  8. X. Han, T. F. J. Pasquier, A. Bates, J. Mickens, and M. Seltzer, “UNICORN: Runtime provenance-based detector for advanced persistent threats,” Proc. NDSS Symp., 2022.
  9. C. Kruegel, E. Kirda, D. Mutz, W. Robertson, and G. Vigna, “Automating mimicry attacks using static binary analysis,” Proc. USENIX Security Symp., pp. 161–176, 2005.
  10. N. K. Niemann and R. G. Blockmon, “Using machine learning to predict the insider threat in a network environment,” M.S. thesis, Naval Postgraduate School, Monterey, CA, USA, 2021.
  11. B. Bin Sarhan and N. Altwaijry, “Insider threat detection using machine learning approach,” Applied Sciences, vol. 13, no. 1, p. 259, 2022.
  12. A. Z. Ahmad, R. Abdullah, and M. F. Abdollah, “Transformer-based anomaly detection for host intrusion detection systems,” IEEE Access, vol. 12, pp. 14221–14235, 2024.
  13. G. Creech and J. Hu, “A semantic approach to host-based intrusion detection systems,” IEEE Trans. Computers, vol. 63, no. 4, pp. 807–819, 2014.
  14. M. Almseidin, M. Alzubi, S. Kovacs, and M. Alkasassbeh, “Evaluation of machine learning algorithms for intrusion detection system,” Proc. IEEE SISY, pp. 277–282, 2017.
  15. University of New South Wales, “ADFA Intrusion Detection Datasets,” 2013. [Online]. Available: https://www.unsw.adfa.edu.au
  16. B. Gregg, BPF Performance Tools: Linux System and Application Observability, 1st ed., Addison-Wesley, 2019.
Index Terms

Computer Science
Information Sciences

Keywords

Host-based intrusion detection anomaly detection eBPF LSTM autoencoder zero-day detection machine learning system calls behavioral analysis