Ensemble Machine Learning for Intrusion Detection: A Comparative Study on Accuracy, False Positives and Feature Importance

Abstract

Cyber threats are becoming increasingly sophisticated, and signature-driven Intrusion Detection Systems (IDS) are less effective at detecting new attacks. Machine learning (ML)based Intrusion Detection Systems (IDS) offer an alternative, as these solutions can learn patterns from historical data. However, class imbalance, false positives, and interpretability issues hinder effective deployment. In this study, five ML classifiers were evaluated on a dataset of 9,537 records with 11 features (e.g., session duration, login attempts, and IP reputation), including Logistic Regression, Random Forest, XGBoost, Support Vector Machine (SVM), and Neural Networks. Random Forest yielded the best overall accuracy rate (88.1%), best F1-score (0.849), and lowest false-positive (0.010) classification rates. It also appeared to generalize better than XGBoost and SVM, which performed with 87.2% and 86.6% accuracy, respectively. Logistic Regression and Neural Networks had higher false-positive rates (0.240 and 0.166) and offered lower classification detection performance than XGBoost, Random Forest, and SVM. Notably, feature importance analysis revealed that authentication patterns (failed logins) and IP reputation were important features that affected the detection of attacks. In conclusion, this study highlights that ensemble ML methods (Random Forest, XGBoost) offer a reasonable trade-off between accuracy and interpretability for real-world deployment in an IDS. Future work could focus on adversarial robustness, temporal modeling, and hybrid detection methods to bridge signature-driven and anomaly-driven detection. Overall, this study contributes to the development of explainable and adaptive IDS for effective network security.