Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research
Supervised learning and pattern recognition is a crucial area of research in information retrieval, knowledge engineering, image processing, medical imaging, and intrusion detection. Numerous algorithms have been designed to address such complex application domains. Despite an enormous array of supe...
Main Authors: | , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2021-03-01
|
Series: | Mathematics |
Subjects: | |
Online Access: | https://www.mdpi.com/2227-7390/9/6/690 |
id |
doaj-3a001a47b01444f291fe44b06afda7d4 |
---|---|
record_format |
Article |
spelling |
doaj-3a001a47b01444f291fe44b06afda7d42021-03-24T00:05:04ZengMDPI AGMathematics2227-73902021-03-01969069010.3390/math9060690Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future ResearchRanjit Panigrahi0Samarjeet Borah1Akash Kumar Bhoi2Muhammad Fazal Ijaz3Moumita Pramanik4Rutvij H. Jhaveri5Chiranji Lal Chowdhary6Department of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, IndiaDepartment of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, IndiaDepartment of Electrical and Electronics Engineering, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, IndiaDepartment of Intelligent Mechatronics Engineering, Sejong University, Seoul 05006, KoreaDepartment of Computer Applications, Sikkim Manipal Institute of Technology, Sikkim Manipal University, Majitar 737136, Sikkim, IndiaDepartment of Computer Science and Engineering, School of Technology, Pandit Deendayal Energy University, Gandhinagar 382007, Gujarat, IndiaSchool of Information Technology & Engineering, Vellore Institute of Technology, Vellore 632014, Tamil Nadu, IndiaSupervised learning and pattern recognition is a crucial area of research in information retrieval, knowledge engineering, image processing, medical imaging, and intrusion detection. Numerous algorithms have been designed to address such complex application domains. Despite an enormous array of supervised classifiers, researchers are yet to recognize a robust classification mechanism that accurately and quickly classifies the target dataset, especially in the field of intrusion detection systems (IDSs). Most of the existing literature considers the accuracy and false-positive rate for assessing the performance of classification algorithms. The absence of other performance measures, such as model build time, misclassification rate, and precision, should be considered the main limitation for classifier performance evaluation. This paper’s main contribution is to analyze the current literature status in the field of network intrusion detection, highlighting the number of classifiers used, dataset size, performance outputs, inferences, and research gaps. Therefore, fifty-four state-of-the-art classifiers of various different groups, i.e., Bayes, functions, lazy, rule-based, and decision tree, have been analyzed and explored in detail, considering the sixteen most popular performance measures. This research work aims to recognize a robust classifier, which is suitable for consideration as the base learner, while designing a host-based or network-based intrusion detection system. The NSLKDD, ISCXIDS2012, and CICIDS2017 datasets have been used for training and testing purposes. Furthermore, a widespread decision-making algorithm, referred to as Techniques for Order Preference by Similarity to the Ideal Solution (TOPSIS), allocated ranks to the classifiers based on observed performance reading on the concern datasets. The J48Consolidated provided the highest accuracy of 99.868%, a misclassification rate of 0.1319%, and a Kappa value of 0.998. Therefore, this classifier has been proposed as the ideal classifier for designing IDSs.https://www.mdpi.com/2227-7390/9/6/690classifiers rankingclass-imbalance learningIDSIDS base learnerintrusion detection systemsnetwork-based IDS |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Ranjit Panigrahi Samarjeet Borah Akash Kumar Bhoi Muhammad Fazal Ijaz Moumita Pramanik Rutvij H. Jhaveri Chiranji Lal Chowdhary |
spellingShingle |
Ranjit Panigrahi Samarjeet Borah Akash Kumar Bhoi Muhammad Fazal Ijaz Moumita Pramanik Rutvij H. Jhaveri Chiranji Lal Chowdhary Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research Mathematics classifiers ranking class-imbalance learning IDS IDS base learner intrusion detection systems network-based IDS |
author_facet |
Ranjit Panigrahi Samarjeet Borah Akash Kumar Bhoi Muhammad Fazal Ijaz Moumita Pramanik Rutvij H. Jhaveri Chiranji Lal Chowdhary |
author_sort |
Ranjit Panigrahi |
title |
Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research |
title_short |
Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research |
title_full |
Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research |
title_fullStr |
Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research |
title_full_unstemmed |
Performance Assessment of Supervised Classifiers for Designing Intrusion Detection Systems: A Comprehensive Review and Recommendations for Future Research |
title_sort |
performance assessment of supervised classifiers for designing intrusion detection systems: a comprehensive review and recommendations for future research |
publisher |
MDPI AG |
series |
Mathematics |
issn |
2227-7390 |
publishDate |
2021-03-01 |
description |
Supervised learning and pattern recognition is a crucial area of research in information retrieval, knowledge engineering, image processing, medical imaging, and intrusion detection. Numerous algorithms have been designed to address such complex application domains. Despite an enormous array of supervised classifiers, researchers are yet to recognize a robust classification mechanism that accurately and quickly classifies the target dataset, especially in the field of intrusion detection systems (IDSs). Most of the existing literature considers the accuracy and false-positive rate for assessing the performance of classification algorithms. The absence of other performance measures, such as model build time, misclassification rate, and precision, should be considered the main limitation for classifier performance evaluation. This paper’s main contribution is to analyze the current literature status in the field of network intrusion detection, highlighting the number of classifiers used, dataset size, performance outputs, inferences, and research gaps. Therefore, fifty-four state-of-the-art classifiers of various different groups, i.e., Bayes, functions, lazy, rule-based, and decision tree, have been analyzed and explored in detail, considering the sixteen most popular performance measures. This research work aims to recognize a robust classifier, which is suitable for consideration as the base learner, while designing a host-based or network-based intrusion detection system. The NSLKDD, ISCXIDS2012, and CICIDS2017 datasets have been used for training and testing purposes. Furthermore, a widespread decision-making algorithm, referred to as Techniques for Order Preference by Similarity to the Ideal Solution (TOPSIS), allocated ranks to the classifiers based on observed performance reading on the concern datasets. The J48Consolidated provided the highest accuracy of 99.868%, a misclassification rate of 0.1319%, and a Kappa value of 0.998. Therefore, this classifier has been proposed as the ideal classifier for designing IDSs. |
topic |
classifiers ranking class-imbalance learning IDS IDS base learner intrusion detection systems network-based IDS |
url |
https://www.mdpi.com/2227-7390/9/6/690 |
work_keys_str_mv |
AT ranjitpanigrahi performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT samarjeetborah performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT akashkumarbhoi performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT muhammadfazalijaz performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT moumitapramanik performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT rutvijhjhaveri performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch AT chiranjilalchowdhary performanceassessmentofsupervisedclassifiersfordesigningintrusiondetectionsystemsacomprehensivereviewandrecommendationsforfutureresearch |
_version_ |
1724205357149978624 |