KubAnomaly: Anomaly detection for the Docker orchestration platform with neural network approaches

• Main Authors: Chin‐Wei Tien, Tse‐Yung Huang, Chia‐Wei Tien, Ting‐Chun Huang, Sy‐Yen Kuo
• Format: Article
• Language: English
• Published: Wiley 2019-12-01
• Series: Engineering Reports
• Subjects: anomaly detection; cloud security; container orchestration; container security; machine learning; neural network
• Online Access: https://doi.org/10.1002/eng2.12080

Kubernetes, which is the most popular orchestration platform for Docker containers, is used widely for developing microservices and automating Docker instance life cycle administration. Because of advancements in containerization technology, a single server can run multiple services and use hardware resources more efficiently. However, containerized environments also bring new challenges in terms of complete monitoring and security provision. Thus, hackers can exploit the security vulnerabilities of containers to gain remote control permissions and cause extensive damage to company assets. Therefore, in this study, we propose KubAnomaly, a system that provides security monitoring capabilities for anomaly detection on the Kubernetes orchestration platform. We develop a container monitoring module for Kubernetes and implement neural network approaches to create classification models that strengthen its ability to find abnormal behaviors such as web service attacks and common vulnerabilities and exposures attacks. We use three types of datasets to evaluate our system, including privately collected and publicly available datasets as well as real‐world experiment data. Furthermore, we demonstrate the effectiveness of KubAnomaly by comparing its accuracy with that of other machine learning algorithms. KubAnomaly is shown to achieve an overall accuracy of up to 96% for anomaly detection. It successfully identifies four real attacks carried out by hackers in September 2018. Moreover, its performance overhead is only 5% greater than that of current methods. In summary, KubAnomaly significantly improves container security by avoiding anomaly attacks.