Real-time feature selection technique with concept drift detection using adaptive micro-clusters for data stream mining

• Main Authors: Badii, A. (Author), Hammoodi, M.S (Author), Stahl, F. (Author)
• Format: Article
• Language: English
• Published: Elsevier B.V. 2018
• Subjects: Classification (of information); Classification algorithm; Concept drift detection; Concept drifts; Data mining; Data stream classifications; Data stream mining; Data Stream Mining; Electronic trading; Feature extraction; Intrusion detection; Learning systems; Real-time feature selection; Real-time features; Real-time sensor networks; Sensor networks; Situation assessment; Statistical summary
• Online Access: View Fulltext in Publisher

 Data streams are unbounded, sequential data instances that are generated with high Velocity. Classifying sequential data instances is a very challenging problem in machine learning with applications in network intrusion detection, financial markets and applications requiring real-time sensor-networks-based situation assessment. Data stream classification is concerned with the automatic labelling of unseen instances from the stream in real-time. For this the classifier needs to adapt to concept drifts and can only have a single pass through the data if the stream is fast moving. This research paper presents work on a real-time pre-processing technique, in particular feature tracking. The feature tracking technique is designed to improve Data Stream Mining (DSM) classification algorithms by enabling and optimising real-time feature selection. The technique is based on tracking adaptive statistical summaries of the data and class label distributions, known as Micro-Clusters. Currently the technique is able to detect concept drifts and identify which features have been influential in the drift. © 2018