A Criterion for Deciding the Number of Clusters in a Dataset Based on Data Depth
Clustering is a key method in unsupervised learning with various applications in data mining, pattern recognition and intelligent information processing. However, the number of groups to be formed, usually notated as k is a vital parameter for most of the existing clustering algorithms as their clus...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
World Scientific Publishing
2020-11-01
|
| Series: | Vietnam Journal of Computer Science |
| Subjects: | |
| Online Access: | http://www.worldscientific.com/doi/pdf/10.1142/S2196888820500232 |
| Summary: | Clustering is a key method in unsupervised learning with various applications in data mining, pattern recognition and intelligent information processing. However, the number of groups to be formed, usually notated as k is a vital parameter for most of the existing clustering algorithms as their clustering results depend heavily on this parameter. The problem of finding the optimal k value is very challenging. This paper proposes a novel idea for finding the correct number of groups in a dataset based on data depth. The idea is to avoid the traditional process of running the clustering algorithm over a dataset for n times and further, finding the k value for a dataset without setting any specific search range for k parameter. We experiment with different indices, namely CH, KL, Silhouette, Gap, CSP and the proposed method on different real and synthetic datasets to estimate the correct number of groups in a dataset. The experimental results on real and synthetic datasets indicate good performance of the proposed method. |
|---|---|
| ISSN: | 2196-8888 2196-8896 |