Agent-oriented method of clustering the wholesale distributor data

Abstract

The purpose of the research is to improve the accuracy of data clustering and to determine the target number of data clusters generated by dynamic economic systems, using an agent-oriented clustering method with the introduction of data preprocessing methods.

Research methods: data processing and preparation methods, elemental distance measures, and clustering methods have been used. The software is developed by using the Python language. The following libraries have also been used: scikit-learn, NumPy, SciPy, Pandas, PyTorch and others.

As a result of the research, the data of the wholesale distributor have been processed by the data pre-processing methods such as the determination of missing values, the determination of asymmetry and the Box-Cox transformation. The normalization of the data with the min-max normalization method and the dimensionality reduction with the PCA and t-SNE methods have been carried out. Afterwards, the agent-oriented clustering method has been applied with the Manhattan distance, Mahalanobis distance with the inverse value of the membership function, Kullback-Leibler divergence and cross-entropy metrics. Kullback-Leibler divergence has shown the best accuracy results and has been chosen for the further testing. The ability of the agent-oriented method to determine the number of clusters has been tested. The use of data preprocessing methods shows the clear presence of 3 target clusters, which was confirmed by the method. Conclusions: The developed method allows for high clustering accuracy due to the performed data processing, the correctly selected measure of elemental distance and the use of an agent-oriented approach. This method can be used to improve the quality of data clustering of dynamic economic systems, but the method requires improvement in order to increase flexibility in determining the size of cluster agents.