Gathering Common-word and Document Reclassification to improve Accuracy of Document Clustering


KIPS Transactions on Software and Data Engineering, Vol. 19, No. 1, pp. 54-63, Jan. 2012
10.3745/KIPSTB.2012.19.1.54, Full Text:

Abstract

Clustering technology is used to deal efficiently with many searched documents in information retrieval system. But the accuracy of the clustering is satisfied to the requirement of only some domains. This paper proposes two methods to increase accuracy of the clustering. We define a common-word, that is frequently used but has low weight during clustering. We propose the method that automatically gathers the common-word and calculates its weight from the searched documents. From the experiments, the clustering error rates using the common-word is reduced to 34% compared with clustering using a stop-word. After generating first clusters using average link clustering from the searched documents, we propose the algorithm that reevaluates the similarity between document and clusters and reclassifies the document into more similar clusters. From the experiments using Naver JiSikIn category, the accuracy of reclassified clusters is increased to 1.81% compared with first clusters without reclassification.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
J. C. Shin, C. Y. Ock and E. B. Lee, "Gathering Common-word and Document Reclassification to improve Accuracy of Document Clustering," KIPS Journal B (2001 ~ 2012) , vol. 19, no. 1, pp. 54-63, 2012. DOI: 10.3745/KIPSTB.2012.19.1.54.

[ACM Style]
Joon Choul Shin, Cheol Young Ock, and Eung Bong Lee. 2012. Gathering Common-word and Document Reclassification to improve Accuracy of Document Clustering. KIPS Journal B (2001 ~ 2012) , 19, 1, (2012), 54-63. DOI: 10.3745/KIPSTB.2012.19.1.54.