Implementation of a Web Robot and Statistics on the Korean Web


The KIPS Transactions:PartC, Vol. 10, No. 4, pp. 509-518, Aug. 2003
10.3745/KIPSTC.2003.10.4.509,   PDF Download:

Abstract

A web robot is a program that downloads and stores web pages. Implementation issues for developing web robots have been studied widely and various web statistics are reported in the literature. First, this paper describes the overall architecture of our robot and implementation decisions on several important issues. Second, we show empirical statistics on approximately 74 million Korean web pages. Third, we monitored 1,424 Korean web sites to observe the changes of web pages. We identify what factors of web pages could affect the changes. The factors may be used for the selection of web pages to be updated incrementally.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
K. S. Jin and L. S. Ho, "Implementation of a Web Robot and Statistics on the Korean Web," The KIPS Transactions:PartC, vol. 10, no. 4, pp. 509-518, 2003. DOI: 10.3745/KIPSTC.2003.10.4.509.

[ACM Style]
Kim Seong Jin and Lee Sang Ho. 2003. Implementation of a Web Robot and Statistics on the Korean Web. The KIPS Transactions:PartC, 10, 4, (2003), 509-518. DOI: 10.3745/KIPSTC.2003.10.4.509.