Automated Scoring System for Korean Short-Answer Questions Using Predictability and Unanimity


KIPS Transactions on Software and Data Engineering, Vol. 5, No. 11, pp. 527-534, Nov. 2016
10.3745/KTSDE.2016.5.11.527,   PDF Download:
Keywords: Machine Learning, Korean Automated-Scoring System, Unanimity, Predictability, Natural Language Processing
Abstract

The emergent information society requires the talent for creative thinking based on problem-solving skills and comprehensive thinking rather than simple memorization. Therefore, the Korean curriculum has also changed into the direction of the creative thinking through increasing short-answer questions that can determine the overall thinking of the students. However, their scoring results are a little bit inconsistency because scoring short-answer questions depends on the subjective scoring of human raters. In order to alleviate this point, an automated scoring system using a machine learning has been used as a scoring tool in overseas. Linguistically, Korean and English is totally different in the structure of the sentences. Thus, the automated scoring system used in English cannot be applied to Korean. In this paper, we introduce an automated scoring system for Korean short-answer questions using predictability and unanimity. We also verify the practicality of the automatic scoring system through the correlation coefficient between the results of the automated scoring system and those of human raters. In the experiment of this paper, the proposed system is evaluated for constructed-response items of Korean language, social studies, and science in the National Assessment of Educational Achievement. The analysis was used Pearson correlation coefficients and Kappa coefficient. Results of the experiment had showed a strong positive correlation with all the correlation coefficients at 0.7 or higher. Thus, the scoring results of the proposed scoring system are similar to those of human raters. Therefore, the automated scoring system should be found to be useful as a scoring tool.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
M. Cheon, C. Kim, J. Kim, E. Noh, K. Sung, M. Song, "Automated Scoring System for Korean Short-Answer Questions Using Predictability and Unanimity," KIPS Transactions on Software and Data Engineering, vol. 5, no. 11, pp. 527-534, 2016. DOI: 10.3745/KTSDE.2016.5.11.527.

[ACM Style]
Min-Ah Cheon, Chang-Hyun Kim, Jae-Hoon Kim, Eun-Hee Noh, Kyung-Hee Sung, and Mi-Young Song. 2016. Automated Scoring System for Korean Short-Answer Questions Using Predictability and Unanimity. KIPS Transactions on Software and Data Engineering, 5, 11, (2016), 527-534. DOI: 10.3745/KTSDE.2016.5.11.527.