Development of automated scoring system for Thai writing ability test of primary education level

Abstract

This research aimed to: (1) develop the automated scoring system for Thai writing ability test of primary education level, and (2) evaluate the efficiency of the system. The study participants were students enrolled in academic year 2020 in schools in Bangkok: (1) 287 sixth-grade students by purposive sampling from schools in Bangkok with the average score of Thai language from O-NET that was high, medium, and low; and (2) 30 sixth-grade students in schools under the Office of the Private Education Commission by purposive sampling because schools participated willingly, having computer labs and internet connection (i.e., to test online system). Research instruments consisted of: an essay test, three evaluation forms, and the online system developed by PHP language and MySQL database. Results were as follows: (1) The automated scoring rubric system for the Thai writing ability test of primary education level was an online system comprised of 3 steps, i.e., data entry answer as text, automated scoring consisting of Thai word segmentation and scoring rubric, and display of output. The testing system found that the score was M = 8.52, SD = 4.07 and CV = .48, and (2) The efficiency evaluation of the system by using three evaluations forms revealed that rater agreed with the system, which had the highest agreement. The automated scoring rubric was able to predict the test score at .05 level of statistical significant, R2 = 66.3 to 87.6 percent. Measure of agreement of scoring result were with ICC = .88, and RMSE ≤ 3.38.