The Study of Validity Evidence on Cognitive Diagnostic Assessment on The Use of Japanese for Communication for Grade 10 Students

Abstract

The objectives of this study were (i) to design and develop the cognitive diagnostic assessment on the utilisation of Japanese for Communication for Mattayom 4 (Grade 10) students, (ii) to develop the cognitive diagnostic assessment reports, and (iii) to investigate the validity evidence of the cognitive diagnostic assessment. The Design and Development Research (DDR) methodology was implemented for this study. The 12 items of the test were constructed along with think aloud and Q-Matrix. Content validity of the test was validated by 4 experts and had been tried out with 30 students to determine difficulty and discrimination through Classical Test Theory or CTT. Group of tests based on Deterministic–input, Noisy–and–Gate Model or DINA Model with 300 students. The results of the study were as follows: a) It was discovered that there were five attributes, the results of the quality inspection of the test are within the appropriate criteria and be able to use the test with students. b) It was ascertained that there were six aspects illustrated outcome reports. The investigation results of the quality were averaged at the maximum level ( = 4.81,). c) It was determined that the study results on the validity evidence of the cognitive diagnostic assessment were as follows: the CVI evidence was equal to 1.00. The validity evidence of the responsive process pointed out that students’ understanding corresponded with their specified characteristics. The structural validity evidence showed that the relationship between all item and the various elements of the assessment accord to the attribute. The validity evidence with other variables showed that the diagnosis results and academic achievement were significantly corresponded with one another at a high level (r_xy=.505). All in all, the validity evidence of the results obtained in the cognitive diagnostic assessment can be beneficial at the highest levels. In terms of the qualitative study, the findings revealed that students, teachers, educational supervisors, and experienced instructors were able to use the diagnosis results to comprehend and acknowledge their limitations.