EFL Thai Students’ Rating Performances in Self- and Peer-Assessments of Writing: A Many-Facets Rasch Analysis
Article Sidebar
Main Article Content
Abstract
Despite the value of self- and peer-assessments in supporting learning, research reports mixed findings about self- and peer-assessments of writing, raising intriguing queries about the validity and utility of student-led assessment in a writing classroom. Applying a many-facets Rasch measurement approach, this quantitative study investigated 53 EFL Thai undergraduates’ rating performances in self- and peer-assessments of writing during an ongoing classroom in a university setting. Main findings revealed that self-assessors tended to rate more leniently and heterogeneously than peer-assessors whose severity levels were far more homogeneous and closer to the level of teacher severity. Across self- and peer-ratings, students could observably distinguish writing performance quality, thereby assigning varying scores across rubric criteria, and lower-ability students were more variable in their levels of severity than those of high ability. The findings highlight the main and interaction effects of student-led assessment type and writing ability level on students’ rating performances. That is, students’ rating performances vary according to different types of student-led assessment and different levels of ability. More specifically, assessment type exercises a greater bearing on lower-ability raters’ rating variability, while ability level exerts a stronger influence on self-raters’ rating variability. The findings provide implications for student-led assessment design and research in the classroom context.
Article Details
References
Al-Mahrooqi, R., & Denman, C. (2018). Alternative assessment. In J. L. Liontas & M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp. 1–6). Wiley-Blackwell. https://doi.org/10.1002/9781118784235.eelt0325
Andrade, H. L. (2019). A critical review of research on student self-assessment. Frontiers in Education, 4(87), 1–13. https://doi.org/10.3389/feduc.2019.00087
Andrade, H. L., & Heritage, M. (2018). Using formative assessment to enhance learning, achievement, and academic self-regulation. Routledge.
Barkaoui, K. (2014). Multifaceted Rasch analysis for test evaluation. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 1–22). John Wiley & Sons, Inc. doi: 10.1002/9781118411360.wbcla070
Brown, G. T. L., & Harris, L. R. (2013). Student self-assessment. In J. H. McMillan (Ed.), Sage handbook of research on classroom assessment (pp. 367–393). SAGE Publications, Inc.
Cheng, W., & Warren, M. (2005). Peer assessment of language proficiency. Language Testing, 22(1), 93–121. https://doi.org/10.1191/0265532205lt298oa
Christison, M. (2018). Student involvement in assessment. In J. L. Liontas & M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp. 1–7). Wiley-Blackwell. https://doi.org/10.1002/9781118784235.eelt0357
Eckes, T. (2015). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments (2nd ed.). Peter Language.
Edwards, J. G. H. (2013). Peer assessment in the classroom. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 730–750). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla002
Erman Aslanoglu, A., Karakaya, I., & Sata, M. (2020). Evaluation of university students’ rating behaviors in self and peer-rating process via many facet Rasch model. Eurasian Journal of Educational Research, 89, 25–46.
Esfandiari, R., & Myford, C. M. (2013). Severity differences among self-assessors, peer-assessors, and teacher assessors rating EFL essays. Assessing Writing, 18(2), 111–131. https://doi.org/10.1016/j.asw.2012.12.002
Fan, J., & Bond, T. (2019). Applying Rasch measurement in language assessment: Unidimensionality and local independence. In V. Aryadoust & M. Raquel (Eds.), Quantitative data analysis for language assessment volume I: Fundamental techniques (pp. 83–102). Routledge.
Han, C., & Riazi, M. (2018). The accuracy of student self-assessments of English-Chinese bidirectional interpretation: A longitudinal quantitative study. Assessment & Evaluation in Higher Education, 43(3), 386–398. https://doi.org/10.1080/02602938.2017.1353062
Harris, L. R., & Brown, G. T. L. (2018). Using self-assessment to improve student learning. Routledge.
Hung, Y.-j., Samuelson, B. L., & Chen, S.-c. (2016). Relationships between peer- and self-assessment and teacher assessment of young EFL learners’ oral presentations. In M. Nikolov (Ed.), Assessing young learners of English: Global and local perspectives (pp. 317–338). Springer, Cham. https://doi.org/10.1007/978-3-319-22422-0
Kim, S. C., & Wilson, M. (2009). A comparative analysis of the ratings in performance assessment using Generalizability theory and the many-facet Rasch model. Journal of Applied Measurement, 10(4), 408–423.
Khamboonruang, A. (2020). Development and validation of a diagnostic rating scale for formative assessment in a Thai EFL university writing classroom: A mixed methods study [Doctoral dissertation, The University of Melbourne]. Minerva Access. http://hdl.handle.net/11343/252672
Li, M., & Zhang, X. (2021). A meta-analysis of self-assessment and language performance in language testing and assessment. Language Testing, 38(2), 189–218. https://doi.org/10.1177/0265532220932481
Linacre, J. M. (1989). Many-facet Rasch measurement. MESA.
Linacre, J. M. (2021). Facets computer program for many-facet Rasch measurement, version 3.83.6. Winsteps.com
Matsuno, S. (2009). Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms. Language Testing, 26(1), 075–100. https://doi.org/10.1177/0265532208097337
McNamara, T., Knoch, U., & Fan, J. (2019). Fairness, justice, and language assessment: The role of measurement. Oxford University Press.
Midraj, J. (2018). Self-assessment. In J. L. Liontas & M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp. 1–7). Wiley-Blackwell. https://doi.org/10.1002/9781118784235.eelt0331
Myford, C. M., & Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386–422.
Oscarson, M. (2013). Self-assessment in the classroom. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 712–729). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla046
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.
Ross, S. (1998). Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1–20. https://doi.org/10.1177/026553229801500101
Saito, H. (2008). EFL classroom peer assessment: Training effects on rating and commenting. Language Testing, 25(4), 553–581. https://doi.org/10.1177/0265532208094276
Saito, H., & Fujita, T. (2004). Characteristics and user acceptance of peer-rating in EFL writing classrooms. Language Teaching Research, 8(1), 31–54. https://doi.org/10.1191/1362168804lr133oa
Sun, Y., & Doman, E. (2018). Peer assessment. In J. L. Liontas & M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp. 1–7). Wiley-Blackwell. https://doi.org/10.1002/9781118784235.eelt0330
Topping, K. J. (2013). Peers as a source of formative and summative assessment. In J. H. McMillan (Ed.), SAGE handbook of research on classroom assessment (pp. 395–412). SAGE Publications, Inc.