Raters’ orientation in a paired speaking test

Abstract

This qualitative research study examines the ways in which raters assessed the performance of students in a paired speaking task. The research objectives were to study 1) how the raters assigned scores to the students’ performance, and 2) the features of interaction that the raters attended to when assessing students’
proficiency in a paired speaking test. Seventeen raters were asked to watch three video clips of three different pairs of students performing a paired task. The raters were subsequently asked to assign scores and provide comments on the features of interaction that they attended to while watching the clips. Their comments were recorded and later transcribed for the analysis. Results indicate all of the raters in the study assigned scores to each individual student rather than as a pair. With the exception of one rater who used analytic rating scales, the remaining sixteen employed holistic rating scales. The analysis of their comments suggest five main categories of interaction that the raters
attended to when assessing the students’ performance, including task completion, accuracy, interactional management, naturalness, and fluency.

Keywords: paired speaking test; rating criteria for speaking test; raters’ orientation