Effects of Technology Modes on Ratings of Learner Recordings - Lavolette, Elizabeth (Betsy) (IALLT Journal of Language Learning Technologies, 2013)
While research has investigated the effect of visuals in tests of listening comprehension (e.g., Suvorov, 2009; Wagner, 2008, 2010), studentrecorded video for oral formative assessment is relatively unexplored. In this study, I examined 15 teachers’ ratings of speech recorded by 39 ESL learners to see if teachers assess speech differently depending on whether it is presented with visuals. The learners recorded 4 speech samples: 2 with webcams, 2 with microphones only. A third speech condition was created by removing the video track from the webcam recordings, resulting in 3 conditions and 6 samples for each individual. The teachers rated all 6 samples. I used repeated-measures ANOVAs to determine whether the teachers assigned significantly different scores based on the speech conditions. The results showed that the teachers rated the audio stripped from the video significantly higher than the video/audio recordings (p = .004, d = .38). This suggests that teachers may be biased in favor of audio-only recordings and that teachers should not give students an option of making either an audio or video recording for a given formative assessment. Further analyses examined how the students’ and teachers’ preferences for audio-only or video recordings were related to the ratings.