Email this article 
Hide
Show all
The Self, the Peer and the Teacher in the EFL Pronunciation Class: A Comparative Study on Assessment, Perceptions and Systematicity
Abstract
This pilot study is aimed at describing, analysing and comparing self, peer and teacher quantitative and qualitative assessment (and consequently also perceptions and degree of systematicity when assessing) in the English as a Foreign Language (EFL) pronunciation class in tertiary education. Accordingly, the main objectives of this study have been to measure, rank and compare the harshness or leniency of the three different types of raters involved and their consistency/systematicity when in this role, as well as to measure the levels of coincidence and/or discrepancy when evaluating from different roles and depending on quantitative or qualitative considerations. The method used for the analysis involves the quantitative and qualitative analyses of the scores obtained in a triple-role assessment task carried out by 16 students and a teacher. The calculation of various statistical measures, the triangulation of the data obtained and the Many-Facet Rasch Measurement analysis of the results have completed the study and constitute the departure point for further larger studies. From the results obtained, it can be highlighted that quantitatively, the self and the teacher’s perceptions seem to be the ones that are more distant or different, whereas the self and the peer's tend to be the most similar. In the same way, qualitative assessments seem to be more lenient, that is to say, slightly higher in mean score and have a lower coefficient of variation than quantitative ones in the three groups analysed. Consistency/systematicity is relatively high but it is still an aspect to be improved on the part of most raters.
Keywords
Full Text:
PDFReferences
Boud, D. and Lublin, J. (1983) Student self assessment: educational benefits within existing resources. In Innovation through Recession, ed. Geoffrey Squires (ed.) Guilford, Surrey: Society for Research into Higher Education, 93-99.
Boud, D. and Middleton, H. (2003) Learning from others at work: Communities of practice and informal learning. In Journal of Workplace Learning, 15(5), 194-202.
Carey, M., Mannell, R.H. and Dunn, P.K. (2011) Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews? Language Testing, 28(2), 201–219.
Cheng, W. and Warren, W. (2005). Peer assessment of language proficiency. In Language Testing 22 (1): 93-121.
Chen, L. and Jang. J.R. (2012) Improvement in automatic pronunciation scoring using additional basic scores and learning to rank. In Proceedings of INTERSPEECH 2012, Portland, Oregon.
Chen, L. and Jang. J.R. (2015). Automatic Pronunciation Scoring with Score Combination by Learning to Rank and Class-Normalized DP-Based Quantization. In IEEE Transactions on Audio, Speech, and Language Processing, 1(23), 1737-1749.
Cincared, T., Gruhn, R., Hacker, C., Nöth, E., and Nakamura, S. (2009) Automatic pronunciation scoring of words and sentences independent from the non-native’s first language. In Computer Speech and Language, 23(1): 65–88.
Cucchiarini, C., Strik, H., and Boves, L. (2000) Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms. In Speech Communication, 30(2-3), 109–119.
Davies, A. (2003) The Native Speaker: Myth and Reality. Clevendon: Multilingual Matters.
Eckes, T. (2015) Introduction to Many-Facet Rasch Measurement. Analyzing and Evaluating Rater-Mediated Assessments. 2nd Revised and Updated Edition. Series: Language Testing and Assessment. Volume 22. Frankfurt am Main, Berlin, Bern, Bruxelles, New York, Oxford, Wien: Peter Lang.
Falchikov, N., and Goldfinch, J. (2000) Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks. In Review of Educational Research, 70(3),287-322.
Franco, H., Neumeyer, L., Kim, Y. and Ronen, O. (1997) Automatic pronunciation scoring for language instruction. Proceedings International Conference on Acoustic, Speech and Signal Processing, 97, 1471-1474.
Franco, H., Neumeyer, L., Digalakis, V. and Ronen, O. (2000) Combination of machine scores for automatic grading of pronunciation quality. In Speech Communication, 30(2-3), 121–130.
Frankland, S. (ed.) 2007. Teaching with Group Work, Peer and Self Assessment. In Enhancing Teaching and Learning through Assessment: Deriving an Appropriate Model, ed. Steve Frankland, 143-195. Netherlands: Springer.
Hacker, C., Batliner, A., Steidl, S., Nöth, E. Niemann, H. and Cincarek, T. (2005) Assessment of Non-Native Children’s Pronunciation: Human Marking and Automatic Scoring. In Proceedings of the 10th International Conference on SPEECH and COMPUTER (SPECOM 2005), ed. George Kokkinakis, Nikos Fakotakis, Evangelos Dermatas, Rodmonga Potapova, 123–126, University of Patras, Moskow State Linguistics University.
IELTS Speaking band descriptors (public version)
https://takeielts.britishcouncil.org/sites/default/files/IELTS_Speaking_band_descriptors.pdf
Johnson, K. (1996) Resonance in an exemplar-based lexicon: The emergence of social identity and phonology. Journal of Phonetics, 34, 485–499.
Jonsson, A. and Svingby, G. (2007) The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144.
Klenowski, V. (1995). Student self-evaluation processes in student-centred teaching and learning contexts of Australia and England. Assessment in Education: Principles, Policy & Practice, 2(2), 145–154.
Langan, M., Shuker, D., Cullen, R., Penney, D. Preziosi, R. and Wheater, P. (2008). Relationships between student characteristics and self‐, peer and tutor evaluations of oral presentations. Assessment & Evaluation in Higher Education 33(2), 179-190.
Li, H., Xiong, Y., Zang, X., Kornhaber, M.L., Lyu, Y., Chung, K.S. and Suen, H.K. (2016). Peer assessment in the digital age: a meta-analysis comparing peer and teacher ratings. Assessment & Evaluation in Higher Education, 41(2), 245-264.
Magin, D. and Churches, A. (1989) Using self and peer assessment in teaching design. In Proceedings,World conference on engineering education for advancing technology. Australia: Institution of Engineers, 89(1), 640-644.
Miller, P.J. (2003). The effect of scoring criteria specificity on peer and self‐assessment. Assessment and Evaluation in Higher Education, 28(4), 383–94.
Moustroufas, N. and Digalakis, V. (2007) Automatic pronunciation assessment of foreign speakers using unknown text. Computer Speech and Language, 21(1), 219–230.
Neumeyer, L., Franco, H., Digalakis, V. and Weintraub, M. (2000) Automatic scoring of pronunciation quality. In Speech Communication, 30(2-3), 83–93.
Ross, J.A. (2006) The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Assessment, 11(10), 1-13.
Smith, H., Cooper, A. and Lancaster, L. (2002) Improving the quality of undergraduate peer assessment: a case for student and staff development. Innovations in Education and Training International, 39(1), 71-81.
Stefani, L.A.J. (1994) Peer, self and tutor assessment: Relative reliabilities. In Studies in Higher Education 19(1): 69-75.
Stevens, K.N. (2002) Toward a Model for Lexical Access Based on Acoustic Landmarks and Distinctive Features, Journal of the Acoustical Society of America, 111, 1872-1891.
Strik, H., Khiet Truong, Febe de Wet and Catia Cucchiarini. (2009) Comparing different approaches for automatic pronunciation error detection. In Speech Communication, 51(10): 845–852.
Taylor, L. (2006). The changing landscape of English: implications for language assessment. ELTJ60 (1), 51-60.
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276.
Witt, S. and Young, S. (2000) Phone-level pronunciation scoring and assessment for interactive language learning. In Speech Communication, 30(2-3), 95–108.
DOI: https://doi.org/10.7575/aiac.ijalel.v.6n.7p.198
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution 4.0 International License.
2012-2023 (CC-BY) Australian International Academic Centre PTY.LTD.
International Journal of Applied Linguistics and English Literature
To make sure that you can receive messages from us, please add the journal emails into your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', check your 'bulk mail' or 'junk mail' folders.