2018•Language Assessment QuarterlyRequires access

Teachers’ Judgement Accuracy Concerning CEFR Levels of Prospective University Students

Johanna Fleckenstein, Michael Leucht, Olaf Köller

Open publisher page 24 citations

Abstract

Most English-medium programs at European universities require prospective students to take standardised tests for English as a foreign language (EFL) to be admitted. However, there are contexts in which individual teachers’ judgements serve the same function, thus having high-stakes consequences for the higher education entrance of their students. The present study investigates the accuracy of CEFR-based judgements of upper secondary school EFL teachers in Germany and compares it to the accuracy of conventional grading. Seventy-three teachers located each of the students in their EFL class onto a CEFR level. Teachers’ judgements were compared to the CEFR levels estimated from the TOEFL ITP using level, differentiation, and ranking component as indicators of judgement accuracy. Findings show that while CEFR-based judgements of relative EFL proficiency within classes (rank component) are rather appropriate, teachers substantially overestimate the total level of their students’ achievement (level component). Compared to conventional grading, however, CEFR-based judgements lead to more accurate estimations of variance between students in a class (differentiation component). The findings are discussed with particular reference to the consequences for higher education entrance policy concerning CEFR levels B2 and C1 as relevant benchmarks.

About this research paper

What this paper is about

Most English-medium programs at European universities require prospective students to take standardised tests for English as a foreign language (EFL) to be admitted. However, there are contexts in which individual teachers’ judgements serve the same function, thus having high-stakes consequences for the higher education entrance of their students. The present study investigates the accuracy of CEFR-based judgements of upper secondary school EFL teachers in Germany and compares it to the accuracy of conventional grading. Seventy-three teachers located each of the students in their EFL class onto a CEFR level. Teachers’ judgements were compared to the CEFR levels estimated from the TOEFL ITP using level, differentiation, and ranking component as indicators of judgement accuracy. Findings show that while CEFR-based judgements of relative EFL proficiency within classes (rank component) are rather appropriate, teachers substantially overestimate the total level of their students’ achievement (level component). Compared to conventional grading, however, CEFR-based judgements lead to more accurate estimations of variance between students in a class (differentiation component). The findings are discussed with particular reference to the consequences for higher education entrance policy concerning CEFR levels B2 and C1 as relevant benchmarks.

Why it matters

OpenAlex reports 24 citations for this work. Citation counts describe recorded attention and do not establish research quality.

Key contribution

A contribution statement is not available in the OpenAlex record.

Method / approach

Method details are not available in the OpenAlex metadata.

Main findings

Findings are not separately available in the OpenAlex metadata.

Limitations

Limitations are not available in the OpenAlex metadata.

Applications

Application details are not available in the OpenAlex metadata.

Available abstract

Most English-medium programs at European universities require prospective students to take standardised tests for English as a foreign language (EFL) to be admitted. However, there are contexts in which individual teachers’ judgements serve the same function, thus having high-stakes consequences for the higher education entrance of their students. The present study investigates the accuracy of CEFR-based judgements of upper secondary school EFL teachers in Germany and compares it to the accuracy of conventional grading. Seventy-three teachers located each of the students in their EFL class onto a CEFR level. Teachers’ judgements were compared to the CEFR levels estimated from the TOEFL ITP using level, differentiation, and ranking component as indicators of judgement accuracy. Findings show that while CEFR-based judgements of relative EFL proficiency within classes (rank component) are rather appropriate, teachers substantially overestimate the total level of their students’ achievement (level component). Compared to conventional grading, however, CEFR-based judgements lead to more accurate estimations of variance between students in a class (differentiation component). The findings are discussed with particular reference to the consequences for higher education entrance policy concerning CEFR levels B2 and C1 as relevant benchmarks.

Key concepts: Grading (engineering), Psychology, Judgement, Mathematics education, Language proficiency, Test of English as a Foreign Language, English as a foreign language, Language assessment

Related papers

Back to paper searchBrowse research topicsOriginal source
Teachers’ Judgement Accuracy Concerning CEFR Levels of Prospective University Students — Research Paper | ScholarLens