Aligning Academic Reading Tests to the Common European Framework of Reference for Languages (CEFR)
Article Sidebar
Main Article Content
Abstract
Given the significant global influence of the Common European Framework of Reference for Languages: Teaching, Learning, and Assessment (CEFR) on English language education, this study deals with aligning a university’s academic reading tests to the CEFR. It aimed at validating the test construct of the academic reading tests in relation to the proficiency levels defined by the CEFR. The study employs two standard setting procedures outlined in the CEFR Manual: the Familiarization procedure and the Specification procedure, to explore the CEFR level of the academic reading tests as well as the prominent characteristics of the reading texts and the test items in terms of their level and key features. Three academic reading tests were randomly selected. The CEFR Content Analysis Grid for Reading was employed to characterize the content of test items and test tasks. The results indicated that 9 out of 18 reading texts were estimated to correspond to the B2 CEFR level. Texts estimated as B1 and C1 levels were evenly distributed, and none of the reading texts were classified as A1 or C2 levels. Moreover, the findings demonstrated a significant prevalence of B2 and B1 level items. Specifically, B2 items represented the largest proportion at 31.88%, closely followed by B1 items at 25.12%.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Alderson, J. C., Figueras, N., Kuijper, H., Nold, G., Takala, S., & Tardieu, C. (2006). Analysing tests of reading and listening in relation to the Common European Framework of Reference: The experience of the Dutch CEFR construct project. Language Assessment Quarterly, 3(1), 3–30.
Ativorakun, C., & Wudthayagorn, J. (2018). Mapping Srinakharinwirot University – Standardized English Test (SWU-SET) onto the Common European Framework of Reference (CEFR). Suranaree Journal of Social Science, 12(2), 69–84. https://doi.org/10.55766/CTUU4836
Bax, S. (2012). Text inspector. https://textinspector.com/
Brunfaut, T. & Harding, L. (2014). Linking the GEPT listening test to the Common European Framework of Reference (LTTC-GEPT Research Report No. RG-05). The Language Training and Testing Center.
Chall, J. S., & Dale, E. (1995). Readability revisited: The new Dale-Chall readability formula. Brookline Books.
Cohen, A. D., & Upton, T. A. (2007). “I want to go back to the text”: Response strategies on the reading subtest of the New TOEFL. Language Testing, 24(2), 209–250. https://doi.org/10.1177/0265532207076364
Council of Europe. (2001). Common European Framework of Reference for languages: Learning, teaching, assessment. Cambridge University Press.
Council of Europe. (2001). Common European Framework of Reference for languages: Learning, teaching, assessment – Structured overview of all CEFR scales. https://rm.coe.int/168045b15e
Council of Europe. (2009). Relating language examinations to the ‘Common European Framework of Reference for Languages’: Learning, teaching, assessment (CEFR). A Manual. http://rm.coe.int/CoERMPublicCommonSearchServices/DisplayDCTMContent?documentId=0900001680667a2d
Figueras, N. (2012). The impact of the CEFR. ELT Journal, 66(4), 477–485. https://doi.org/10.1093/elt/ccs037
Fry, E. (1968). A readability formula that saves time. Journal of Reading, 11(7), 265–271.
Graduate School of Khon Kaen University. (2022). Graduate student manual 2022. https://app.gs.kku.ac.th/images/img/news/announcement/2-1GS-77-66.pdf
Green, A. (2008). English profile: Functional progression in materials for ELT. Research Notes, 33, 19–25.
Hidri, S. (2021). Linking the International English Language Competency Assessment suite of examinations to the Common European Framework of Reference. Language Testing in Asia, 11(1), 1–24. https://doi.org/10.1186/s40468-021-00123-8
Huang, L., Kubelec, S., Keng, N., & Hsu, L. (2018). Evaluating CEFR rater performance through the analysis of spoken learner corpora. Language Testing in Asia, 8(1), 1–17. https://doi.org/10.1186/s40468-018-0069-0
Jaturapitakkul, N. & Todd, R. W. (2018). Test of English for Thai engineers and technologists (TETET). Technical Report.
https://sola.kmutt.ac.th/tetet/doc/TETET_Technical_report_NJ_RT_Complete.pdf
Jin, Y., Wu, Z., Alderson, C., & Song, W. (2017). Developing the China standards of English: Challenges at macropolitical and micropolitical levels. Language Testing in Asia, 7(1), 1–19. https://doi.org/10.1186/s40468-017-0032-5
Kane, M. T. (2012). Validating score interpretations and uses. Language Testing, 29(1), 3–17. https://doi.org/10.1111/jedm.12000
Kirkpatrick, A. (2008). English as the official working language of the Association of Southeast Asian Nations (ASEAN): Features and strategies. English Today, 24(2), 27–34. https://doi.org/10.1017/S0266078408000175
Knoch, U., & Frost, K. (2016). Linking the GEPT writing sub-test to the Common European Framework of Reference (CEFR) (LTTC-GEPT Research Report No. RG-08). The Language Training and Testing Center.
Milanović, M. (2011). The construct of reading and its operationalization in the internet-based test of English as a foreign language. Philologia, 9(1), 73–82. https://doi.org/10.18485/philologia.2011.9.9.8
Nakanitanon, P. (2021). Linking an English proficiency test to the CEFR: Setting validity cut scores. The New English Teacher, 15(1), 53–64.
North, B. (2007). The CEFR: Development, theoretical and practical issues. Babylonia, 1, 22–29.
North, B. (2011). Putting the Common European Framework of Reference to good use. Language Teaching, 47(2), 1–22.
Office of the Basic Education Commission. (2014). Guidelines for practices by Ministry of Education: English education policy reform. http://upload.snru.ac.th/download.aspx?NFILE=24569561651071773219718420214022421089204114
Office of the Higher Education Commission. (2015). Policy of upgrading English education standards of higher education institutions. http://www.dqe.mhesi.go.th/front_home/Data%20Bhes_2559/04052559.pdf
O’Sullivan, B. (2010). The city & guilds communicator examination linking project: A brief overview with reflections on the process. In W. Martyniuk (Ed.), Relating language examinations to the Common European Framework of Reference for Languages: Case studies and reflections on the use of the Council of Europe’s draft manual (pp. 33–49). Cambridge University Press.
Papageorgiou, S. (2007). Relating the Trinity College London GESE and ISE examinations to the Common European Framework of Reference (Final Project Report, February 2007). Trinity College London. http://www.trinitycollege.co.uk/resource/?id=2261.
Papageorgiou, S. (2010). Investigating the decision-making process of standard setting participants. Language Testing, 27(2), 261–282.
Poonpon, K. (2021). Test takers’ perceptions of design and implementation of an online language testing system at a Thai university during the COVID-19 pandemic. PASAA, 62, 1–28.
Reynolds, C. R., Altmann, R. A., & Allen, D. N. (2021). The meaning of test scores. In C. R. Reynolds, R. A. Altman & D. N. Allen (Eds.), Mastering modern psychological testing (pp. 91–131). Springer. http://dx.doi.org/10.1007/978-3-030-59455-8_3
Shin, S. Y., Sanonguthai, S., & Tangkiengsirisin, S. (2022). Aligning the CEFR levels with the TU-GET CBT scores. Thammasat University. https://litu.tu.ac.th/wp-content/uploads/2023/09/Aligning-the-CEFR-Levels-withthe-TU-GET-CBT-Scores.pdf
Srisawat, C., & Poonpon, K. (2023). Revision of an academic English writing rubric for a graduate school admission test. PASAA, 65, 234–262.
Tannenbaum, R. J., & Cho, Y. (2014). Critical factors to consider in evaluating standard-setting studies to map language test scores to frameworks of language proficiency. Language Assessment Quarterly, 11, 233–249. https://doi.org/10.1080/15434303.2013.869815
Tannenbaum, R. J., & Wylie, E. C. (2008). Linking English-language test scores onto the Common European Framework of Reference: An application of standard-setting methodology. ETS Report Series, 2008(1), 1–75.
Tannenbaum, R. J., & Wylie, E. C. (2019). Mapping the TOEIC Tests on the Common European Framework of Reference. Educational Testing Service. https://www.ets.org/pdfs/toeic/toeic-mapping-cefr-reference.pdf
Thongyoi, K., & Poonpon, K. (2020). Phrasal complexity measures as predictors of EFL university students’ English academic writing proficiency. rEFLections, 27(1), 44–61. https://doi.org/10.61508/refl.v27i1.241750
Wu, J. R., & Wu, R. Y. (2007). Using the CEFR in Taiwan: The perspective of a local examination board. The Language Training and Testing Center Annual Report, 56, 1–20.
Wu, J. R., & Wu, R. Y. (2010). Relating the GEPT reading comprehension tests to the CEFR. In W. Martyniuk (Ed.), Aligning tests with the CEFR (pp. 204–224). Cambridge University Press.
Wudthayagorn, J. (2018). Mapping the CU-TEP to the Common European Framework of Reference (CEFR). LEARN Journal, 11(2), 163–180.
Xinli, D. (2019). Linking college English reading comprehension tests band four to the Common European Framework of Reference for Languages (CEFR) [Unpublised master’s thesis]. Suranaree University of Technology.
Zou, Y. Y. (2016). Aligning National Matriculation English Test (NMET) with the Common European Framework of Reference (CEFR) [Unpublished master's thesis]. Guangdong University of Foreign Language Studies.