A Case of the Validity Investigation of Concordance-based Cloze Testing: Construct Relevance Revisited
Main Article Content
Abstract
Investigating construct validity is a process that is essential especially for devising a new item type. However, the distinction between construct-relevant and construct-irrelevant variances may not always be sufficient. In this article, the construct of the concordance-based cloze item type (henceforth ConCloze) will be defined through two research projects. The first one is the doctoral dissertation (Kongsuwannakul, 2017 Investigating the Construct Validity of a Concordance-based Cloze Test: A Mixed-methods Study), in which the construct validity of the item type is defined with an iterative research design. Item components and a variety of changes to them are used insofar as information about the language processes and domains is obtained out of the data. The other research project is a follow-up study (Kongsuwannakul, 2019 Suranaree University of Technology Students’ Language Domains in Engaging with a Concordance-based Cloze Test: A Contrastive Approach), in which a contrastive approach is used for identifying distinguishing language domains in ConCloze. It is found that language domains such as knowledge of lexical semantics are not sufficiently distinguishing the construct of the item type. As for theoretical implications, it will be argued that construct-relevant variance could be divided into construct variance and construct-peripheral variance.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Cobb, T. (ca. 2011). Frequency Based Vocabulary Tests. Montreal: Tom Cobb; Université du Québec à Montréal. Available: http://www.lextutor.ca/tests/.
Cronbach, L. J., & Meehl, P. E. (1955). Construct Validity in Psychological Tests. Psychological Bulletin. 52(4): 281–302.
Embretson, S. E. (2007). Construct Validity: A Universal Validity System or Just Another Test Evaluation Procedure? Educational Researcher. 36(8): 449–455.
Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), The American Council on Education/Macmillan Series on Higher Education. Educational measurement (pp. 105–146). Macmillan Publishing Co, Inc; American Council on Education.
Javidanmehr, Z., & Sarab, M. R. A. (2019). Retrofitting Non-Diagnostic Reading Comprehension Assessment: Application of the G-DINA Model to A High Stakes Reading Comprehension Test. Language Assessment Quarterly. 16(3): 294–311. doi:10.1080/15434303.2019.1654479
Kongsuwannakul, K. (2019). Suranaree University of Technology Students' Language Domains in Engaging with a Concordance-based Cloze Test (ConCloze): A Contrastive Approach. (In Thai). Research report no. X2-203-61-12-06. X Province: X University.
Kongsuwannakul, K. (2017). Investigating the Construct Validity of a Concordance-based Cloze Test: A Mixed-methods Study. Ph.D. dissertation, University of Leicester. doi://hdl.handle.net/2381/40396
Lewis, M. (2000). Introduction. In M. Lewis (Ed.), Teaching Collocation: Further Developments in the Lexical Approach (pp. 8–9). Massachusetts, USA: Thomson.
Lissitz, R. W., & Samuelsen, K. (2007). A Suggested Change in Terminology and Emphasis Regarding Validity and Education. Educational Researcher. 36(8): 437-448. doi:10.3102/0013189X07311286
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education; Collier Macmillan.
Messick, S. (1995). Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. American Psychologist. 50(9): 741–749.
Mislevy, R. J. (2007). Validity by Design. Educational Researcher. 36(8): 463–469.
Moss, P. A. (2007). Reconstructing Validity. Educational Researcher. 36(8): 470–476.
Oller, J. W., Jr. (1979). Language Tests at School: A Pragmatic Approach. London: Longman.
Reardon, S. F., Kalogrides, D., Fahle, E. M., Podolsky, A., & Zárate, R. C. (2018). The Relationship Between Test Item Format and Gender Achievement Gaps on Math and ELA Tests in Fourth and Eighth Grades. Educational Researcher. 47(5): 284–294. Available: https://doi.org/10.3102/0013189X18762105
Schmitt, N., Dörnyei, Z., Adolphs, S., & Durow, V. (2004). Knowledge and Acquisition of Formulaic Sequences: A Longitudinal Study. In N. Schmitt (Ed.), Formulaic Sequences: Acquisition, Processing and Use. Amsterdam: John Benjamins, 54–86.
Sinclair, J. (2004). Trust the Text: Language, Corpus and Discourse. Routledge: London.