Development of Item Pools Strategies by Using Interval a-parameter-Stratification with Content Balancing
Article Sidebar
Main Article Content
Abstract
The item pools strategy is a procedure for organizing and categorizing test items to be ready for use. This research aimed to 1) develop the interval a-parameter- stratification with content balancing item pools strategy; 2) compare the efficiency of the developed strategy with the constraint-weighted a-stratification method using simulation scenario of test-takers’ estimative efficiency, that is, average bias and root mean square error (RMSE), employing Wilcox Test, and item utilizable efficiency, that is, overexposed items, underutilized items, item overlap rate, and item exposure rate distribution, using Chi-square test analysis.
The results showed that 1) there were three steps of the interval a-parameter- stratification with content balancing item pools strategy: First, dividing Item pools into 4 classes interval a-parameter- stratification. Second, balancing of each class of the item pools according to the subject matter of the test. Third, controlling Item exposure with Stratified Random Sampling; 2) the developed strategy with the test-takers’ performance in each competency level of the discriminative power and content balancing method was more efficient than the constraint-weighted a-stratification method.
Article Details
References
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., & Wittrock, M. C. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Abridged Edition. White Plains, New Jersey: Longman.
Babcock, B., & Weiss, D. J. (2013). Termination criteria in computerized adaptive tests: Do variable-length CATs provide efficient and effective measurement?. Journal of Computerized Adaptive Testing, 1(1), 1-18.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of Educational Objetives: the Classification of Educational Goals: Handbook I: Cognitive Domain (No. 373.19 C734t). New York, US: D. Mckay.
Chang, H. H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213-229.
Cheng, Y., Chang, H. H., Douglas, J., & Guo, F. (2009). Constraint-weighted a-stratification for computerized adaptive testing with nonstatistical constraints: Balancing measurement efficiency and exposure control. Educational and Psychological Measurement, 69(1), 35-49.
De Ayala, R. J. (2009). The Theory and Practice of Item Response Theory. London: The Guilford Press.
Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Mahwah, New Jersey: Lawrence Erlbaum.
Finch, H. (2010). Item parameter estimation for the MIRT model bias and precision of confirmatory factor analysis-based models. Applied Psychological Measurement, 34(1), 10-26.
Georgiadou, E. G., Triantafillou, E., & Economides, A. A. (2007). A review of item exposure control strategies for computerized adaptive testing developed from 1983 to 2005. The Journal of Technology, Learning and Assessment, 5(8), 1-39.
Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory into Practice, 41(4), 212-218.
Ozturk, N. B., & Dogan, N. (2015). Investigating item exposure control methods in computerized adaptive testing. Educational Sciences: Theory and Practice, 15(1), 85-98.
Reckase, M. (2009). Multidimensional Item Response Theory (Vol. 150). New York: Springer.
Segall, D. O. (2010). Principles of Multidimensional Adaptive Testing. In Elements of AdaptiveTtesting (pp. 57-75). Springer New York.
Seo, D. G. (2011). Application of the Bifactor Model to Computerized Adaptive Testing. ProQuest.
Taylor, B. W. (2009). Introduction to Management Science (11th ed.). Upper Saddle River, New Jersey: Prentice Hall.
Thissen, David. (1990). Reliability and Measurement Precision. In: Howard Wainer et al.,editors. Computerized Adaptive Testing: A Primer. New Jersey: Lawrence Erlbaum; 1990. pp. 161-186.
Thompson, N. A., & Weiss, D. J. (2011). A framework for the development of computerized adaptive tests. Practical Assessment, Research & Evaluation, 16(1), 1-9.
Weiss, D. J. (2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), 1-27.
Urry, V. W. (1977). Tailored testing: A successful application of latent Trait Theory. Journal of Educational Measurement, 14(2), 181-196.
Yan, D., von Davier, A. A., & Lewis, C. (2016). Computerized Multistage Testing: Theory and Applications. Princton, New Jersey: CRC Press.
Zhou, X. (2012). Designing P-Optimal Item Pools in Computerized Adaptive Tests with Polytomous Items. ProQuest LLC.