Estimated Parameters of the Testlet-Test under Item Response Theory and Testlet Response Theory Model
Article Sidebar
Published:
May 20, 2018
Main Article Content
Abstract
This study aimed to investigate an estimation of the parameters of the testlet-test including discrimination, difficulty, guessing, and ability parameter estimated under 3PL IRT and the 3PL TRT Models. Item scores of English Language of 1,500 Grade-9 students’ tests from the 2015 Ordinary National Education Test (O-NET), determined by multi-stage sampling, were used for estimating the parameters of the test. The parameters were then analyzed for Pearson’s Correlation Coefficient, and the standard errors (SE) of the two models were compared by Paired t-test.
Results revealed that;
- Estimated parameters of the Testlet-Test (discrimination, difficulty, and guessing) and ability parameters of 3PLIRT and the 3PL TRT were significantly correlated in all aspects (p<.05), with very high correlation (0.935, 0.983, 0.972 and 0.994 respectively).
- Standard error of the item parameters (discrimination, difficulty, and guessing) and ability parameters estimated under 3PL IRT and the 3PL TRT were significantly different in all aspects (p<.05).
Article Details
Section
Research Article
References
กระทรวงศึกษาธิการ. (2559). สถิติทางการศึกษาปี 2558. วันที่ค้นข้อมูล 30 กันยายน 2559, เข้าถึงได้จาก http://www.bopp-obec.info/home/wp-content/ uploads/ 2015/09/t32.pdf.
ศิริชัย กาญจนาวสี. (2555). ทฤษฎีการทดสอบแนวใหม่ (Modern test theory) (พิมพ์ครั้งที่ 4). กรุงเทพฯ : โรงพิมพ์แห่งจุฬาลงกรณ์มหาวิทยาลัย.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability.In F.M. Lord & M.R. Novick (Eds.), Statistical theories of mental test score(pp. 397-479).Reading, MA: Addison-Wesley.
Campbell, D.T., & Fiske, D.W. (1959). Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56, 81-105.
Chen, J. (2014). Model selection for IRT equating of Testlet-based tests in therandom groups design. PhD (Doctor of Philosophy) thesis, University of Iowa.
De Ayala, R. J. (2009). The theory and practice of item response theory. New York : Guilford.
Demars, C.E. (2006).Application of the bifactormultidimentionsional item response theory model to testlet-based test. Journal of Educational Measurement, 43, 145-168.
Eckes, T. (2014). Examining testlet effects in the TestDaf listening section: A testlet response. Language Testing, 31(1), 39-61.
Foley, B. F. (2010). Improving IRT item parameter estimates with small sample sizes: Evaluating the efficacy of a new data augmentation technique. Doctoral dissertation, Philosophy, Graduate College at the University of Nebraska.
Kogar, E. Y., & Kelecioglu, H. (2017). Examination of Different ItemResponse Theory Models on Tests Composed of Testlets. Journal of Education and Learning, 6(4), 113-126.
Lam, W. (2009). Linking current and future score scales for the AICPA Uniform CPA Exam. Boston: Technical Report in University of Massachusetts Amherst.
Lee, G., Kolen, M. J., Frisbie, D. A., &Ankenmann, R. D. (2001). Comparison of dichotomous and polytomous item response models in equating scores from tests composedof testlets. Applied Psychological Measurement, 25, 357-372.
Lei, M., Lomax, & R.G., (2005). The effect of varying degrees of nonnormality instructural equation modeling. structural equation modeling. 12 (1), 1–27.
Min, S., & He, L. (2014). Applying unidimensional item response theory models in testlet-basted reading assessment. Language Testing, 31(4), 453-447.
Sireci, S. G., Thissen, D.,& Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
Wainer, H., Bradlow, E. T., & Wang, X. (2007).Testlet Response Theory and Its Application. Cambridge,UK: Cambridge University Press.
Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A case for testlet. Journal of Educational Measurement, 24, 185-201.
Wang, X., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Applied Psychological Measurement, 26, 190-128.
Zhang, J. (2007). Dichotomous or Polytomous model? Equating of testlet-based tests in light of conditional item pair correlations. Philosophy degree in Psychological and Quantitative Foundations (Educational Measurement and Statistics) in the Graduate College of The University of Iow.
ศิริชัย กาญจนาวสี. (2555). ทฤษฎีการทดสอบแนวใหม่ (Modern test theory) (พิมพ์ครั้งที่ 4). กรุงเทพฯ : โรงพิมพ์แห่งจุฬาลงกรณ์มหาวิทยาลัย.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability.In F.M. Lord & M.R. Novick (Eds.), Statistical theories of mental test score(pp. 397-479).Reading, MA: Addison-Wesley.
Campbell, D.T., & Fiske, D.W. (1959). Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56, 81-105.
Chen, J. (2014). Model selection for IRT equating of Testlet-based tests in therandom groups design. PhD (Doctor of Philosophy) thesis, University of Iowa.
De Ayala, R. J. (2009). The theory and practice of item response theory. New York : Guilford.
Demars, C.E. (2006).Application of the bifactormultidimentionsional item response theory model to testlet-based test. Journal of Educational Measurement, 43, 145-168.
Eckes, T. (2014). Examining testlet effects in the TestDaf listening section: A testlet response. Language Testing, 31(1), 39-61.
Foley, B. F. (2010). Improving IRT item parameter estimates with small sample sizes: Evaluating the efficacy of a new data augmentation technique. Doctoral dissertation, Philosophy, Graduate College at the University of Nebraska.
Kogar, E. Y., & Kelecioglu, H. (2017). Examination of Different ItemResponse Theory Models on Tests Composed of Testlets. Journal of Education and Learning, 6(4), 113-126.
Lam, W. (2009). Linking current and future score scales for the AICPA Uniform CPA Exam. Boston: Technical Report in University of Massachusetts Amherst.
Lee, G., Kolen, M. J., Frisbie, D. A., &Ankenmann, R. D. (2001). Comparison of dichotomous and polytomous item response models in equating scores from tests composedof testlets. Applied Psychological Measurement, 25, 357-372.
Lei, M., Lomax, & R.G., (2005). The effect of varying degrees of nonnormality instructural equation modeling. structural equation modeling. 12 (1), 1–27.
Min, S., & He, L. (2014). Applying unidimensional item response theory models in testlet-basted reading assessment. Language Testing, 31(4), 453-447.
Sireci, S. G., Thissen, D.,& Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
Wainer, H., Bradlow, E. T., & Wang, X. (2007).Testlet Response Theory and Its Application. Cambridge,UK: Cambridge University Press.
Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A case for testlet. Journal of Educational Measurement, 24, 185-201.
Wang, X., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Applied Psychological Measurement, 26, 190-128.
Zhang, J. (2007). Dichotomous or Polytomous model? Equating of testlet-based tests in light of conditional item pair correlations. Philosophy degree in Psychological and Quantitative Foundations (Educational Measurement and Statistics) in the Graduate College of The University of Iow.