Development of Integrated Concepts into School Management Best Practices Selection Manual for Sciences, Mathematics, and Technology based on IPST Quality School Standards: Psychometric Property Evidences of Interrater Agreement

Abstract

        The objectives of this research were: (1) to develop indicators, evaluation checklists, criteria, and a manual for evaluating school management and best practices of Sciences, Mathematics, and Technology (SMT) schools based on IPST quality school standards; (2) to implement the manual for evaluation and selection of SMT schools; and (3) to validate quality of the manual for evaluation and selection of SMT schools. The research results indicated that (1) the manual for evaluation and selection of SMT schools consisted of introduction, objectives, scope of selection, a form for submitting integrated concepts into school management best practices, scoring framework, and guidelines for submission; there were 5 indicators of SMT school management and 7 indicators of SMT school best practice; (2) the 21 experts applied the manual for evaluating and selecting 191 SMT schools; and (3) the qualities of manual as considered by inter-rater agreement of c² = 107.89, df =9, p = .00, and measure of agreement (Kappa) = .59, p = .00. In addition, the inter-rater reliability ranged from .733-.970, intra-class-correlation coefficient ranged from .837-.978, and G-coefficient of .85.