A Randomized Controlled Trial of Data Driven Science Education Reform to Investigate an Educational Value-Added Outcome, Project Sustainability, and Its Impact on Educational Value-Added Outcome and Growth of Secondary School Science Quality

Abstract

        The goal of this research was to conduct an experiment to assess the extent to which the data-driven reform approach was effective in enhancing the quality of science instruction of science teachers. The objectives of this research included 1) to assess quality of science instruction, determine strengths and weaknesses of science instruction, to provide informative assessment reports to teachers and students, and  to monitor how teachers solved their instructional problems,  2) to examine the value-added of student achievement which was a consequence of the data-driven reform intervention, 3) to examine the impact of the data-driven reform intervention on the value-added scores and the achievement growth of secondary school students, 4) to examine conditions in which value-added scores and  achievement growth could be leveraged, 5) to provide guidelines to leverage value-added scores and achievement growth in the data-driven reform context, and 6) to analyze the sustainability of the data-driven reform intervention.  Samples were 78 teachers and 2,569 students at the secondary school level. These samples were drawn from a combination of purposive sampling and voluntary basis and78 teachers were selected. The selected 48 teachers were assigned to the treatment group and 30 teachers were assigned to the control group. There were 1,415 students in the treatment group, while the control group had 1,154 students. Instruments used in this research were science achievement tests and classroom instruction questionnaires for teachers and students. Data was analyzed through content analysis, descriptive statistics, Chi-square, diagnostic assessment using G-DINA model, t-test, multiple regression and multilevel analysis.

         The major findings were as follows:

1. The most significant weakness of students was the ability to use scientific evidence, followed by the ability to explain phenomena scientifically, and the ability to identify scientific issues, respectively.

         2.Value-added scores ranged from -3.951 to 2.474. There were 41 classrooms (52.56%) with positive value-added scores and 37 classrooms (47.44%) with negative value-added scores. Nearly all classrooms with positive value-added scores were from the experiment group.

1. According to the comparison of value-added scores between the treatment and control groups, the treatment group had higher value-added scores than the control group in 8 subjects and there was no statistical difference in the value-added score between the treatment and control groups in 3 subjects.


2. Factors influencing value-added scores were the student-centered instruction with effective feedback. However, there was no factor that influenced the growth in students’ achievement.


3. To leverage value-added scores and students’ growth, it was suggested to improve teachers’ instructional and assessment practices, role and leadership of school administrations, and student’s motivation to learn and academic responsibility.


4. After controlling for gender, socioeconomic status, pretest score, teachers’ teaching experience, school size, and location, it was found that the experimental group and the control group had no difference in posttest scores, suggesting that the sustainability of the data-driven reform intervention was attained.