The Effectiveness of Professional Development on Teacher Best Practices and Student Learning: Conflicting Results From Multimethod/Multisource Data Sets
Analyses of these data, which include pre-post comparisons and control vs. treatment comparisons, revealed areas of disconnect between gains and changes in behavior which are reflected in teacher self report data versus data relying on student perceptions, classroom observer perceptions, and student year to year gains in achievement.
Mean teacher participation indices ranged from a high of 70% (Cohort 2) to a low of 45% (Cohort 2). Teacher self-report instruments identified both gains in participation as well as barriers to implementation. Throughout the study the question of alignment and the question of individual time in planning and implementing the activities surfaced as a significant barrier for teachers across all cohorts.
Student achievement data on the California Standards Tests from 2003 to 2005 examined positive changes at the school building level in the ranks of proficiency (Far Below Basic, Below Basic, Basic, Proficient, and Advanced). Using two different analyses we found that there was not a statistically significant difference between treatment Schools and the wait-list Control schools.
An HLM analysis of student perceptions of mathematics lessons in the classroom, as measured by the Student Motivation Assessment survey, and correlated with student absolute achievement, indicated no differences between teachers who had or had not participated in Math ACTS. However, depending on grade level, several factors of the survey were significantly related to student achievement. Across all grade levels, student self-reported interest and competence in mathematics was positively correlated with math achievement.
Analysis of the classroom observation data provides validation of the instrument constructs. A principal components analysis indicates one meaningful component factor which explains 55% of the variation. The overall rating of effective instruction increases with a simultaneous increase in teacher accuracy and clarity of concept, student application of prior knowledge, student expression of reasoning, frequent assessment of student understanding, and teacher actions that encourage student persistence. However, there was no difference over three years of teacher classroom observations between treatment and control teachers in the delivery of mathematics lessons.
