Teacher Effectiveness Initiatives

12.29.11 By Jesse Solomon

BTR works to prepare all of its residents to be data-driven teachers: teachers who are constantly looking at what their students are learning and using that information to plan instruction. We do the same thing as a program, looking at how our residents and graduates are doing and using that information to plan for improvement. Recently, BTR initiated several studies to gain more information about the effectiveness of our graduates, using emerging methodologies to identify what we are doing well and where we can improve.

This habit of self-analysis is not new: Since we started in 2003, we have conducted surveys of principals, tracked our success recruiting and retaining teachers in high-needs areas, and gathered lots of data. We’ve met many of the program’s original targets: recruiting talented, diverse, committed people who stay in the BPS for much longer than the average new teacher and who are highly rated by their principals. And yet… we wanted to know more about the learning of students in BTR graduates’ classes. After all, that’s what has us all doing this work.

We recently commissioned a study by Harvard’s Center for Educational Policy Research (CEPR) that uses value-added modeling to examine student performance on the MCAS. Value-added modeling estimates the contributions that teachers make to the learning of the students in their classrooms.  While there is currently a lively debate about the uses and efficacy of value-added modeling, and no one methodology can provide a comprehensive picture of a teacher or a program’s effectiveness, we do feel that this study provides valuable information.

A report with preliminary results of this ongoing study has just been released by CEPR. The findings are mixed: BTR graduates in their first few years teaching 4th-8th grade math appear to be slightly less effective at raising students’ MCAS scores than other teachers with comparable experience. But BTR graduates catch up and are actually more effective than their peers by their fourth and fifth years in the classroom.

Given BTR’s very high teacher retention rates (80% of BTR grads continue teaching for three years, compared with 63% of other new BPS teachers; 75% stay to year five, compared with 51% of their colleagues), the CEPR study reports that BTR graduates are likely to have a significant, positive impact on student learning in the long term. In English language arts, the study found no significant different between BTR graduates and other BPS teachers.

Although this first study could only include 20% of BTR graduates, and none who teach at the high school level, its preliminary results are informative and actionable. The study highlights several of BTR’s successes, while giving the organization clear indications of areas for further focus and improvement.

We at BTR still have much to learn about the effectiveness of our graduates. We have one goal in evaluating our work—to get better results for students—and we believe that transparency regarding our data and operations is crucial. The discussions we are having about these data have already led us to make significant changes in our program, which will ultimately accelerate our improvement, and, we hope, benefit the field as a whole.  We welcome your comments and questions.

Read a summary of the CEPR findings in Education Week.

Read our own summary, with more details about the ongoing improvement process at BTR.

Read the full study from CEPR at the NBER website (subscribers only – though it will soon be available for free at the CEPR website).