The article says that the system will calculate “the value teachers add to their students’ achievement, based on changes in test scores from year to year, and how the students perform compared with others in their grade.”
Education Secretary Arne Duncan briefly weighed in to support the newspaper’s work, calling it an exercise in health transparency.
“There are real issues and competing priorities and values that we must work through together — balancing transparency, privacy, fairness and respect for teachers,” Mr. Duncan was quoted in the NY Times as saying. On The Los Angeles Times’s publication of the teacher data, he added, “I don’t advocate that approach for other districts.”
However, as someone familiar with statistical research, I think caution is in order before such rankings are reported. For example, I would want to see how the scores play out over a period of time. What if the results vary widely from one year to another? In that case, you would be firing or embarrassing a teacher based on a single data-point, rather than a series of data.
Stanford Professor Edward Haertel noted that the methodology could be unreliable. “If these teachers were measured in a different year, or a different model were used, the rankings might bounce around quite a bit,” he was quoted in the New York Times article as saying. “People are going to treat these scores as if they were reflections on the effectiveness of the teachers, without any appreciation of how unstable they are.”
Defenders are quick to argue that these methods are sound – but is anyone truly comfortable using a single year’s data to determine whether someone should be retained or fired? I would want to see at least a few years of data to see if the ranking is stable.
If you see teachers’ ranks moving around from the top to the middle to the bottom, then you have a problem with the data. If you see stability in the rankings, then we can say that the rankings are stable, but not necessarily accurate. A single year should not be driving policy and personnel decisions, especially for a new test whose reliability no one can be truly certain about.
The other problem is that, while teachers are important, they are not the only variable in the equation. Work known as the “Broader, Bolder Approach” argues that “communities need to improve the educational environment, not just hire and fire new teachers.”
A group called Educational Justice asks, “Why do politicians and news writers avoid this broader-bolder approach?”
The answer, they say, is that, “It seems that they avoid it, argue against it, oppose it, because it would require that the politicians, the communities, the news writers do something positive, not simply cause the dismissal of some teachers.”
“Politicians would have to adequately fund the schools and the supporting health and community centers needed to promote education,” they continue.
One problem is that neither are excellent teachers are recognized, or poor teachers assisted, coached, and helped to improve.
Part of that problem is the assumption that poor teachers always have been that way, and always will remain that way. It may be that the professional environment of teachers eventually wears down the good ones, who would then appear to be poor teachers at one glance. While in reality, they’ve become frustrated over time at the inability of the system to address the needs of their students.
As the Economic Policy Institute argues, “Every classroom should have a well-educated, professional teacher, and school systems should recruit, prepare, and retain teachers who are qualified to do the job. Yet in practice, American public schools generally do a poor job of systematically developing and evaluating teachers.”
“Many policy makers have recently come to believe that this failure can be remedied by calculating the improvement in students’ scores on standardized tests in mathematics and reading, and then relying heavily on these calculations to evaluate, reward, and remove the teachers of these tested students,” the EPI writes on August 27.
They continue, “If new laws or policies specifically require that teachers be fired if their students’ test scores do not rise by a certain amount, then more teachers might well be terminated than is now the case.”
However, they argue, “there is not strong evidence to indicate either that the departing teachers would actually be the weakest teachers, or that the departing teachers would be replaced by more effective ones. There is also little or no evidence for the claim that teachers will be more motivated to improve student learning if teachers are evaluated or monetarily rewarded for student test score gains.”
“A review of the technical evidence leads us to conclude that, although standardized test scores of students are one piece of information for school leaders to use to make judgments about teacher effectiveness, such scores should be only a part of an overall comprehensive evaluation,” the EPI writs.
“Some states are now considering plans that would give as much as 50% of the weight in teacher evaluation and compensation decisions to scores on existing tests of basic skills in math and reading. Based on the evidence, we consider this unwise,” they continue. “Any sound evaluation will necessarily involve a balancing of many factors that provide a more accurate view of what teachers in fact do in the classroom and how that contributes to student learning.”
Anyone with a background in the classroom knows that there is a good amount of variability between classes. Some students are better than others. Some classes are better than others. I am very uncomfortable with a single year’s worth of tests supposedly determining the quality of a teacher. A bigger sample over time is needed to be able to properly measure that.
Moreover, teaching is not the only variable. Performance of individual students needs to be looked at over time as well.
The reports indicate that if the schools are not providing the proper resources and support for their teachers and not helping teachers improve, then simply firing teachers is no guarantee that performance will improve. All you may be achieving is to rotate a series of teachers through the system without much study or evidence guiding those decisions.
—David M. Greenwald reporting