I’m a public school teacher.
Am I any good at my job?
There are many ways to find out. You could look at how hard I work, how many hours I put in. You could look at the kinds of things I do in my classroom and examine if I’m adhering to best practices. You could look at how well I know my students and their families, how well I’m attempting to meet their needs.
Or you could just look at my students’ test scores and give me a passing or failing grade based on whether they pass or fail their assessments.
It’s called Value-Added Measures (VAM) and at one time it was the coming fad in education. However, after numerous studies and lawsuits, the shine is fading from this particularly narrow-minded corporate policy.
Most states that evaluate their teachers using VAM do so because under President Barack Obama they were offered Race to the Top grants and/or waivers.
Now that the government isn’t offering cash incentives, seven states have stopped using VAM and many more have reduced the weight given to these assessments. The new federal K-12 education law – the Every Student Succeeds Act (ESSA) – does not require states to have educator evaluation systems at all. And if a state chooses to enact one, it does not have to use VAM.
That’s a good thing because the evidence is mounting against this controversial policy. An evaluation released in June of 2018 found that a $575 million push by the Bill and Melinda Gates Foundation to make teachers (and thereby students) better through the use of VAM was a complete waste of money.
Meanwhile a teacher fired from the Washington, DC, district because of low VAM scores just won a 9-year legal battle with the district and could be owed hundreds of thousands of dollars in back pay as well as getting his job back.
But putting aside the waste of public tax dollars and the threat of litigation, is VAM a good way to evaluate teachers?
Is it fair to judge educators on their students’ test scores?
Here are the top 10 reasons why the answer is unequivocally negative:
1) VAM was Invented to Assess Cows.
I’m not kidding. The process was created by William L. Sanders, a statistician in the college of business at the University of Knoxville, Tennessee. He thought the same kinds of statistics used to model genetic and reproductive trends among cattle could be used to measure growth among teachers and hold them accountable. You’ve heard of the Tennessee Value-Added Assessment System (TVAAS) or TxVAAS in Texas or PVAAS in Pennsylvania or more generically named EVAAS in states like Ohio, North Carolina, and South Carolina. That’s his work. The problem is that educating children is much more complex than feeding and growing cows. Not only is it insulting to assume otherwise, it’s incredibly naïve.
2) You can’t assess teachers on tests that were made to assess students.
This violates fundamental principles of both statistics and assessment. If you make a test to assess A, you can’t use it to assess B. That’s why many researchers have labeled the process “junk science” – most notably the American Statistical Association in 2014. Put simply, the standardized tests on which VAM estimates are based have always been, and continue to be, developed to assess student achievement and not growth in student achievement nor growth in teacher effectiveness. The tests on which VAM estimates are based were never designed to estimate teachers’ effects. Doing otherwise is like assuming all healthy people go to the best doctors and all sick people go to the bad ones. If I fail a dental screening because I have cavities, that doesn’t mean my dentist is bad at his job. It means I need to brush more and lay off the sugary snacks.
3) There’s No Consistency in the Scores.
Valid assessments produce consistent results. This is why doctors often run the same medical test more than once. If the first try comes up positive for cancer, let’s say, they’re hoping the second time will come up negative. However, if multiple runs of the same test produce the same result, that diagnosis gains credence. Unfortunately, VAM scores are notoriously inconsistent. When you evaluate teachers with the same test (but different students) over multiple years, you often get divergent results. And not just by a little. Teachers who do well one year may do terribly the next. This makes VAM estimates extremely unreliable. Teachers who should be (more or less) consistently effective are being classified in sometimes highly inconsistent ways over time. A teacher classified as “adding value” has a 25 to 50% chance of being classified as “subtracting value” the next year, and vice versa. This can make the probability of a teacher being identified as effective no different than the flip of a coin.
4) Changing the test can change the VAM score.
If you know how to add, it doesn’t matter if you’re asked to solve 2 +2 or 3+ 3. Changing the test shouldn’t have a major impact on the result. If both tests are evaluating the same learning and at the same level of difficulty, changing the test shouldn’t change the result. But when you change the tests used in VAM assessments, scores and rankings can change substantially. Using a different model or a different test often produces a different VAM score. This may indicate a problem with value added measures or with the standardized tests used in conjunction with it. Either way, it makes VAM scores invalid.
5) VAM measures correlation, not causation.
Sometimes A causes B. Sometimes A and B simply occur at the same time. For example, most people in wheelchairs have been in an accident. That doesn’t mean being in a wheelchair causes accidents. The same goes for education. Students who fail a test didn’t learn the material. But that doesn’t mean their teacher didn’t try to teach them. VAM does not measure teacher effectiveness. At best it measures student learning. Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model. For instance, the student may have a learning disability, the student may have been chronically absent or the test, itself, may be an invalid measure of the learning that has taken place.
6) Vam Scores are Based on Flawed Standardized Tests.
When you base teacher evaluations on student tests, at very least the student tests have to be valid. Otherwise, you’ll have unfairly assessed BOTH students AND teachers. Unfortunately standardized tests are narrow, limited indicators of student learning. They leave out a wide range of important knowledge and skills leaving only the easiest-to-measure parts of math and English curriculum. Test scores are not universal, abstract measures of student learning. They greatly depend on a student’s class, race, disability status and knowledge of English. Researchers have been decrying this for decades – standardized tests often measure the life circumstances of the students not how well those students learn – and therefore by extension they cannot assess how well teachers teach.
7) VAM Ignores Too Many Factors.
When a student learns or fails to learn something, there is so much more going on than just a duality between student and teacher. Teachers cannot simply touch students’ heads and magically make learning take place. It is a complex process involving multiple factors some of which are poorly understood by human psychology and neuroscience. There are inordinate amounts of inaccurate or missing data that cannot be easily replaced or disregarded – variables that cannot be statistically controlled for such as: differential summer learning gains and losses, prior teachers’ residual effects, the impact of school policies such as grouping and tracking students, the impact of race and class segregation, etc. When so many variables cannot be accounted for, any measure returned by VAMs remains essentially incomplete.
8) VAM Has Never been Proven to Increase Student Learning or Produce Better Teachers.
That’s the whole purpose behind using VAM. It’s supposed to do these two things but there is zero research to suggest it can do them. You’d think we wouldn’t waste billions of dollars and generations of students on a policy that has never been proven effective. But there you have it. This is a faith-based initiative. It is the pet project of philanthrocapitalists, tech gurus and politicians. There is no research yet which suggests that VAM has ever improved teachers’ instruction or student learning and achievement. This means VAM estimates are typically of no informative, formative, or instructional value.
9) VAM Often Makes Things Worse.
Using these measures has many unintended consequences that adversely affect the learning environment. When you use VAMs for teacher evaluations, you often end up changing the way the tests are viewed and ultimately the school culture, itself. This is actually one of the intents of using VAMs. However, the changes are rarely positive. For example, this often leads to a greater emphasis on test preparation and specific tested content to the exclusion of content that may lead to better long-term learning gains or increasing student motivation. VAM incentivizes teachers to wish for the most advanced students in their classes and to push the struggling students onto someone else so as to maximize their own personal VAM score. Instead of a collaborative environment where everyone works together to help all students learn, VAM fosters a competitive environment where innovation is horded and not shared with the rest of the staff. It increases turnover and job dissatisfaction. Principals stack classes to make sure certain teachers are more likely to get better evaluations or vice versa. Finally, being unfairly evaluated disincentives new teachers to stay in the profession and it discourages the best and the brightest from ever entering the field in the first place. You’ve heard about that “teacher shortage” everyone’s talking about. VAM is a big part of it.
10) An emphasis on VAM overshadows real reforms that actually would help students learn.
Research shows the best way to improve education is system wide reforms – not targeting individual teachers. We need to equitably fund our schools. We can no longer segregate children by class and race and give the majority of the money to the rich white kids while withholding it from the poor brown ones. Students need help dealing with the effects of generational poverty – food security, psychological counseling, academic tutoring, safety initiatives, wide curriculum and anti-poverty programs. A narrow focus on teacher effectiveness dwarfs all these other factors and hides them under the rug. Researchers calculate teacher influence on student test scores at about 14%. Out-of-school factors are the most important. That doesn’t mean teachers are unimportant – they are the most important single factor inside the school building. But we need to realize that outside the school has a greater impact. We must learn to see the whole child and all her relationships –not just the student-teacher dynamic. Until we do so, we will continue to do these children a disservice with corporate privatization scams like VAM which demoralize and destroy the people who dedicate their lives to helping them learn – their teachers.
NOTE: Special thanks to the amazingly detailed research of Audrey Amrein-Beardsley whose Vamboozled Website is THE on-line resource for scholarship about VAM.
Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!
WANT A SIGNED COPY?
[…] Top 10 Reasons You Can’t Fairly Evaluate Teachers on Student Test Scores […]
LikeLiked by 1 person
Thank you! I didn’t know about the cows.
Rosie
On Mon, Aug 6, 2018, 12:21 PM gadflyonthewallblog wrote:
> stevenmsinger posted: ” I’m a public school teacher. Am I any good at > my job? There are many ways to find out. You could look at how hard I > work, how many hours I put in. You could look at the kinds of things I do > in my classroom and examine if I’m ad” >
LikeLike
Strange but true.
LikeLike
I am amazed! And really disgusted. Cattle? Really? Cattle? I’m so glad I retired even though I mss teaching. This is unbelievable!
LikeLike
[…] school excellent. They want all children to have the resources they need to succeed. They want to assess students, teachers and the system fairly to clearly understand what children are learning, what educators are doing to help them learn and […]
LikeLike
Reblogged this on David R. Taylor-Thoughts on Education.
LikeLike
[…] Source: Top 10 Reasons You Can’t Fairly Evaluate Teachers on Student Test Scores | gadflyonthewallblog […]
LikeLike
Reblogged this on caffeinated rage.
LikeLike
[…] find out which diagnostic exams you have to give your students and when. You find out what your Pennsylvania Value Added Assessment Score (PVAAS) is – how good a teacher you are based on how well your students from last year did on the […]
LikeLike
[…] out which diagnostic exams you have to give your students and when. You find out what your Pennsylvania Value Added Assessment Score (PVAAS) is – how good a teacher you are based on how well your students from last year did […]
LikeLike
[…] stakes standardized testing isn’t about helping students learn. Neither is Common Core, value-added measures or a host of top-down corporate policies championed by lions of the left and supply-side […]
LikeLike
[…] stakes standardized testing isn’t about helping students learn. Neither is Common Core, value-added measures or a host of top-down corporate policies championed by lions of the left and supply-side patriots. […]
LikeLike
[…] their employees where doing a good job. Now even that decision has been taken away and replaced by junk science formulas that claim to evaluate a teacher’s entire impact on a student’s life with …. However, local principals and administrators are there in the school building every day. They know […]
LikeLike
[…] Teaching evaluations are still based on test scores. Schools evaluations are still based on test scores. […]
LikeLike
[…] highly suspect practice of evaluating teachers on student test scores has been dropped in Connecticut and the weight it is given has been reduced in New […]
LikeLike
[…] 11) Top 10 Reasons You Can’t Fairly Evaluate Teachers on Student Test Scores […]
LikeLike
[…] would add that we also need to design fair evaluation systems for both principals and teachers that aren’t based …. We need to stop contracting out our assessments to corporations and trust our systems of […]
LikeLike
[…] to withhold their diplomas and any chance of ever earning more than minimum wage. In many cases we tie teachers salaries, reputations and even employment to these same scores. Sometimes we even take away their parents right to govern their children’s schools so that all […]
LikeLike
[…] white kids who get the highest test scores doesn’t help the struggling students. It just means fewer educators will want to teach the underprivileged because they can’t take the financial hit that comes with it. […]
LikeLike
[…] the end of the year, I always give my students a survey to gauge how they think I did as their teacher. It’s not graded, and they can even turn it in […]
LikeLike
[…] First of all, the federal government requires that all public school children take these assessments in 3-8th grade and once in high school. Second, many states require teachers be evaluated by their students’ test scores. […]
LikeLike
[…] You can’t raise expectations while taking away resources, union protections, and fair ways to evaluate their work. […]
LikeLike
[…] year I’m told that my worth as a professional is mainly defined by student test scores – that I should use those scores to drive my entire class, that my major goal should be […]
LikeLike
[…] and local governments often still insist on enacting it despite all the evidence against it. Teachers have literally committed suicide over these unfair evaluations, but it hasn’t stopped Gates from continuing to experiment on the rest of humanity with his […]
LikeLike
[…] is less critical of high stakes testing. He sees more of a problem in using student test scores to assess teacher performance. But even he thinks the tests and the scores are being over valued and misunderstood in a wider […]
LikeLike
[…] to do their jobs, they would be empowered to accomplish more. And I don’t mean blind trust. I don’t mean closing our eyes and letting teachers do whatever they want unimpeded, unadvised and …. I mean letting teachers do the work in the full light of day with observation by trained […]
LikeLike
[…] That may seem simple or even obvious with reflection, but it also goes counter to nearly every teacher evaluation system in practice in the United States. […]
LikeLike
[…] that resulted in a teachers strike. He still doesn’t comprehend why this was a bad idea – that tying teachers salaries to student test scores makes for educators who only teach to the test, …. Moreover, he thinks there’s a difference between public and private charter schools – […]
LikeLike
[…] that resulted in a teachers strike. He still doesn’t comprehend why this was a bad idea – that tying teachers salaries to student test scores makes for educators who only teach to the test, …. Moreover, he thinks there’s a difference between public and private charter schools – there […]
LikeLike
[…] what the issue – the school-to-prison pipeline, Common Core, racist discipline policies, value added teacher evaluations, runaway ed tech – we’ve come together to fight as […]
LikeLike
[…] evaluating teachers and schools based on the poverty, race and ethnicities of the children they […]
LikeLike
[…] need to be free to try something and not be able to codify why they’re doing it at the moment. Only later, perhaps at the end of the day, can it be helpful to sit back and reflect on what you […]
LikeLike
[…] micromanagement. Should administrators monitor what their teachers are doing? Absolutely. But the best way to do that is to actually observe the teacher in the classroom doing the work. And to conference with the teacher before and after the observation with the goal of understanding […]
LikeLike
[…] No more teaching to the test. No more narrowing the curriculum. No more pressure to increase test scores. […]
LikeLike
[…] Student test scores are poor ways to assess teachers. The best way is peer observation of teachers in a classroom context with the nonpunitive goal of improving instruction. […]
LikeLike
[…] also championed the idea that competing for test scores would result in better teachers. However, that didn’t happen either. Instead, educators were forced to narrow the curriculum to cover mostly what was assessed, reduce […]
LikeLike
[…] So this proposed teacher preparation and professional development is of what kind exactly? I’ll bet it’s mostly reeducation to accept corporate education reform. I’ll bet it’s focused on ways to increase student test scores which will then be used to evaluate teacher effectiveness – a program that has been roundly disproven for decades. […]
LikeLike
[…] So this proposed teacher preparation and professional development is of what kind exactly? I’ll bet it’s mostly reeducation to accept corporate education reform. I’ll bet it’s focused on ways to increase student test scores which will then be used to evaluate teacher effectiveness – a program that has been roundly disproven for decades. […]
LikeLike
Great read thankyyou
LikeLike
[…] is why we never get rid of standardized testing, charter schools, evaluating teachers on student test scores, and a hundred other practices that have demonstrably failed over-and-over […]
LikeLike
[…] use such measures to sneak in unnecessary and destructive policies like more standardized testing, evaluating teachers on student test scores and increased funding for charter schools and school voucher […]
LikeLike