Standardized Tests Are Not Objective Measures of Anything

Screen Shot 2019-06-29 at 12.27.12 AM

 
When it comes to standardized tests, most people are blinded by science.

 

Or at least the appearance of science.

 

Because there is little about these assessments that is scientific, factual or unbiased.

 

And that has real world implications when it comes to education policy.

 

First of all, the federal government requires that all public school children take these assessments in 3-8th grade and once in high school. Second, many states require teachers be evaluated by their students’ test scores.

 

Why?

 

It seems to come down to three main reasons:

 

1) Comparability
2) Accountability
3) Objectivity

 

COMPARABILITY

 

First, there is a strong desire to compare students and student groups, one with the other.

 

We look at learning like athletics. Who has shown the most success, and thereby is better than everyone else?

 

This is true for students in a single class, students in a single grade, an entire building, a district, a state, and between nations, themselves.

 

If we keep questions and grading methods the same for every student, there is an assumption that we can demonstrate which group is best and worst.

 

ACCOUNTABILITY

 

Second, we want to ensure all students are receiving the best education. So if testing can show academic success through its comparability, it can also be used as a tool to hold schools and teachers accountable. We can simply look at the scores and determine where academic deficiencies exist, diagnose them based on which questions students get incorrect and then focus there to fix the problem. And if schools and teachers can’t or won’t do that, it is their fault. Thus, the high stakes in high stakes testing.

 

Obviously there are other more direct ways to determine these facts. Historically, before standardized testing became the centerpiece of education policy, we’d look at resource allocation to determine this. Are we providing each student with what they need to learn? Do they have sound facilities, wide curriculum, tutoring, proper nutrition, etc.? Are teachers abiding by best practices in their lessons? Many would argue this was a better way of ensuring accountability, but if standardized assessments produce valid results, they are at least one possible way to ensure our responsibilities to students are being met.

 

OBJECTIVITY

 

Third, and most importantly, there is the assumption that of all the ways to measure learning, only standardized testing produces objective results. Classroom grades, student writing, even high school graduation rates are considered subjective and thereby inferior.

 

Questions and grading methods are identical for every student, and a score on the test is proof that a student is either good or bad at a certain subject. Moreover, we can use that score to keep the entire education system on track and ensure it is functioning correctly.

 

So this third reason for standardized testing is really the bedrock rationale. If testing is not objective, it doesn’t matter if it’s comparable or useful for accountability.

 

After all, we could hold kids accountable for the length of their hair, but if that isn’t an objective measure of what they’ve learned, we’re merely mandating obedience not learning.

 

The same goes for comparability. We could compare all students academic success by their ability to come up with extemporaneous rhymes. But as impressive as it is, skill at spitting out sick rhymes and matching them to dope beats isn’t an objective measure of math or reading.

 

Yet in a different culture, in a different time or place, we might pretend that it was. Imagine how test scores would change and which racial and socioeconomic groups would be privileged and which would suffer. It might – in effect – upend the current trend that prizes richer, whiter students and undervalues the poor and minorities.

 
So let’s begin with objectivity.

 

ARE STANDARDIZED TESTS OBJECTIVE?

 
There is nothing objective about standardized test scores.

 

Objective means something not influenced by personal feelings or opinions. It is a fact – a provable proposition about the world.

 

An objective test would be drawing someone’s blood and looking for levels of nutrients like iron and B vitamins.

 

These nutrients are either there or not.

 

A standardized test is not like that at all. It tries to take a series of skills in a given subject like reading and reduce them to multiple choice questions.

 

Think about how artificial standardized tests are: they’re timed, you can’t talk to others, the questions you’re allowed to ask are limited as is the use of references or learning devices, you can’t even get out of your seat and move around the room.  This is nothing like the real world – unless perhaps you’re in prison.

 

Moreover, this is also true of the questions, themselves.

 

If you’re asking something simple like the addition or subtraction of two numbers or for readers to pick out the color of a character’s shirt in a passage, you’re probably okay.

 

However, the more advanced and complex the skill being assessed, the more it has to be dumbed down so that it will be able to be answered with A, B, C or D.

 

The answer does not avoid human influences or feelings. Instead it assesses how well the test taker’s influences and feelings line up with those of the test maker.

 

If I ask you why Hamlet was so upset by the death of his father, there is no one right answer. It could be because his father was murdered, because his uncle usurped his father’s position, because he was experiencing an Oedipus complex, etc. But the test maker will pick one answer and expect test takers to pick the same one.

 

If they aren’t thinking like the test maker, they are wrong. If they are, they are right.

 

MISUNDERSTANDINGS

 
Yet we pretend this is scientific – in fact, that it’s the ONLY scientific way to measure student learning.

 

And the reason we make this leap is a misunderstanding.

 

We misconstrue our first reason for testing with our third. What we take for objectivity is actually just consistency again.

 

Since we give the same tests to every student in a given state, they show the same things about all students.

 

Unfortunately, that isn’t learning. It’s likemindedness. It’s the ability to conform to one particular way of thinking about things.

 

This is one of the main reason the poor and minorities often don’t score as highly on these assessments as middle class and wealthy white students. These groups have different frames of reference.

 

The test makers generally come from the same socioeconomic group as the highest test takers do. So it’s no wonder that children from that group tend to think in similar ways to adults in that group.

 

This isn’t because of any deficiency in the poor or minorities. It’s a difference in what they’re exposed to, how they’re enculturated, what examples they’re given, etc.

 

And it is entirely unfair to judge these children based on these factors.

 

UNDERESTIMATING HUMAN PSYCHOLOGY

 

The theory of standardized testing is based on a series of faulty premises about human psychology that have been repeatedly discredited.

 

First, they were developed by eugenicists like Lewis Terman who explicitly was trying to justify a racial hierarchy. I’ve written in detail about how in the 1920s and 30s these pseudoscientists tried to rationalize the idea that white Europeans were genetically superior to other races based on test scores.

 

Second, even if we put blatant racism to one side, the theory is built on a flawed and outmoded conception of the human mind – Behaviorism. One of the pioneers of the practice was Edward Thorndike, who used experiments on rats going through mazes as the foundation of standardized testing.

 

This is all good for Mickey and Minnie Mouse, but human beings are much more complicated than that.

 

The idea goes like this – all learning is a combination of stimulus and response. Teaching and learning follow an input-output model where the student acquires information through practice and repetition.

 

This was innovative stuff when B. F. Skinner was writing in the 20th Century. But we live in the 21st.

 

We now know that there are various complex factors that come into play during learning – bio-psychological, developmental and neural processes. When these are aligned to undergo pattern recognition and information processing, people learn. When they aren’t, people don’t learn.

 

However, these factors are much too complicated to be captured in a standardized assessment.

 

As Noam Chomsky wrote in his classic article  “A Review of B. F. Skinner’s Verbal Behavior,” this theory fails to recognize much needed variables in development, intellectual adeptness, motivation, and skill application. It is impossible to make human behavior entirely predictable due to its inherent cognitive complexity.

 

IMPLICATIONS

 

So we’re left with the continued use of widespread standardized testing attached to high stakes for students, schools and teachers.

 

And none of it has a sound rational basis.

 

It is far from objective. It is merely consistent. Therefore it is useless for accountability purposes as well.

 

Since children from different socioeconomic groups have such varying experiences, it is unfair.

 

Demanding everyone to meet the same measure is unjust if everyone isn’t given the same resources and advantages from the start. And that’s before we even recognize that what it consistently shows isn’t learning.

 

The assumption that other measures of academic success are inferior has obscured these truths. While quantifications like classroom grades are not objective either, they are better assessments than standardized tests and produce more valid results.

 

Given the complexity of the human mind, it takes something just as complex to understand it. Far from disparaging educators’ judgement of student performance, we should be encouraging it.

 

It is the student-teacher relationship which is the most scientific. Educators are embedded with their subjects, observe attempts at learning and can then use empirical data to increase academic success on a student-by-student basis as they go. The fact that these methods will not be identical for all students is not a deficiency. It is the ONLY way to meet the needs of diverse and complex humanity – not standardization.

 

Thus we see that the continued use of standardized testing is more a religion – an article of faith – than it is a science.

 

Yet this fact is repeatedly ignored by the media and public policymakers because there has grown up an entire industry around it that makes large profits from the inequality it recreates.

 

In the USA, it is the profit principle that rules all. We adjust our “science” to fit into our economic fictions just as test makers require students to adjust their answers to the way corporate cronies think.

 

In a land that truly was brave and free, we’d allow our children freedom of thought and not punish them for cogitating outside the bubbles.

 


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-3

The Trouble with Test-Obsessed Principals

thumbnail_IMG_8535

 

 

When I was a child, I couldn’t spell the word “principal.”

 

I kept getting confused with its homonym “principle.”

 

I remember Mr. Vay, the friendly head of our middle school, set me straight. He said, “You want to end the word with P-A-L because I’m not just your principal, I’m your pal!”

 

And somehow that corny little mnemonic device did the trick.

 

Today’s principals have come a long way since Mr. Vay.

 

Many of them have little interest in becoming anyone’s pal. They’re too obsessed with standardized test scores.

 

I’m serious.

 

They’re not concerned with student culture, creativity, citizenship, empathy, health, justice – they only care about ways to maximize that little number the state wants to transform our children into.

 

And there’s a reason for that. It’s how the school system is designed to operate.

 

A new research brief from the Tennessee Education Research Alliance concluded that the lowest rated principals generally work at schools with the most economically disadvantaged students.

 

So schools serving students with the highest poverty and lowest test scores often have the least experienced and least effective principals.

 

thumbnail_Screen Shot 2019-02-08 at 11.27.32 AM

 

Now the first question I had when reading this report was “How do they measure effectiveness?” After all, if they rate principals primarily on student test scores, then obviously those working at the poorest schools will be least effective. Poor kids earn low test scores. That’s all the scores consistently show – the relative wealth of students’ parents. If you define an ineffective principal as one who works in a building with low scoring students, it would be no shock that those principals worked in the poorest schools.

 

However, researchers didn’t fall entirely into this trap. According to the working paper:

 

“We measure principal quality in two ways: years of experience in the principal position and rubric-based ratings of effective principal practice taken from the state’s evaluation system.”

 

 

In Tennessee this means evaluating principals partially on student test scores at their buildings – 35%, in fact – higher than the 20% of classroom teachers’ evaluations. However, the remaining pieces of principals’ effectiveness are determined by an observation from a more senior administrator (50%) and an agreed upon score by the principal and district (15%).

 

 

Since researchers are relying at least in part on the state’s evaluation system, they’re including student test scores in their own metric of whether principals are effective or not. However, since they add experience, they’ve actually created a more authentic and equitable measure than the one used by the state.

 

It just goes to show how standardized testing affects nearly every aspect of the public education system.

 

The testing industrial complex is like a black hole. Not only does it suck up funding that is desperately needed elsewhere without providing anything of real value in return, its enormous gravity subverts and distorts everything around it.

 

It’s no wonder then that so many principals at high poverty schools are motivated primarily by test scores, test prep, and test readiness. After all, it makes up a third of their own evaluations.

 

They’ve been dropped into difficult situations and made to feel that they were responsible for numerous factors beyond their control. They didn’t create the problem. They didn’t disadvantage these students, but they feel the need to prove to their bosses that they’re making positive change.

 

But how do you easily prove you’ve bettered the lives of students?

 

Once again, standardized test scores – a faux objective measurement of success.

 

Too many principals buy into the idea that if they can just make a difference on this one metric, it will demonstrate that they’re effective and thus deserve to be promoted out of the high poverty schools and into the well-resourced havens.

 

Yet it’s a game that few principals are able to win. Even those who do distinguish themselves in this way end up doing little more for their students than setting up a façade to hide the underlying problems of poverty and disinvestment.

 

Most principals at these schools wind up endlessly chasing their tails while ignoring opportunities for real positive change. Thus they end up renewing the self-fulfilling prophesy of failure.

 

Researchers noticed the pattern of low performing principals at high poverty schools after examining a decade’s worth of data and found it to hold true in urban, rural and suburban areas. And even though it is based on Tennessee data, the results hold true pretty consistently nationwide, researchers say.

 

Interestingly enough, the correlation doesn’t hold for teachers.

 

Jason Grissom, an associate professor at Vanderbilt University and the faculty director of the Research Alliance, says that the problem stems from issues related specifically to principals.

 

For instance, districts are hiring lower-rated principals for high poverty schools while saving their more effective leaders for buildings with greater wealth and resources.

 

thumbnail_Screen Shot 2019-02-08 at 11.28.20 AM

 

As a result, turnover rates for principals at these schools are much higher than those for classroom educators. Think about what that means – schools serving disadvantaged students are more likely to have new principal after new principal. These are leaders with little experience who never stick around long enough to learn from their mistakes.

 

And since these principals rarely have had the chance to learn on the job as assistant principals, they’re more likely to be flying by the seat of their pants when installed at the head of a school without first receiving the proper training and mentorship that principals at more privileged buildings routinely have.

 

As such, it’s easy for inexperienced principles to fall into the testing trap. They buy into the easy answers of the industry but haven’t been around long enough to learn that the solution they’re being sold is pure snake oil.

 

This has such a large effect because of how important principals are. Though they rarely teach their own classes, they have a huge impact on students. Out-of-school factors are ultimately more important, but in the school building, itself, only teachers are more vital.

 

This is because principals set the tone. They either create the environment where learning can flourish or smother it before the spark of curiosity can ignite. Not only that, but they create the work environment that draws and keeps the best teachers or sends them running for the hills.

 

 

The solution isn’t complicated, says Grissom. Districts need to work to place and keep effective and experienced principals in the most disadvantaged schools. This includes higher salaries and cash bonuses to entice the best leaders to those buildings. It also involves providing equitable resources for disadvantaged schools so that principals have the tools needed to make authentic positive change.

 

I would add that we also need to design fair evaluation systems for both principals and teachers that aren’t based on student test scores. We need to stop contracting out our assessments to corporations and trust our systems of government and schools to make equitable judgments about the people in their employ.

 

Ultimately, what’s required is a change in attitude.

 

Too many principals look at high poverty schools as a stepping stone to working at a school with endless resources and a different class of social issues. Instead, the goal of every excellent school leader should be to end their career working where they are needed most.

 

Such professionalism and experience would loosen the stranglehold of test-and-punish and allow our schools not to simply recreate the inequalities already present in our society. It would enable them to heal the divide.

 

As John Dewey wrote in 1916:

 

“Were all instructors to realize that the quality of mental process, not the production of correct answers, is the measure of educative growth, something hardly less than a revolution in teaching would be worked.”

 

And that’s what’s needed – a revolution.


 

Still can’t get enough Gadfly? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-4

No One Ever Remembered a Teacher for Raising Standardized Test Scores

gradhug

 

It’s the day before school begins.

 

I’m out to eat with my family and have just taken a big bite of a juicy beef taco.

 

That’s when I notice someone standing right next to me at the restaurant.

 

So I raise my eyes upward, a meat filled tortilla overfilled with lettuce and beans hanging from my mouth, and I’m greeted with a familiar face.

 

“Mr. Singer!” the woman says with a nervous smile on her lips.

 

“Do you remember me?”

 

I think for a moment but realize I have more pressing concerns. I couldn’t reply with an answer to the woman’s question even if I did remember her.

 

So I chew and swallow and then look again.

 

“It’s me,” she says. “Tamarind.”

 

And then it hits me like a flash.

 

The face in front of me ages backward. The adult eyes soften. The taut cheeks become chubbier. And her whole figure shrinks three feet closer to the ground.

 

“Oh my God! Tamarind! Of course I remember you!” I say.

 

She smiles and blushes. I’m surprised by how nervous she is. I’m no one to inspire anxiety. I’m just a guy out with his wife, daughter and father-in-law shoving a taco in his face.

 

“When I saw you here I just had to come up to you,” she said. “I was in your 6th grade class.”

 

“I think it was 8th grade, wasn’t it?” I said.

 

“Yes! That’s right! Eighth grade!”

 

“How old are you now? My gosh I remember you when you only came up to here off the ground.”

 

“I’m 22. I’m doing really well. I just wanted you to know that you taught me how to write. If it wasn’t for you I never would have made it anywhere. I just wanted to thank you so much for everything you did for me.”

 

We chatted a bit more and then she left us to finish our meal.

 

But, of course, the whole interaction got me thinking.

 

As a teacher, you are something of a minor public figure.

 

When you’re out and about – especially if you’re somewhere in your district – you’re bound to be recognized and invariably someone will want to chat.

 

I remember one time at the bakery counter a former student gave me my order and told me he threw in a few donuts.

 

I remember laughing and telling him he didn’t need to do that.

 

“Nah, Mr. Singer, you never wrote me up for falling asleep in your class. You knew I was watching my brothers and sisters at home and never gave me shit for it. You keep those donuts.”

 

Another time at the theater I was almost late to my movie because I was listening to a former student at the concession stand catch me up on her life and what all of her friends from my class were doing these days.

 

So many students. So many kids that have now become adults.

 

You lose track of how many lives you’ve had an impact on.

 

The first few days of school are always filled with endless administrative meetings. The superintendent welcomes you with testing data. Then your principal breaks it down by building and subject.

 

You find out which diagnostic exams you have to give your students and when. You find out what your Pennsylvania Value Added Assessment Score (PVAAS) is – how good a teacher you are based on how well your students from last year did on the state standardized test.

 

On the one hand, I suppose I have no reason to feel like much of a good teacher.

 

Most of my students didn’t pass the test. They rarely do.

 

The same number of 7th graders (that’s what I taught last year) passed the reading test as in previous years. However, many more passed that were expected to fail.

 

The state uses a mystery metric based on Classroom Diagnostic Assessment (CDT) data to come up with a prediction of who they expect to pass and who they expect to fail. No one really knows how they calculate this. For all we know, the state secretary of education could examine a pile of chicken entrails before entering it all into the system.

 

Does all that data mean I’m a good teacher or not?

 

I don’t know.

 

But I do know what Tamarind thinks.

 

And I know what a host of former students have told me. I know how they react when they see me out in the wild, just living my life.

 

I’m sure there are probably former students who don’t like me. There must be those who hold a grudge for getting a 59% on an assignment. Or maybe they remember me yelling at them for something. Or – who knows – maybe they just didn’t respond to The Singer Charm.

 

But an awful lot of people come up to me who don’t have to.

 

Yesterday was the first day of classes for the year.

 

For the first time, all my classes were looped. I taught 7th grade Language Arts last year and I’m teaching the 8th grade course this year.

 

When those kids came into the class on Friday, it was like a homecoming.

 

So many smiles. So much laughter and joy. And, yes, impromptu hugs.

 

It felt like a family gathering, not a school function.

 

As I left the building feeling more exhausted than I have in months, another teacher stopped me.

 

“Steve! I wanted to catch you before you left!” she said.

 

She told me that she gave her students a survey in her class as an icebreaker. One of the questions was to name their favorite teacher from last year. My name came up a lot.

 

What can you say about that?

 

I’m actually getting choked up just typing this.

 

In my years in the classroom, I’ve helped a lot of kids get better test scores.

 

But that’s not why they come up to me. That’s not why they remember me.

 

I touched their lives in some meaningful way.

 

And they have done the same for me.

 

I’m just a guy who should really take smaller bites of his tacos.

 

But they make me feel like a hero.

 

I am so grateful.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-2

Top 10 Reasons You Can’t Fairly Evaluate Teachers on Student Test Scores

Screen Shot 2018-08-02 at 12.49.24 AM

 

I’m a public school teacher.

 

Am I any good at my job?

 

There are many ways to find out. You could look at how hard I work, how many hours I put in. You could look at the kinds of things I do in my classroom and examine if I’m adhering to best practices. You could look at how well I know my students and their families, how well I’m attempting to meet their needs.

 

Or you could just look at my students’ test scores and give me a passing or failing grade based on whether they pass or fail their assessments.

 

It’s called Value-Added Measures (VAM) and at one time it was the coming fad in education. However, after numerous studies and lawsuits, the shine is fading from this particularly narrow-minded corporate policy.

 

Most states that evaluate their teachers using VAM do so because under President Barack Obama they were offered Race to the Top grants and/or waivers.

 

Now that the government isn’t offering cash incentives, seven states have stopped using VAM and many more have reduced the weight given to these assessments. The new federal K-12 education law – the Every Student Succeeds Act (ESSA) – does not require states to have educator evaluation systems at all. And if a state chooses to enact one, it does not have to use VAM.

 

That’s a good thing because the evidence is mounting against this controversial policy. An evaluation released in June of 2018 found that a $575 million push by the Bill and Melinda Gates Foundation to make teachers (and thereby students) better through the use of VAM was a complete waste of money.

 

Meanwhile a teacher fired from the Washington, DC, district because of low VAM scores just won a 9-year legal battle with the district and could be owed hundreds of thousands of dollars in back pay as well as getting his job back.

 

But putting aside the waste of public tax dollars and the threat of litigation, is VAM a good way to evaluate teachers?

 

Is it fair to judge educators on their students’ test scores?

 

Here are the top 10 reasons why the answer is unequivocally negative:

 

 

1) VAM was Invented to Assess Cows.

I’m not kidding. The process was created by William L. Sanders, a statistician in the college of business at the University of Knoxville, Tennessee. He thought the same kinds of statistics used to model genetic and reproductive trends among cattle could be used to measure growth among teachers and hold them accountable. You’ve heard of the Tennessee Value-Added Assessment System (TVAAS) or TxVAAS in Texas or PVAAS in Pennsylvania or more generically named EVAAS in states like Ohio, North Carolina, and South Carolina. That’s his work. The problem is that educating children is much more complex than feeding and growing cows. Not only is it insulting to assume otherwise, it’s incredibly naïve.

 

2) You can’t assess teachers on tests that were made to assess students.

This violates fundamental principles of both statistics and assessment. If you make a test to assess A, you can’t use it to assess B. That’s why many researchers have labeled the process “junk science” – most notably the American Statistical Association in 2014. Put simply, the standardized tests on which VAM estimates are based have always been, and continue to be, developed to assess student achievement and not growth in student achievement nor growth in teacher effectiveness. The tests on which VAM estimates are based were never designed to estimate teachers’ effects. Doing otherwise is like assuming all healthy people go to the best doctors and all sick people go to the bad ones. If I fail a dental screening because I have cavities, that doesn’t mean my dentist is bad at his job. It means I need to brush more and lay off the sugary snacks.

 

3) There’s No Consistency in the Scores.

Valid assessments produce consistent results. This is why doctors often run the same medical test more than once. If the first try comes up positive for cancer, let’s say, they’re hoping the second time will come up negative. However, if multiple runs of the same test produce the same result, that diagnosis gains credence. Unfortunately, VAM scores are notoriously inconsistent. When you evaluate teachers with the same test (but different students) over multiple years, you often get divergent results. And not just by a little. Teachers who do well one year may do terribly the next. This makes VAM estimates extremely unreliable. Teachers who should be (more or less) consistently effective are being classified in sometimes highly inconsistent ways over time. A teacher classified as “adding value” has a 25 to 50% chance of being classified as “subtracting value” the next year, and vice versa. This can make the probability of a teacher being identified as effective no different than the flip of a coin.

 

4) Changing the test can change the VAM score.

If you know how to add, it doesn’t matter if you’re asked to solve 2 +2 or 3+ 3. Changing the test shouldn’t have a major impact on the result. If both tests are evaluating the same learning and at the same level of difficulty, changing the test shouldn’t change the result. But when you change the tests used in VAM assessments, scores and rankings can change substantially. Using a different model or a different test often produces a different VAM score. This may indicate a problem with value added measures or with the standardized tests used in conjunction with it. Either way, it makes VAM scores invalid.

 

5) VAM measures correlation, not causation.

Sometimes A causes B. Sometimes A and B simply occur at the same time. For example, most people in wheelchairs have been in an accident. That doesn’t mean being in a wheelchair causes accidents. The same goes for education. Students who fail a test didn’t learn the material. But that doesn’t mean their teacher didn’t try to teach them. VAM does not measure teacher effectiveness. At best it measures student learning. Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model. For instance, the student may have a learning disability, the student may have been chronically absent or the test, itself, may be an invalid measure of the learning that has taken place.

 

6) Vam Scores are Based on Flawed Standardized Tests.

When you base teacher evaluations on student tests, at very least the student tests have to be valid. Otherwise, you’ll have unfairly assessed BOTH students AND teachers. Unfortunately standardized tests are narrow, limited indicators of student learning. They leave out a wide range of important knowledge and skills leaving only the easiest-to-measure parts of math and English curriculum. Test scores are not universal, abstract measures of student learning. They greatly depend on a student’s class, race, disability status and knowledge of English. Researchers have been decrying this for decades – standardized tests often measure the life circumstances of the students not how well those students learn – and therefore by extension they cannot assess how well teachers teach.

 

7) VAM Ignores Too Many Factors.

When a student learns or fails to learn something, there is so much more going on than just a duality between student and teacher. Teachers cannot simply touch students’ heads and magically make learning take place. It is a complex process involving multiple factors some of which are poorly understood by human psychology and neuroscience. There are inordinate amounts of inaccurate or missing data that cannot be easily replaced or disregardedvariables that cannot be statistically controlled for such as: differential summer learning gains and losses, prior teachers’ residual effects, the impact of school policies such as grouping and tracking students, the impact of race and class segregation, etc. When so many variables cannot be accounted for, any measure returned by VAMs remains essentially incomplete.

 

8) VAM Has Never been Proven to Increase Student Learning or Produce Better Teachers.

That’s the whole purpose behind using VAM. It’s supposed to do these two things but there is zero research to suggest it can do them. You’d think we wouldn’t waste billions of dollars and generations of students on a policy that has never been proven effective. But there you have it. This is a faith-based initiative. It is the pet project of philanthrocapitalists, tech gurus and politicians. There is no research yet which suggests that VAM has ever improved teachers’ instruction or student learning and achievement. This means VAM estimates are typically of no informative, formative, or instructional value.

 

9) VAM Often Makes Things Worse.

Using these measures has many unintended consequences that adversely affect the learning environment. When you use VAMs for teacher evaluations, you often end up changing the way the tests are viewed and ultimately the school culture, itself. This is actually one of the intents of using VAMs. However, the changes are rarely positive. For example, this often leads to a greater emphasis on test preparation and specific tested content to the exclusion of content that may lead to better long-term learning gains or increasing student motivation. VAM incentivizes teachers to wish for the most advanced students in their classes and to push the struggling students onto someone else so as to maximize their own personal VAM score. Instead of a collaborative environment where everyone works together to help all students learn, VAM fosters a competitive environment where innovation is horded and not shared with the rest of the staff. It increases turnover and job dissatisfaction. Principals stack classes to make sure certain teachers are more likely to get better evaluations or vice versa. Finally, being unfairly evaluated disincentives new teachers to stay in the profession and it discourages the best and the brightest from ever entering the field in the first place. You’ve heard about that “teacher shortage” everyone’s talking about. VAM is a big part of it.

 

10) An emphasis on VAM overshadows real reforms that actually would help students learn.

Research shows the best way to improve education is system wide reforms – not targeting individual teachers. We need to equitably fund our schools. We can no longer segregate children by class and race and give the majority of the money to the rich white kids while withholding it from the poor brown ones. Students need help dealing with the effects of generational poverty – food security, psychological counseling, academic tutoring, safety initiatives, wide curriculum and anti-poverty programs. A narrow focus on teacher effectiveness dwarfs all these other factors and hides them under the rug. Researchers calculate teacher influence on student test scores at about 14%. Out-of-school factors are the most important. That doesn’t mean teachers are unimportant – they are the most important single factor inside the school building. But we need to realize that outside the school has a greater impact. We must learn to see the whole child and all her relationships –not just the student-teacher dynamic. Until we do so, we will continue to do these children a disservice with corporate privatization scams like VAM which demoralize and destroy the people who dedicate their lives to helping them learn – their teachers.

 


NOTE: Special thanks to the amazingly detailed research of Audrey Amrein-Beardsley whose Vamboozled Website is THE on-line resource for scholarship about VAM.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-1

When Students Stay Up All Night Playing Fortnite and You’ve got to Teach Them in the Morning

ne13feb40-ALL-addicted-to-video-games-L13FCA50-51A

 

There is something monstrously unfair about our teacher evaluation systems.

 

If your students fail because they were up all night playing video games, it’s your fault.

 

Seriously.

 

When students fail at academic tasks, there is no responsibility attributed to the students, no responsibility attributed to the parents and certainly no responsibility given to society.

 

It’s all just thrown on the teacher because, hey, someone’s got to be responsible. And it might as well be them.

 

I’ve written scores of articles about how standardized tests forced on students by the federal government are unfair.

 

They are developmentally inappropriate, culturally biased, and subject to a deep conflict of interest because the people making the tests get more money if test takers fail.

 

The tests drive the curriculum instead of the other way around. The scores needed to pass change from year-to-year invalidating annual comparisons. And many lawmakers pushing for these assessments are funded by the school privatization industry that uses failing test scores to sell its own fly-by-night brand of education.

 

These are real problems our education system faces every day.

 

But we mustn’t forget an even more fundamental one: we’re all responsible for student success or failure.

 

Not just teachers. EVERYONE.

 

Society, lawmakers, business people, parents – but those most responsible are the students, themselves.

 

Case and Point—

 

Over the last few months a word has entered my students’ vocabulary that hadn’t been there before: Fortnite.

 

It’s not that they’re so interested in an antiquated term for a two-week period. It’s the name of a popular multiplayer on-line shoot-em-up video game for PlayStation 4, Xbox One, Windows, and Mac. Players build forts with teammates to defend against other players or enemies.

 

Apparently, many of my students got it for Christmas. Or since there’s a free on-line version, they were turned on to it by others who had gotten the deluxe version as a present.

 

It started as an undercurrent of trash talk. “You suck at Fortnite.” “You can’t beat me on Fortnite.” “You just wish you could take me on Fortnite.”

 

And then it started to manifest physically.

 

Those same kids would come in to school with Fortnite Face – glassy red eyes, heads slumped on the table and the inability to stay awake for more than 10 minutes at a time.

 

It’s not all of my students, but it’s a significant percentage. Almost all boys. And almost all at a distinct learning disadvantage.

 

Teaching them is like teaching someone in a deep sea diver suit. They can’t really see or hear you very well. And any message you get back from them sounds like it’s coming from the bottom of the ocean.

 

When I noticed it, I cleared as much of my schedule as I could to call parents. It’s hard because administration decided not to fill positions in my department for teachers who retired last year – so all our classes are larger. And they gave me a new class I haven’t taught in years so the planning load is more cumbersome.

 

Plus I have as many special education students as legally allowed in every class, which requires mountains of extra paperwork and monitoring for each child.

 

And of course the phone in my room doesn’t call out and the cell reception is terrible, so I have to move to one of the few phones that will actually allow me to contact parents and try to communicate my concerns.

 

Most parents I talked to noticed the same things I had. Fortnite was taking over their children’s lives. Their kids were playing the game at every opportunity and ignoring most everything else.

 

However, most parents I couldn’t reach. Those cricket burner phones get disconnected quick. Others go straight to a voicemail box that’s so full it won’t accept new messages. Others allow me to leave a message that will never be returned.

 

But sometimes I did get through. And sometimes parents didn’t simply throw up their hands and say they don’t know what to do. Sometimes a parent actually laid down ground rules or took the game away.

 

However, if I’m being honest, contacting parents did not solve my problem.

 

I’m not blaming them. Most of my students live below the poverty line. That means their folks are working two or three jobs just to make ends meet. Or they’re grandparents raising their sons’ or daughters’ kids. Or they’re foster parents with a full house.

 

They’re doing the best they can. But it doesn’t end up stopping the addiction.

 

And – let’s be honest – it is an addiction.

 

For the first time in 2018, the World Health Organization recognized video game addiction as a real thing. Not every video game. Not every time someone sits down to play a video game. But video games can lead to addictive behavior.

 

That’s what I’m seeing in my students.

 

So after talking with as many parents as I could, I came to a mostly dead end.

 

My next step was to try to use student interests to influence instruction.

 

We were in the middle of a poetry writing unit. So I allowed students to write their poems about Fortnite.

 

That perked up a few heads.

 

Here’s a cinquain about Fortnite. Here’s an acrostic, a narrative, a concrete poem in the shape of a soldier or his gun.

 

To be honest, none of them were masterpieces.

 

They were just the normal trash talk and braggadocio written down in verse.

 

So I got an idea. Use the heightened competitive urge to push artistry.

 

We came to limericks – a difficult but fun type of poetry with five lines, a specific rhyme scheme and meter.

 

We read funny examples, we sang the rhythm together in chorus – da Dum da da Dum da da Dum – and then I set them the task of writing their own limericks.

 

With one twist. Whoever wrote the best limerick would get a homework pass.

 

That got them going like a shot.

 

All of my Fortniters perked up.

 

They wrote like I’d never seen.

 

Each wanted to one-up the others. And no one wrote about the game.

 

By the end of class, we had some pretty good poems. I wouldn’t say they are the best ever written, but they were miles better than where we were before.

 

So what does it all mean?

 

When we talk about video games these days, the conversation usually strays toward violence.

 

Pundits caution that video games will desensitize children and make them more prone to aggression and acting out. It might even contribute to the creation of school shooters.

 

Wrong.

 

In general, video games don’t make children more violent. Fortnite is a game where students shoot each other with guns all night long and it hasn’t made my students any more aggressive or violent than they already were.

 

Many cultures like the Japanese are much more into video games than ours and they have fewer violent incidents or school shootings.

 

However, video game addiction is a real thing and it impacts learning.

 

Some corporations want to try to harness this addiction to push learning. Hence the move to personalized or competency based education. That’s pure rubbish.

 

It’s a way to monetize education without paying attention to what’s best for kids. The same with gamification – using game theory to drive instruction.

 

And don’t think I’ve lost sight of my own use of competition in class. I haven’t.

 

Games and competition can be used to positive ends in moderation.

 

You can motivate reluctant kids to do things they wouldn’t normally do with competition. But it doesn’t work for everyone and it doesn’t work all the time.

 

It needs to be a novelty. Any tool can be overused.

 

Even video games aren’t bad in moderation. I used to be a gamer, myself.

 

The problem is when it becomes an addiction.

 

Our social structures can’t handle it.

 

Game corporations only care if it makes money. Parents are often stressed to the limit just to provide the basics.

 

The only group we require to be responsible is teachers.

 

And that’s just not going to work.

 

Video game addiction is another area where it becomes painfully clear how much work we all need to do to help our children succeed.

Teacher Seniority – the Seat Belts of the Education Profession

Lady-Seatbelt(1)

You wouldn’t travel a long distance in your car without strapping on a seatbelt.

So why do you think teachers should spend 30 plus years in the classroom without seniority?

Everywhere you look, billionaires are paying millionaires in government to pass laws to cut taxes, slash funding and find cheaper ways to run public schools for pleb kids like yours and mine. And that often means finding ways to weaken protections for teachers, fire those with the most experience and replace them with glorified WalMart greeters.

“Hello. Welcome to SchoolMart. Please plug into your iPad and begin today’s lesson.”

This is class warfare cloaked as a coupon. It’s sabotage described as savings.

And the only way they get away with it is because reasonable people buy the steaming load of manure they’re selling.

MYTH: Seniority with Tenure means a Job for Life

Tell that to the hundreds of thousands of teachers out of work.

Tell that to all the optimistic go getters who prance out of college ready to change the world as teachers and fizzle out during the first five years.

Tell it to the handful of truly terrible teachers who for reasons only they can explain stay in a job they hate through countless interventions and retrainings until the principal has no choice but to give them their walking papers.

Oh, yes. Teachers DO get fired. I’ve seen it with my own eyes numerous times. And in each case, they truly deserved it.

(Any “bad teachers” still on the job mean there’s a worse administrator somewhere neglecting to do his or her duty.)

So what does “Seniority” and “Tenure” even mean for teachers?

Basically, it means two things:

(1) If you want to fire a teacher, you have to prove he or she deserves it. That’s Tenure.

(2) When public school districts downsize, they can’t just lay off people based on their salaries. That’s Seniority.

If you think about it, both of these are good things.

It is not a good work environment for teachers or students when educators can be fired without cause at the whim of incoming administration or radical, newly-elected school board members. Teaching is one of the most political professions we have. Tenure shields educators from the winds of partisanship. It allows them to grade children fairly whose parents have connections on the school board, it allows them to speak honestly and openly about school policy, and it empowers them to act in the best interests of their students – all things that otherwise could jeopardize their jobs.

Likewise, seniority stops the budget butchers from making experience and stability a liability.

It stops number crunchers from saying:

Hey, Mrs. Wilson has been here for 25 years. She’s got a shelf full of teaching awards. Parents and students love her. But she’s at the top of the salary scale so she’s gotta’ go.

I know what you’re going to say: Aren’t there younger teachers who are also outstanding?

Yes. There are.

However, if you put all the best teachers in one group, most of them will be more experienced.

It just makes sense. You get better at something – anything – the more you do it. This could be baking pies, building houses or teaching children how to read and write.

So why don’t we keep the best teachers and get rid of those who aren’t up to their level?

Because determining who’s the best is subjective. And if you let the moneymen decide – POOF! – suddenly the teachers who make the most money will disappear and only the cheapest ones will be left.

Couldn’t you base it on something more universal like student test scores?

Yes, you could, but student test scores are a terrible way to evaluate teachers. If you wanted to get rid of the highest paid employees, all you’d have to do is give them the most struggling students. Suddenly, their students have the worst test scores, and they’re packing up their stuff in little cardboard boxes.

Almost any stat can be gamed.

The only one that is solidly unbiased? Seniority.

You’ve either been here 15 years or you haven’t. There’s not much anyone can do to change that fact.

That’s why it prevents the kind of creative accounting you see from penny pinching number crunchers.

Along with Tenure, Seniority is a safety net. Pure and simple. It helps keep the most qualified teachers in the room with kids. Period.

But look. It’s not perfect.

Neither are seat belts.

If you’re in a car crash on a bridge where it’s necessary to get out of your vehicle quickly before it plunges into the water below, it’s possible your seat belt may make it more difficult to reach safety. This is rather rare, and it doesn’t stop most people from buckling up.

I’ve known excellent teachers who were furloughed while less creative ones were kept on. It does happen.

But if we got rid of seniority, it would happen way more often.

That’s the bottom line.

Instead of finding more leeway to fire more teachers, we should be finding ways to increase school funding – especially at the most under-resourced schools – which, by the way, are the ones where lawmakers most want to eliminate seniority. We should be looking for ways to make downsizing unnecessary. We should be investing in our children and our future.

We’ll never improve the quality of the public school system by firing our way to the bottom. That’s like trying to lose weight by hacking at yourself with a straight razor. It just won’t work.

We need to commit to public schools. We need to commit to public school students. And the best way to do that is to support the teachers who devote their lives showing up every day to help them learn.

Who is Responsible for Student Achievement?

achievementseries

  

Billy is an average middle school student.

 

He sits down and takes a test.

 

The grade comes back.

 

Who is responsible for that grade?

 

This should be the dumbest question you can ask in the field of education.

 

The answer should be obvious.

 

Billy is responsible.

 

Billy did the work, he took the test, he earned the grade.

 

But all across this great country of ours we’re giving the wrong answer.

 

We’re saying teachers are responsible for that grade.

 

This is ridiculous. Teachers could not do the work for the student. Teachers could not take the test for the student. How can you possibly assume the teacher is responsible for the grade?

 

In fact, if the teacher really were responsible – if she did all the work and took the test – how could you rationally say this grade belonged to the student? No, it wouldn’t be Billy’s, it would be his teacher’s.

 

The truth goes something like this: You are responsible only for things within your control. The greater your degree of control, the greater your degree of responsibility.

 

This is not complicated.

 

It is simple logic. Cause and effect.

 

But ignoring it is perhaps the most virulent, incorrigible, fact-resistant mistake in the entire field of public education.

 

Lawmakers are getting it wrong. The media is getting it wrong. Superintendents, principals – even teachers are getting it wrong.

 

And the reason is somewhat pernicious.

 

We’ve been sold a lie.

 

We’ve been told for so long that educators are responsible for their students’ work that we’ve begun to accept it without question.

 

Just today at a training in my district, I was shown a spreadsheet of student test scores and told in no uncertain terms that this was something I have control over.

 

I DON’T.

 

I don’t have control over the raw scores. I don’t even have control over how much a student improves from one year to another.

 

The student does.

 

HE controls how hard he works on assignments. HE exhibits the most control over the results of his assessments.

 

This doesn’t mean I’m completely helpless.

 

I do have control over certain aspects of students’ academic experience.

 

I control what work is assigned, when it is assigned and to whom.

 

I control whether there is extra credit, what counts as homework, who gets extra help, etc.

 

In many cases, I even get to decide whether students have completed their work and if assessments have been completed successfully.

 

As long as I am exhibiting best practices, giving age-appropriate work and evaluating it fairly, I’m doing my part.

 

It is not then justified to assume I am solely responsible for the end result.

 

I raise the hurdles, but the student actually goes through the obstacle course.

 

The teacher is a factor, but not the largest one. That is the student, Billy.

 

Yet he is not alone here. Besides, me, his teacher, there is also the principal, the student’s parents, his friends and even society as a whole.

 

All of these and more contribute to student success.

 

The principal controls school policy. He determines what discipline the student receives, the clarity of school rules, etc.

 

Likewise, students’ friends are part of their social network. They can help with homework, form a study group, or distract from school work, denigrate work ethic, etc.

 

Society also plays a role. If a student is part of a community that values education and work ethic, that student will more likely put forth more effort. If the student lives in a community where school is seen as unimportant and teachers are not respected, that will have a negative impact, etc.

 

And the number one factor other than the student, himself, that contributes to his success is parents. They control home life, emotional support, tutoring, nutrition, etc.

 

All of these complex factors combine to add up to an individual student’s success. However, at the end of the day, it is the student, himself, that bears the brunt of the responsibility for what he does.

 

That’s why we call it HIS grade and not someone else’s.

 

This is the most obvious thing in the world, but it has certain consequences for education policy.

 

For instance, it immediately invalidates the majority of teacher evaluations given throughout the country. The reason? Most evaluations are based at least in part on student test scores.

 

As we’ve seen, this misrepresents the student-teacher relationship. It blames the teacher for things well beyond his or her control.

 

It turns students into passive objects acted on by magical super teachers who can somehow make them learn simply by – what – endless repetition of test prep materials?

 

Why would students put forth their best in this scenario? If they’re failing, it’s somehow not their fault. It’s their teachers!

 

But even worse than this misrepresentation, it completely ignores a plethora of vital factors in the education process.

 

Parents, for instance, are crucially important, but we’re leaving them completely out of the loop.

 

When parents struggle to fulfill their responsibilities, why is there little to no help? The answer: because we’ve hidden the fact that such responsibilities even exist. We’ve thrown it all on the teacher and the school.

 

All these out-of-school factors are obscured, yet taken together they are almost determinate. After all, this is why poor and minority students disproportionately struggle academically.

 

You can demand every student jump six feet straight up, but those with the best resources will meet this goal much more frequently than those without.

 

And who is in control of those resources? Who decides which children get the smallest class sizes, the best home environments, the most conducive social networks, etc.?

 

The myth of teacher accountability is what stops such resources from being sent.

 

We’re told all you need is a good teacher.

 

But this is not true.

 

You need much more.

 

The ultimate responsibility may rest with the student, but until we all realize and acknowledge our collective responsibilities to all students, success will always be out of reach for far too many of them.

 

Billy may take the test, but it is society that is failing to meet its responsibilities.