Standardized Tests Are Not Objective Measures of Anything

Screen Shot 2019-06-29 at 12.27.12 AM

 
When it comes to standardized tests, most people are blinded by science.

 

Or at least the appearance of science.

 

Because there is little about these assessments that is scientific, factual or unbiased.

 

And that has real world implications when it comes to education policy.

 

First of all, the federal government requires that all public school children take these assessments in 3-8th grade and once in high school. Second, many states require teachers be evaluated by their students’ test scores.

 

Why?

 

It seems to come down to three main reasons:

 

1) Comparability
2) Accountability
3) Objectivity

 

COMPARABILITY

 

First, there is a strong desire to compare students and student groups, one with the other.

 

We look at learning like athletics. Who has shown the most success, and thereby is better than everyone else?

 

This is true for students in a single class, students in a single grade, an entire building, a district, a state, and between nations, themselves.

 

If we keep questions and grading methods the same for every student, there is an assumption that we can demonstrate which group is best and worst.

 

ACCOUNTABILITY

 

Second, we want to ensure all students are receiving the best education. So if testing can show academic success through its comparability, it can also be used as a tool to hold schools and teachers accountable. We can simply look at the scores and determine where academic deficiencies exist, diagnose them based on which questions students get incorrect and then focus there to fix the problem. And if schools and teachers can’t or won’t do that, it is their fault. Thus, the high stakes in high stakes testing.

 

Obviously there are other more direct ways to determine these facts. Historically, before standardized testing became the centerpiece of education policy, we’d look at resource allocation to determine this. Are we providing each student with what they need to learn? Do they have sound facilities, wide curriculum, tutoring, proper nutrition, etc.? Are teachers abiding by best practices in their lessons? Many would argue this was a better way of ensuring accountability, but if standardized assessments produce valid results, they are at least one possible way to ensure our responsibilities to students are being met.

 

OBJECTIVITY

 

Third, and most importantly, there is the assumption that of all the ways to measure learning, only standardized testing produces objective results. Classroom grades, student writing, even high school graduation rates are considered subjective and thereby inferior.

 

Questions and grading methods are identical for every student, and a score on the test is proof that a student is either good or bad at a certain subject. Moreover, we can use that score to keep the entire education system on track and ensure it is functioning correctly.

 

So this third reason for standardized testing is really the bedrock rationale. If testing is not objective, it doesn’t matter if it’s comparable or useful for accountability.

 

After all, we could hold kids accountable for the length of their hair, but if that isn’t an objective measure of what they’ve learned, we’re merely mandating obedience not learning.

 

The same goes for comparability. We could compare all students academic success by their ability to come up with extemporaneous rhymes. But as impressive as it is, skill at spitting out sick rhymes and matching them to dope beats isn’t an objective measure of math or reading.

 

Yet in a different culture, in a different time or place, we might pretend that it was. Imagine how test scores would change and which racial and socioeconomic groups would be privileged and which would suffer. It might – in effect – upend the current trend that prizes richer, whiter students and undervalues the poor and minorities.

 
So let’s begin with objectivity.

 

ARE STANDARDIZED TESTS OBJECTIVE?

 
There is nothing objective about standardized test scores.

 

Objective means something not influenced by personal feelings or opinions. It is a fact – a provable proposition about the world.

 

An objective test would be drawing someone’s blood and looking for levels of nutrients like iron and B vitamins.

 

These nutrients are either there or not.

 

A standardized test is not like that at all. It tries to take a series of skills in a given subject like reading and reduce them to multiple choice questions.

 

Think about how artificial standardized tests are: they’re timed, you can’t talk to others, the questions you’re allowed to ask are limited as is the use of references or learning devices, you can’t even get out of your seat and move around the room.  This is nothing like the real world – unless perhaps you’re in prison.

 

Moreover, this is also true of the questions, themselves.

 

If you’re asking something simple like the addition or subtraction of two numbers or for readers to pick out the color of a character’s shirt in a passage, you’re probably okay.

 

However, the more advanced and complex the skill being assessed, the more it has to be dumbed down so that it will be able to be answered with A, B, C or D.

 

The answer does not avoid human influences or feelings. Instead it assesses how well the test taker’s influences and feelings line up with those of the test maker.

 

If I ask you why Hamlet was so upset by the death of his father, there is no one right answer. It could be because his father was murdered, because his uncle usurped his father’s position, because he was experiencing an Oedipus complex, etc. But the test maker will pick one answer and expect test takers to pick the same one.

 

If they aren’t thinking like the test maker, they are wrong. If they are, they are right.

 

MISUNDERSTANDINGS

 
Yet we pretend this is scientific – in fact, that it’s the ONLY scientific way to measure student learning.

 

And the reason we make this leap is a misunderstanding.

 

We misconstrue our first reason for testing with our third. What we take for objectivity is actually just consistency again.

 

Since we give the same tests to every student in a given state, they show the same things about all students.

 

Unfortunately, that isn’t learning. It’s likemindedness. It’s the ability to conform to one particular way of thinking about things.

 

This is one of the main reason the poor and minorities often don’t score as highly on these assessments as middle class and wealthy white students. These groups have different frames of reference.

 

The test makers generally come from the same socioeconomic group as the highest test takers do. So it’s no wonder that children from that group tend to think in similar ways to adults in that group.

 

This isn’t because of any deficiency in the poor or minorities. It’s a difference in what they’re exposed to, how they’re enculturated, what examples they’re given, etc.

 

And it is entirely unfair to judge these children based on these factors.

 

UNDERESTIMATING HUMAN PSYCHOLOGY

 

The theory of standardized testing is based on a series of faulty premises about human psychology that have been repeatedly discredited.

 

First, they were developed by eugenicists like Lewis Terman who explicitly was trying to justify a racial hierarchy. I’ve written in detail about how in the 1920s and 30s these pseudoscientists tried to rationalize the idea that white Europeans were genetically superior to other races based on test scores.

 

Second, even if we put blatant racism to one side, the theory is built on a flawed and outmoded conception of the human mind – Behaviorism. One of the pioneers of the practice was Edward Thorndike, who used experiments on rats going through mazes as the foundation of standardized testing.

 

This is all good for Mickey and Minnie Mouse, but human beings are much more complicated than that.

 

The idea goes like this – all learning is a combination of stimulus and response. Teaching and learning follow an input-output model where the student acquires information through practice and repetition.

 

This was innovative stuff when B. F. Skinner was writing in the 20th Century. But we live in the 21st.

 

We now know that there are various complex factors that come into play during learning – bio-psychological, developmental and neural processes. When these are aligned to undergo pattern recognition and information processing, people learn. When they aren’t, people don’t learn.

 

However, these factors are much too complicated to be captured in a standardized assessment.

 

As Noam Chomsky wrote in his classic article  “A Review of B. F. Skinner’s Verbal Behavior,” this theory fails to recognize much needed variables in development, intellectual adeptness, motivation, and skill application. It is impossible to make human behavior entirely predictable due to its inherent cognitive complexity.

 

IMPLICATIONS

 

So we’re left with the continued use of widespread standardized testing attached to high stakes for students, schools and teachers.

 

And none of it has a sound rational basis.

 

It is far from objective. It is merely consistent. Therefore it is useless for accountability purposes as well.

 

Since children from different socioeconomic groups have such varying experiences, it is unfair.

 

Demanding everyone to meet the same measure is unjust if everyone isn’t given the same resources and advantages from the start. And that’s before we even recognize that what it consistently shows isn’t learning.

 

The assumption that other measures of academic success are inferior has obscured these truths. While quantifications like classroom grades are not objective either, they are better assessments than standardized tests and produce more valid results.

 

Given the complexity of the human mind, it takes something just as complex to understand it. Far from disparaging educators’ judgement of student performance, we should be encouraging it.

 

It is the student-teacher relationship which is the most scientific. Educators are embedded with their subjects, observe attempts at learning and can then use empirical data to increase academic success on a student-by-student basis as they go. The fact that these methods will not be identical for all students is not a deficiency. It is the ONLY way to meet the needs of diverse and complex humanity – not standardization.

 

Thus we see that the continued use of standardized testing is more a religion – an article of faith – than it is a science.

 

Yet this fact is repeatedly ignored by the media and public policymakers because there has grown up an entire industry around it that makes large profits from the inequality it recreates.

 

In the USA, it is the profit principle that rules all. We adjust our “science” to fit into our economic fictions just as test makers require students to adjust their answers to the way corporate cronies think.

 

In a land that truly was brave and free, we’d allow our children freedom of thought and not punish them for cogitating outside the bubbles.

 


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-3

Modernizing Education Starts With Questioning Our Assumptions

iStock_000040220958_Small-685x485

 

When it comes to education, we take an awful lot for granted.

 

For example, we look at learning almost entirely from a behavioral standpoint.

 

Teachers provide inputs. Students give outputs. And those outputs demonstrate the intended learning.

 

Yet this framework was developed in the early 1900s. Using it today is to ignore a century of subsequent psychological advancements. It glosses over the impact of the unconscious, the social nature of understanding, physical differences, even the mediating thought processes between stimulus and response such as memory and problem solving.

 

Instead, we force students into inauthentic laboratory conditions (i.e. the classroom) upon which they are passive actors to be molded and shaped by expert educators.

 

Every time we post our learning objectives on the board or when we write our lesson plans beginning with the old chestnut – Students Will Be Able To (SWBAT) – we are hearkening back to early 20th Century thinking a hundred years out of date.

 

We are enshrining a host of assumptions long past their fresh by date:

-Learning is observable.

 

-It happens immediately.

 

-It is measurable.

 

-Once you learn something it never goes away.

 

-Most problems with learning are attributable to inputs provided by the teacher.

 

None of these assumptions have been proven.

 

In fact, there is considerable evidence against each and every one of these premises, yet our entire system of corporate education is based on them like a house built on a foundation.

 

If we are truly to create a 21st Century school system, the only place to begin is here. Recognize our bedrock beliefs are mere speculation and question whether we should really support everything else that’s been built on such shaky ground.

 

WHAT IS LEARNING?

 

It is an empirical fact that human beings are capable of learning. It’s something we do every day. But what exactly does it consist of? What happens when a person learns?

 

Perhaps it’s best to start with a definition. We generally characterize learning as the acquisition of knowledge; the possession of facts, information or skills.

 

But how does one gain knowledge? How does one possess the intangible?

 

It seems that learning always involves thoughts – usually conscious impressions but sometimes unconscious ones, as well. However, not all thoughts qualify, only thoughts of a certain kind.

 

The notion must be true of the world. And often it is an idea that has surfaced before but that now can be recalled at will and used to create new concepts.

 

Perhaps I’m wrong, but it seems that no matter how you flesh it out, we’re talking about internal mind states.

 

Learning takes place in and of the brain. And this has consequences for our education system – an apparatus designed to make these brain states more frequent along certain prescribed lines.

 

IS LEARNING OBSERVABLE?

 

That depends. Can we lop off the top of students’ heads and peer at the gelatinous mass inside?

 

Not really. And even if we could, we wouldn’t understand what we were seeing.

 

Even if learning may be reducible to a complex set of on-and-off switches among synapses, that does not make it generally observable – certainly not without greater knowledge of how the brain works and advanced neural imaging equipment.

 

As such, the idea that learning is directly perceptible is not necessarily true. It may be evident in some second hand manner, but this is not the same as first hand experience. At best, what we see is a pale shadow of what’s actually going on in students’ gray matter.

 

That alone should send shock waves through the edifice of modern corporate education. We’ve built an entire apparatus to label and sort kids based on observing students. If those observations are inadequate to give us the full picture of these internal learning states, our system is likewise inadequate.

WHAT IS THE ROLE OF TEACHER INPUT?

 

To answer this question we must start further back – when and why does learning takes place.

 

A student experiences a new neural state that constitutes the acquisition of knowledge. Why?

 

Does it happen because of the input made by a teacher? Is it the result of experience? Is it the result of some other input – reading, interacting, writing, doing something? Or is it the result of something even the student him- or herself cannot easily identify or explain?

 

All of these are possible. All of these (and more) are the catalyst to learning at various times.

 

Thus we lose another premise – that teacher input is the essential cause of inadequate learning. If we cannot place a primacy on the teacher, we cannot wholly place blame there either.

 

Certainly teachers are important. They can have a tremendous impact on their students. But they are not strictly necessary. They are not even the prime cause of learning. They facilitate learning in the way a doctor facilitates healing. The surgeon may set the broken bone, but it is the body that actually does the healing. And in the case of learning, the action is not entirely involuntary. It is much more active and intentional.

 

In short, teachers can call students attention to something that sparks learning. They can bring about optimal conditions for learning to take place. But they are not by themselves sufficient for that learning. They cannot make it happen. Insofar as it is voluntary at all, it is up to the student. To give teachers sole reward or blame for student learning is absurd.

 

IS LEARNING IMMEDIATE?

 

Learning may be a response to stimulus of some kind. But when does that response take place? Is it immediate?

 

There is no evidence that it must be so. Certainly there are times when one has learned something immediately. When a child first puts her finger in the flame, she quickly learns to remove it. However, there are some lessons that we don’t learn until many years after that stimulus. For instance, that our parents’ advice was often more sage than we initially gave it credit.

 

Thus, again it is inadequate to place reward or blame on teachers for their students’ learning. You can judge a teacher for what he or she did to help, but not what you take to be the result. Just because the teacher’s input may not have sparked learning in the student now, that doesn’t mean that the same input might not engender learning at a later date, given time.

 

IS LEARNING PERMANENT?

 

Which brings up another question – once you learn something, does it remain yours forever or is it susceptible to degradation?

 

If learning is an internal state – if it is the result of neural connections like any thought or memory – it is susceptible to fading. It can be lost or degraded.

 

Therefore, when students enter a class without prerequisite knowledge, it is not necessarily the fault of their previous teachers. Like any skill, memory or thought – recall is enhanced through repetition. Using the knowledge often results in greater retention.

 

If we want a more intellectual society, we should habitualize critical thinking and reward intelligence in our public interactions. Not the exact opposite.

 

CAN LEARNING BE MEASURED?

 

And finally, we are brought to perhaps the most vital question in the field of education – measurement.

 

What did students grasp and to what degree was it mastered?

 

There is an entire industry based on providing accurate accounting of learning.

 

There are corporations making billions of dollars based on providing this service. Moreover, the school privatization industry is almost completely predicated on the “failure” of public schools as shown by the measurements of these testing corporations.

 

As such, there is a tremendous amount of economic pressure to keep this premise that learning can be accurately measured. However, when looked at logically, it cannot be supported.

 

When we measure learning, what are we measuring? And how are we quantifying it?

 

If learning is an internal state, how do we calculate that? Possibly at some point in the future, we’ll be able to look at real time pictures of the brain and be able to tell which information has been learned and to what degree. But we are not at that point now. Perhaps we will never be.

 

Even if we were, what exactly would we be measuring? What units would we be using? Volts? Amps? Some new element susceptible to subdivision?

 

The fact that we can’t give a definitive answer to that simple question illustrates how vast our ignorance is of learning. We do not understand what goes on in our own heads that constitutes understanding expect in the broadest possible terms.

 

Yet how much importance we put on these crude attempts to measure the ineffable!

 

Grades and test scores are but the rudest approximations of the real phenomena hidden inside our skulls. Yet we sort and rank students on the pedagogical equivalent of cave paintings.

 

“It is easier to measure the number of semicolons used correctly in an essay than the wonderful ideas contained within it,” said Alfie Kohn. “The more focused you are on measurable outcomes, the more trivial your teaching tends to become.”

 

Or as Linda McNeil of Rice University famously observed, “Measurable outcomes may be the least significant results of learning.”

 

Kohn has repeatedly suggested that McNeil’s statement ought to be printed out in “36-point Helvetica, framed, and tacked to the wall of every school administrator’s office in the country” for these same reasons.

 

When we talk about knowledge and learning, we don’t know what we’re talking about.

 

CONSEQUENCES

 

That should make us reluctant to say anything definitive about learning beyond our own ignorance of it.

 

Yet, as in so much of human affairs, when has ignorance ever stopped us?

 

We have to go about the business of educating. We have a society to run, markets to establish and consumers to exploit.

 

Imagine if, instead, we approached learning like explorers or scientists, mapping the shores of our ignorance and determining what helps us comprehend more and better.

 

There are so many tantalizing clues about what helps students learn, ways to foster the spark of inspiration, creativity and critical thinking.

 

I wish we were invested in that activity instead of a capitalist sham of education. We talk much about the skills gap between white and black kids without doing anything constructive about it – a chasm predicated on the fact that one category is predominantly poor and the other privileged.

 

Perhaps we would do better to talk about the ignorance gap of our own understanding of what it means to understand.

 

Perhaps then we wouldn’t be so bold as to monetize that which is fallacious and foolhardy.

 

Perhaps then we would be more curious, thoughtful and kind.

 

Perhaps then we could build a truly modern system of education that values students and not just how they can be transformed into profit.


Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-2