Greater Test Scores Often Mean Less Authentic Learning

Screen Shot 2019-10-04 at 4.25.32 PM

 
The main goal of schooling is no longer learning.

 

It is test scores.

 

Raising them. Measuring growth. Determining what each score means in terms of future instruction, opportunities, class placement, special education services, funding incentives and punishments, and judging the effectiveness of individual teachers, administrators, buildings and districts.

 

We’ve become so obsessed with these scores – a set of discrete numbers – that we’ve lost sight of what they always were supposed to be about in the first place – learning.

 

In fact, properly understood, that’s the mission of the public school system – to promote the acquisition of knowledge and skills. Test scores are just supposed to be tools to help us quantify that learning in meaningful ways.

 
Somewhere along the line we’ve misconstrued the tool for the goal. And when you do that, it should come as no surprise that you achieve the goal less successfully.

 

There are two kinds of standardized assessment – aptitude and achievement tests. Both are supposed to measure scholarship and skill – though in different ways.

 

Aptitude tests are designed to predict how well a student will do in the future. Achievement tests are designed to determine how much a student knows now.

 

There is, of course, intense overlap between these two types because aptitude tests base their predictions on assessment of achievements. So they’re basically achievements tests that go one step further. They ask questions designed to give more information than just the present state but also about whether a student has progressed to a state which is most likely to then give way to another state in the future.

 

Either way, standardized assessments are supposed to be based on what students have learned. But the problem is that not all learning is equal.

 

For example, a beginning chef needs to know how to use the stove, have good knife skills and how to chop an onion. But if you give her a standardized test, it instead might focus on how long to stir the risotto.

 

That’s not as important in your everyday life, but the tests make it important by focusing on it.

 

The fact of the matter is that standardized tests do NOT necessarily focus on the most important aspects of a given task. They focus on obscurities – things that most students don’t know.

 

This is implicit in the design of these exams and is very different from the kinds of tests designed by classroom teachers.

 

When a teacher makes a test for her students, she’s focused on the individuals in her classes. She asks primarily about the most essential aspects of the subject and in such a way that her students will best understand. There may be a few obscure questions, but the focus is on whether the test takers have learned the material or not.

 

When psychometricians design a standardized test, on the other hand, they aren’t centered on the student. They aren’t trying to find out if the test taker knows the most important facts or has the most essential skills in each field. Instead, there is a tendency to eliminate the most important test questions so that the test – not the student – will be better equipped to make comparisons between students based on a small set of questions. After all, a standardize test isn’t designed for a few classes – it is one size fits all.

 

New questions are field tested. They are placed randomly on an active test but don’t count toward the final score. Test takers aren’t told which questions they’ll be graded on and which are just practice questions being tried out on students for the first time. So students presumably give their best effort to both types. Then when the test is scored, the results of the field test questions determine if they’ll be used again as graded questions on a subsequent test.

 

According to W. James Popham, professor emeritus at the University of California and a former president of the American Educational Research Association, standardized test makers take pains to spread out the scores. Questions answered correctly by too many students – regardless of their importance or quality – are often left off the test.

 

If 40 to 60 percent of test takers answer the question correctly, it might make it onto the test. But questions that are answered correctly by 80 percent or more of test takers are usually jettisoned.

 

He writes:

 

“As a consequence of the quest for score variance in a standardized achievement test, items on which students perform well are often excluded. However, items on which students perform well often cover the content that, because of its importance, teachers stress. Thus, the better the job that teachers do in teaching important knowledge and/or skills, the less likely it is that there will be items on a standardized achievement test measuring such knowledge and/or skills.”

 

Think about what this means.

 

We are engaged in a system of assessment that isn’t concerned with learning so much as weeding people out. It’s not about who knows what, but about which questions to ask that will achieve the predetermined bell curve.

 

We talk about leaving no child left behind, and making sure all students do better on standardized tests, but these tests are norm-referenced. By definition, all students cannot score well no matter how great their knowledge or skills. If you gave a standardized test to a class of genius-level intellects, there would still be the same percentage of failures and outstanding scores with the majority clustered in the middle. That’s how the tests are designed.

 

And if this highly suspect method of question selection, alone, doesn’t achieve that end, the test companies have a way to correct the scores at the end of the process through the way they grade them.

 

These tests are graded with cut scores. In other words, the state or the testing company or the graders, themselves, decide anew each year which scores are passing and which failing.

 

One year a 1200 might be proficient. Another year it’s basic. It all depends on what the decision makers come up with on a given year.

 

What do they base this on? No one has ever given a definitive answer. In fact, I doubt there is one. In each case, the deciding body just makes it up.

 

We’ve seen countless times when state scores are criticized for being too low one year, and then they miraculously bounce up the next. It’s not that students score differently, it’s that the cut score was raised. Why? Perhaps to stifle questions about the test’s validity. After all, people are less angry when more students pass.

 

The goal is always getting the bell curve. That is what validates the tests. But it’s a human construction, not a function of assessment. It says less about the test takers than the test makers and their enablers.

 

This has huge implications for the quality of education being provided at our schools. Since most administrators have drunk deep of the testing Kool-Aid, they now force teachers to use test scores to drive instruction. So since the tests don’t focus on the most essential parts of Reading, Writing, Math, and Science, neither does much of our instruction.

 

We end up chasing the psychometricians. We try to guess which aspects of a subject they think most students don’t know and then we teach our students that to the exclusion of more important information. And since what students don’t know changes, we end up having to change our instructional focus every few years based on the few bread crumbs surreptitiously left for us by the state and the testing corporations.

 

That is not a good way to teach someone anything. It’s like teaching your child how to ride a bike based on what the neighbor kid doesn’t know.

 

It’s an endless game of catch up that only benefits the testing industry because they cash in at every level. They get paid to give the tests, to grade the tests and when students fail, they get paid to sell us this year’s remediation material before kids take the test again, and – you guessed it – the testing companies get another check!

 

It’s a dangerous feedback loop, a cycle that promotes artificially prized snippets of knowledge over constructive wholes. But this degradation of education isn’t even the worst part.

 

The same method of question selection also builds economic and racial bias into the very fabric of the enterprise.

 

According to Prof. Martin Shapiro of Emory University, when test makers select questions with the greatest gaps between high and low scorers, they are selecting against minorities. Think about it – if they pick questions based on the majority getting it right, which minority got it wrong? In many cases, it’s a racial minority. In fact, this may explain why white students historically do better on standardized tests than black and Hispanic students.

 

This process may factor non-school learning and social background into the questions. They are based on the experiences of white middle-to-upper class children.

 

So when we continually push for higher test scores, not only are we ultimately dumbing down the quality of education in our schools, but we’re also explicitly lobbying for greater economic and racial bias in our curriculum trickling down from our assessments.

 

As Ibram X. Kendi, author of “How to be an Antiracist” puts it:

 

“Standardized tests have become the most effective racist weapon ever devised to objectively degrade Black minds and legally exclude their bodies.”

 

 

71087927_10157069943303860_6347601423106048000_n

Popham is less critical of high stakes testing. He sees more of a problem in using student test scores to assess teacher performance. But even he thinks the tests and the scores are being over valued and misunderstood in a wider context.

 

He writes:

 

“Merely because these test scores are reported in numbers (sometimes even with decimals!) should not incline anyone to attribute unwarranted precision to them. Standardized achievement test scores should be regarded as rough approximations of a student’s status with respect to the content domain represented by the test.”

 

I’d go even further.

 

Standardized test scores are tools used by big business to make money. That is as far as their validity goes.

 

And the fact that we make so many vital educational decisions on them is nothing less than criminal.

 

The tests are bogus nonsense at best and a conspiracy against the poor and minorities at worst.

 

When well-meaning people let themselves get wrapped up in knots over low scores and what that means for student learning, they are actually hurting the very thing that they value.

 

Student learning is not bettered by higher test scores. It is often made worse by them.

 

High test scores don’t mean greater learning. They often mean learning the knowledge du jour to the detriment of what’s really important. They mean biased education against the poor and minorities.

 

And they make those with real concerns complicit in a sham being perpetrated on our children and our society.

 


 

 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-4

 

Don’t Worry About Grade Inflation. Worry About Grading Fairly.

your-childs-grades-go-down-1494627682

 

Hard work should be rewarded.

 

If you earn an A in a given class, you should get an A on your report card.

 

And this is true no matter how many of your classmates work just as hard as you do.

 

If everyone in class gives it their best shot, they should all get A’s. It is not the teacher’s job to split hairs and sort kids into arbitrary categories in order to preserve a monetary myth about grades’ value based on a model of scarcity.

 

Those who demand otherwise are under the spell of one of the oldest myths in academia – grade inflation.

 

It goes like this: You can’t give all your students excellent marks! That would devalue what it means to get an A!

 

To which I reply: Bullshit.

 

Almost every plane that leaves an airport lands safely. Does that devalue what it means to travel? When you arrive at your destination, are you upset that everyone else has arrived safely or would you feel better if some of the planes crashed?

 

According to the American Journal of Public Health, 93% of New York City restaurants earn an “A” from the health department. Does that shake your faith in the food service industry? Would you feel better if more restaurants were unsanitary? Would your food digest more efficiently if there were more people going home with stomach pain and food poisoning?

 

Of course not! In fact, these stats actually reassure us about both industries. We’re glad air travel and eating out is so safe. Why would we feel any different about academia?

 

The idea of grade inflation is a simple imposition of the concept of economic value onto learning. It has no meaning in the field of academics, psychology or ethics. It is just some fools who worship money imaging that the whole world works the same way – and if it doesn’t, it should.

 

It’s nothing new.

 

Conservatives have been whining about grade inflation for at least a century. It’s not that the quality of teachers has declined and they’re letting all their students pass without doing the work. It’s that certain types of curmudgeons want to justify their own intelligence by denying others the same privilege.

 

It’s the “I’ve Got Mine” philosophy.

 

We see the same thing with Baby Boomers who grew up in the counter culture and pushed for progressive values in their youth. Once they got everything they wanted for themselves, they became conservatives in their old age and worked to deny the same things for subsequent generations.

 

It’s the very definition of Age scoffing at Youth – a pathology that goes back at least to Hesiod if not further. (Golden age of man, my foot!)

 

Moreover, there is no authentic way to prove grade inflation is actually happening. Grades are a subjective measure of student learning. They are human beings’ attempt to gauge an invisible mental process. At best they are frail approximations of a complex neural process that is not even bounded temporally or causally. If a student doesn’t know something now, they may come to know it later even without further academic stimulus. Moreover, isolating the stimulus that produced the learning is also nearly impossible.

 

The important thing is not grade inflation. It is ensuring that grades are given fairly.

 

If students work hard, they should be rewarded.

 

I am very upfront with my students about this. And doing so seems to have a positive and motivating effect on them.

 

This year, I had students who told me they had never read a book from cover-to-cover before my class. I’ve had students look at their report cards in shock saying they’ve never received such high marks in Language Arts before. And doing so makes them want to try all the harder next year to repeat the results.

 

They leave me excited about learning. They feel empowered and ready to give academics their all. Because the greatest lesson a teacher can instill is that the student is capable of learning.

 

Don’t get me wrong. I don’t just hand out these grades. Students have to earn them. They have to demonstrate that they have actually learned something.

 

Everyone rarely measures up to the challenge. But that’s not the point. Everyone COULD. There is nothing in my design that prohibits that outcome. I don’t start with the assumption that I’ll only have 3 A’s, 4 B’s, 10 C’s, etc.

 

In fact, it is THAT scarcity model that dumbs down academics. If I grade on a curve like that, I have to give out a certain number of high marks regardless of achievement. I’m committed to giving out those 3 A’s regardless of whether that trio of students deserve it or not. However, in my abundance model, I give exactly the number of A’s that are deserved. If that’s zero, no one gets an A. If that’s everyone, then everyone gets an A. It all depends on what students actually deserve, not some preconceived notion about how the world works.

 

To do this, I give very few tests. I just don’t find them to be very helpful assessments.

 

A test is a snapshot of student learning. It has its place, but the information it gives you is very limited.

 

 

Most of my grades are based on projects, homework, essays, class discussion, creative writing, journaling, poetry, etc. Give me a string of data points from which I can extrapolate a fair grade – not just one high stakes data point.

 

This may work to some degree because of the subject I teach. Language arts is an exceptionally subjective subject, after all. It may be more challenging to do this in math or science. However, it is certainly attainable because it is not really that hard to determine whether students have given you their best work.

 

Good teaching practices lend themselves to good assessment.

 

You get to know your students. You watch them work. You help them when they struggle. By the time they hand in their final product, you barely need to read it. You know exactly what it says because you were there for its construction.

 

For me, this doesn’t mean I have no students who fail. Almost every year I have a few who don’t achieve. This is usually because of attendance issues, lack of sleep, lack of nutrition, home issues or simple laziness.

 

I only have control over what happens in the classroom, after all. I can call home and try to work with parents, but if those parents are – themselves – absent, unavailable or unwilling to work with me, there’s little I can do.

 

And before you start on about standardized testing and the utopia of “objectivity” it can bring, let me tell you about one such student I had who was not even trying in my class.

 

He never turned in homework, never tried his best on assignments, rarely attended and sleepwalked through the year. However, he knew his only chance was the state mandated reading test – so for three days he was present and awake. The resulting test score was the only reason he moved on to the next grade.

 

Was he smart? Yes. Did he deserve to go on to the next grade not having learned the important lessons of his classmates? No. But your so-called “objective” measure valued three days of effort over 180.

 

The problem is that we are in love with certain academic myths.

 

MYTH 1: Grading must be objective.

 

WRONG! Grading will never be objective because it is done by subjective humans. These standardized tests you’re so in love with are deeply biased on economic and racial lines. Whether you pass or fail is determined by a cut score and a grading curve that changes from year-to-year making them essentially useless for comparisons and as valid assessments. They’re just a tool for big business to make money off the academic process.

 

MYTH 2: Learning is Economics.

 

WRONG! Grades are not money. They don’t function in any way like currency or capital. They aren’t something to be bought and sold. They are an approximate indication of academic success. Treating them as a commodity only degrades their value and the value of students and learning, itself.

 

Treating grades economically actually represses the desire to learn, dispels curiosity and eliminates the intrinsic value of education.

 

So go ahead – inflate the “value” of your grades.

 

Give A’s to every student who deserves it.

 

That’s how you promote learning and fairly assess it.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-1