NAEP Test Scores Show How Stupid We Are… To Pay Attention to NAEP Test Scores

thumbnail_Taking-a-test-567860523df78ccc15365b27
 
Brace yourselves!

 

America’s NAEP test scores in 2019 stayed pretty much the same as they were in 2018!

 

And the media typically set its collective hair on fire trying to interpret the data.

 

Sometimes called the Nations Report Card, the National Assessment of Educational Progress (NAEP) test is given to a random sampling of elementary, middle and high school students in member countries to compare the education systems of nations.

 

And this year there was one particular area where US kids did worse than usual!

 

Our scores went down in 8th grade reading!

 

To be honest, scores usually go up or down by about one or two points every year averaging out to about the same range.

 

But this year! Gulp! They went down four points!

 

FOUR POINTS!

 

What does that mean?

 

Absolutely nothing.

 

They’re standardized test scores. They’re terrible assessments of student learning.

 

You might as well compare the relative body temperatures of randomly selected students and wonder why we aren’t bridging the body warmth gap with the somber hummingbird! I mean it has an average  temperature of 114 F! And the best we can do is a measly 98.6 F! Why won’t enough kids get a fever for America!?

 

If test scores have any meaning at all – it’s parental wealth. Rich kids tend to score higher than poor kids. That’s partially because of the inequality of resources each receive, but also because of racial, cultural and economic bias embedded in the questions.

 

So the NAEP shows us what any study of parental income would show. America has a lot of poor kids and underfunded schools.

 

Thanks, NAEP! There’s no way we could ever have figured that out without you!

 

But having this information come to us via test scores allows us to deflect from the real problem and instead continually blame the victim.

 

Why can’t these poor kids from impoverished schools score as well as kids from richer countries with more well-funded schools?

 

I can’t imagine!

 

Typically politicians used the results to push their pet policies.

 

Education Secretary Betsy DeVos used the scores to wash her hands of the entire public education system. I know – isn’t her job to safeguard public schools? It’s like a zoo keeper complaining that the penguins aren’t bringing in enough visitors and then refusing to feed them.

 

DeVos proposed we improve test scores by cutting $4.8 billion from public schools in 2020 and instead pumping $5 billion to a tax credit school voucher scheme that props up private schools.

 

I know that sounds dumb, but before you judge her, realize she also proposed cutting federal funding for afterschool programs, teacher professional development, student support and enrichment programs.

 

So there.

 

Education Blogger Peter Greene claims that this move is based on a reading comprehension problem the Education Secretary is having, herself.

 

She says that the NAEP results mean that 2/3 of American students read below grade level. However, Greene points out that she is conflating two different things – grade level proficiency and NAEP proficiency.

 

Here’s what the NAEP wrote:

 

“The NAEP Proficient achievement level does not represent grade-level proficiency, but rather competency over challenging subject matter. NAEP Achievement levels are to be used on a trial basis and should be interpreted and used with caution.”

 
Which kind of begs the question of why we need these scores in the first place.

 

There is much clearer data out there.

 

A study by the Center on Budget and Policy Priorities concluded that 29 states spent less per student in 2015 than they had before the Great Recession.

 

And the federal government has done little to help. Since 2011, spending on major K-12 programs – including Title I grants for underprivileged students and special education – has been basically flat.

 

According to the Economic Policy Institute in Washington, today’s public schools employ at least 250,000 fewer people than they did before the recession of 2008–09. Meanwhile enrollment has increased by at least 800,000 students.

 

So to ensure our students had the same quality of service children received only a decade ago, we’d need to hire almost 400,000 more teachers!

 

That’s how you cut class size down from the 20, 30, even 40 students packed into a room that you can routinely find in some districts today.

 

If we looked at realities like these instead of test scores – which at best provide us data at several removes – we might actually be motivated to reach solutions.

 

For instance, the U.S. is one of the only countries in the world – if not probably the ONLY country – that funds schools based largely on local taxes. Other developed nations either equalize funding or provide extra money for kids in need. In the Netherlands, for example, national funding is provided to all schools based on the number of pupils enrolled. But for every guilder allocated to a middle-class Dutch child, 1.25 guilders are allocated for a lower-class child and 1.9 guilders for a minority child – exactly the opposite of the situation in the U.S.

 

If we want to compare the US to other countries, this is a perfect place to start.

 

But a focus on test scores obscures the differences.

 

Virtually all of the top scoring countries taking these exams have much less child poverty than the U.S. If they had the same percentage of poor students that we do, their scores would be lower than ours. Likewise, if we had the same percentage of poor students that they do, our scores would go through the roof! We would have the best scores in the world!

 

These scores just mirror back to us our child poverty rate – that more than 1/3 of our students live below the poverty line and more than half of public school students qualify for free or reduced lunches.

 

But this myopic focus on standardized tests also blinds us to the ways our system is superior to that of many other countries.

 

We do something that many international systems do not. We educate everyone! Foreign systems often weed children out by high school. They don’t let every child get 13 years of grade school (counting kindergarten). They only school their highest achievers.

 

So when we compare ourselves to these countries, we’re comparing ALL of our students to only SOME of theirs – their best academic pupils, to be exact. Yet we still hold our own given these handicaps!

 
This suggests that the majority of problems with our public schools are monetary. Pure and simple.

 

At least House Democrats passed a Labor-HHS-Education funding bill to increase public school funding by $3.5 billion. Even if it were somehow passed by the Republican controlled Senate, that’s a drop in the bucket after decades of neglect – but it’s something!

 

It’s certainly better than DeVos who claims that funding somehow doesn’t matter for public schools – only for her pet charter and voucher schools.

 

A 2018 review by Northwestern University found that in 12 out of 13 studies increased spending had a positive effect on student outcomes. And that result has been verified by studies since then in California, Texas, Wisconsin and other states.

 
Money makes a difference.

 

Money spent on students – not more testing.

 

So why the drop in this year’s 8th grade reading scores?

 

Who knows? It could be a spike in the rate or effect of child poverty in the middle school years.

 

It could be the impact of decades of high stakes testing on middle school curriculum – narrowing what is taught and muscling out authentic instruction.

 

Frankly it doesn’t matter because the data is suspect.

 

Standardized testing will never give us an accurate picture of what is going on with our students or our schools.

 

And until we, as a society, finally realize that and focus on things that actually matter, we will continue to fail the only test that matters – how well we provide for our children.

 

 


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-4

The Absurdity of Standardized Testing: Caught Between Prediction and Assessment

thumbnail_Screen Shot 2019-09-19 at 2.07.03 PM

 
Standardized testing is truly absurd.

 
It’s both a prediction and an assessment.

 

You take a test to determine what you’ve learned and that will in turn predict what you will be able to learn in the future.

 

We hardly ever do this anywhere else in life.

 

We don’t measure babies’ leg muscles to predict whether they’re ready to walk. We let them do what they do, possibly with some encouragement and positive models of locomotion, and they do it.

 

There are cognitive and developmental benchmarks we look for, and if children don’t hit them, we provide help. But no further prediction is necessary – certainly not based on artificial markers put together by corporate interests.

 

In most situations, predictions are superfluous. We just assume that everyone can learn if they so desire – unless something happens to make us think otherwise. And whether someone actually learns something is demonstrated by doing the thing, itself.

 

The only time we link prediction and assessment so closely is when the consequences of failure are irreversible – like when you’re going solo skydiving for the first time. If you jump out of an airplane and don’t know how to pull the ripcord to get your parachute to work, you probably won’t get a second chance to try again.

 

But most things in life aren’t so dire.

 

The world of standardized testing is very different. The high stakes nature of the assessments are what ramp out the consequences and thus the severity.

 

Testing looks at learning like two points on a map and sets up a gate between points A and B.

 

In order to cross, you have to determine if you’ve passed through the previous gate. And only then can you be allowed to progress on to point C.

 

But this is wrong on so many levels.

 

First, you don’t need a test to determine which point you’re at. If Point B is the ability to add, you can simply add. If it’s the ability to write a complete sentence, you can simply write a sentence.

 

There is no need to fill out a formal multiple-choice assessment that – depending on the complexity of the task being considered – is completely inadequate to capture the subtleties involved. The task, itself, is enough.

 

Imagine if you were testing whether someone had learned how to drink a glass of water. You could just give them a cup filled with H2O and see if they can gulp it down. Or you could have them sharpen their number 2 pencil and answer questions about how their throat works, their digestive and excretory systems and the chemical composition of agua – all answers predetermined to A, B, C or D.

 

Observation of a skill, we are told, is not enough to determine success because it relies on the judgment of an observer. A standardized test replaces the observer with an impersonal, distant testing corporation which then assesses only predetermined markers and makes decisions devoid of any situational context.

 

This is done to remove observational bias but it doesn’t avoid bias altogether. In setting up the markers and deciding which elements of the task are to be assessed (or in fact can be assessed in such a distant manner), the testing corporation is inserting its own biases into the process. In fact, in any assessment conducted by human beings, this would be inevitable. So going through this maze of perceived objectivity is really just a matter of subterfuge meant to disguise the biases of the corporation.

 

Second, assessing people in this way is extremely unnatural because very few fields of knowledge can be divided and subdivided into two or more discrete points.

 

When writing a complex sentence, for example, you need to know not just spelling and grammar but logic, handwriting, subject matter, colloquialisms, literary devices, and a plethora of other cultural and linguistic artifacts.

 

Moreover, there is not always a natural progression from Point A to B to C. Sometimes A jumps directly to C. Sometimes B leads directly to A. Sometimes A leads to Z.

 

Knowledge, skills and human cognition are far too complex a web to ever hope to be captured by such a reductive enterprise. But by insisting that we make this complexity fit into such a small box, we end up depriving people of the right to move on. We say predictive models show they aren’t ready to move forward and so we bury them in remediation. Or we deny them access to important opportunities like advanced classes, electives, field trips, extracurricular clubs or even post-secondary education.

 

Third, this emphasis on knowledge as discrete bits of information or skills (often called standards) leads to bad teaching.

 

Assessment expert W. James Popham provides a helpful distinction: “curriculum teaching” vs. “item teaching.” Curriculum teachers focus on the full body of knowledge and skills represented by test questions. For instance, if the test is expected to include questions about decimals, the teacher will cover the full range of knowledge and skills related to decimals so students understand what they are, know how to manipulate them, understand how to use them to solve more complex problems, and are able to communicate about them.

 

By contrast, item teaching involves narrowing instruction, organizing lessons around look-a-like questions that are taken directly from the test or represent the kinds of questions most likely to be found on the test. In this way, the teacher only provides the chunks of knowledge students are most likely to encounter on exams. For instance, item teachers might drill students on a certain set of vocabulary words that are expected to be assessed rather than employing instructional strategies that help students build a rich vocabulary that best contributes to strong reading comprehension.

 

A focus on standardized testing or even trying to educate in a system where these tests are attached to high stakes, results in an increase in item teaching. We often call it teaching to the test.

 

I’m not saying that item teaching is always bad. But curriculum teaching is to be much preferred. It is a best practice. The problem is when we resort to endless drills and give students innumerable questions of the exact type we expect to be on the test.

 

So when we find students who have made dramatic improvements on standardized tests, we often don’t find equal improvements in their over all knowledge or ability.

 

Test scores are often a false positive. They show students have mastered the art of taking the test but not necessarily the knowledge or skills it was meant to assess.

 

They are more like trained circus animals who can jump through flaming hoops but would be lost in the wild.

 

That’s why certain computer modeled artificial intelligences are able to pass standardized tests but would fail preschool.

 

These reflections have troubling implications for our system of standardized testing.

 

The false curtain of objectivity we’ve set up in our assessments may also be hiding from us what authentic learning is taking place and it may even hinder such learning from taking place at all.

 

Any sane society would halt such a system with these drawbacks. It would stop, regroup and devise a better alternative.

 

To continue with such a pedagogical framework truly would be the most absurd thing of all!


 

 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

book-2

Don’t Worry About Grade Inflation. Worry About Grading Fairly.

your-childs-grades-go-down-1494627682

 

Hard work should be rewarded.

 

If you earn an A in a given class, you should get an A on your report card.

 

And this is true no matter how many of your classmates work just as hard as you do.

 

If everyone in class gives it their best shot, they should all get A’s. It is not the teacher’s job to split hairs and sort kids into arbitrary categories in order to preserve a monetary myth about grades’ value based on a model of scarcity.

 

Those who demand otherwise are under the spell of one of the oldest myths in academia – grade inflation.

 

It goes like this: You can’t give all your students excellent marks! That would devalue what it means to get an A!

 

To which I reply: Bullshit.

 

Almost every plane that leaves an airport lands safely. Does that devalue what it means to travel? When you arrive at your destination, are you upset that everyone else has arrived safely or would you feel better if some of the planes crashed?

 

According to the American Journal of Public Health, 93% of New York City restaurants earn an “A” from the health department. Does that shake your faith in the food service industry? Would you feel better if more restaurants were unsanitary? Would your food digest more efficiently if there were more people going home with stomach pain and food poisoning?

 

Of course not! In fact, these stats actually reassure us about both industries. We’re glad air travel and eating out is so safe. Why would we feel any different about academia?

 

The idea of grade inflation is a simple imposition of the concept of economic value onto learning. It has no meaning in the field of academics, psychology or ethics. It is just some fools who worship money imaging that the whole world works the same way – and if it doesn’t, it should.

 

It’s nothing new.

 

Conservatives have been whining about grade inflation for at least a century. It’s not that the quality of teachers has declined and they’re letting all their students pass without doing the work. It’s that certain types of curmudgeons want to justify their own intelligence by denying others the same privilege.

 

It’s the “I’ve Got Mine” philosophy.

 

We see the same thing with Baby Boomers who grew up in the counter culture and pushed for progressive values in their youth. Once they got everything they wanted for themselves, they became conservatives in their old age and worked to deny the same things for subsequent generations.

 

It’s the very definition of Age scoffing at Youth – a pathology that goes back at least to Hesiod if not further. (Golden age of man, my foot!)

 

Moreover, there is no authentic way to prove grade inflation is actually happening. Grades are a subjective measure of student learning. They are human beings’ attempt to gauge an invisible mental process. At best they are frail approximations of a complex neural process that is not even bounded temporally or causally. If a student doesn’t know something now, they may come to know it later even without further academic stimulus. Moreover, isolating the stimulus that produced the learning is also nearly impossible.

 

The important thing is not grade inflation. It is ensuring that grades are given fairly.

 

If students work hard, they should be rewarded.

 

I am very upfront with my students about this. And doing so seems to have a positive and motivating effect on them.

 

This year, I had students who told me they had never read a book from cover-to-cover before my class. I’ve had students look at their report cards in shock saying they’ve never received such high marks in Language Arts before. And doing so makes them want to try all the harder next year to repeat the results.

 

They leave me excited about learning. They feel empowered and ready to give academics their all. Because the greatest lesson a teacher can instill is that the student is capable of learning.

 

Don’t get me wrong. I don’t just hand out these grades. Students have to earn them. They have to demonstrate that they have actually learned something.

 

Everyone rarely measures up to the challenge. But that’s not the point. Everyone COULD. There is nothing in my design that prohibits that outcome. I don’t start with the assumption that I’ll only have 3 A’s, 4 B’s, 10 C’s, etc.

 

In fact, it is THAT scarcity model that dumbs down academics. If I grade on a curve like that, I have to give out a certain number of high marks regardless of achievement. I’m committed to giving out those 3 A’s regardless of whether that trio of students deserve it or not. However, in my abundance model, I give exactly the number of A’s that are deserved. If that’s zero, no one gets an A. If that’s everyone, then everyone gets an A. It all depends on what students actually deserve, not some preconceived notion about how the world works.

 

To do this, I give very few tests. I just don’t find them to be very helpful assessments.

 

A test is a snapshot of student learning. It has its place, but the information it gives you is very limited.

 

 

Most of my grades are based on projects, homework, essays, class discussion, creative writing, journaling, poetry, etc. Give me a string of data points from which I can extrapolate a fair grade – not just one high stakes data point.

 

This may work to some degree because of the subject I teach. Language arts is an exceptionally subjective subject, after all. It may be more challenging to do this in math or science. However, it is certainly attainable because it is not really that hard to determine whether students have given you their best work.

 

Good teaching practices lend themselves to good assessment.

 

You get to know your students. You watch them work. You help them when they struggle. By the time they hand in their final product, you barely need to read it. You know exactly what it says because you were there for its construction.

 

For me, this doesn’t mean I have no students who fail. Almost every year I have a few who don’t achieve. This is usually because of attendance issues, lack of sleep, lack of nutrition, home issues or simple laziness.

 

I only have control over what happens in the classroom, after all. I can call home and try to work with parents, but if those parents are – themselves – absent, unavailable or unwilling to work with me, there’s little I can do.

 

And before you start on about standardized testing and the utopia of “objectivity” it can bring, let me tell you about one such student I had who was not even trying in my class.

 

He never turned in homework, never tried his best on assignments, rarely attended and sleepwalked through the year. However, he knew his only chance was the state mandated reading test – so for three days he was present and awake. The resulting test score was the only reason he moved on to the next grade.

 

Was he smart? Yes. Did he deserve to go on to the next grade not having learned the important lessons of his classmates? No. But your so-called “objective” measure valued three days of effort over 180.

 

The problem is that we are in love with certain academic myths.

 

MYTH 1: Grading must be objective.

 

WRONG! Grading will never be objective because it is done by subjective humans. These standardized tests you’re so in love with are deeply biased on economic and racial lines. Whether you pass or fail is determined by a cut score and a grading curve that changes from year-to-year making them essentially useless for comparisons and as valid assessments. They’re just a tool for big business to make money off the academic process.

 

MYTH 2: Learning is Economics.

 

WRONG! Grades are not money. They don’t function in any way like currency or capital. They aren’t something to be bought and sold. They are an approximate indication of academic success. Treating them as a commodity only degrades their value and the value of students and learning, itself.

 

Treating grades economically actually represses the desire to learn, dispels curiosity and eliminates the intrinsic value of education.

 

So go ahead – inflate the “value” of your grades.

 

Give A’s to every student who deserves it.

 

That’s how you promote learning and fairly assess it.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-1