The majority of teachers and principals in Pennsylvania hate standardized tests.
An increasing number of parents are refusing to allow their kids to take the tests.
And there may be better alternatives to the state’s Keystone Exams.
These were just some of the key findings of a blockbuster report from June 2019 by the state General Assembly’s Legislative Budget and Finance Committee.
However, now that things are returning to some semblance of normalcy, it seems that bureaucrats from the state Department of Education (PDE) are taking the wrong lessons from the report while the legislature seems to have forgotten it entirely.
The report was conducted because of legislation written by state Sen. Ryan P. Aument (R-Lancaster County). It directed the Legislative Budget and Finance Committee to “study the effectiveness of standardized testing, including the Keystone Exams and SATs, and their use as indicators of student academic achievement, school building performance, and educator effectiveness.”
The key findings are as follows:
1)The majority of principals and teachers disapprove of the state’s standardized tests – both the Pennsylvania System of School Assessment (PSSA) tests given in grades 3-8 and Keystone Exams given in high school. They think these tests are ineffective, expensive and harmful to district curriculum and students.
2) State law allows parents to opt their children out of testing for one reason only – religious grounds. Parents are using this religious exemption in increasing numbers. This puts districts in danger of violating federal participation and accountability standards.
3) It has been suggested that the state allow two additional reasons for parents to opt their children out of testing – philosophical grounds and health concerns. It is unclear whether doing so would increase overall opt outs or not.
4) The federal Every Student Succeeds Act (ESSA) passed in 2015 allows the use of the SAT and/or ACT test to take the place of high school standardized testing. It has been suggested the Commonwealth replace the Keystone Exams with these tests. The report finds the ACT and/or SAT would successfully determine college readiness and reduce the overall amount of standardized testing. However, this would not allow other uses of current state tests like evaluating teacher effectiveness and school building performance.This may not matter though because the report also casts doubt on whether the current tests (PSSA and Keystone Exams) do an adequate job of assessing teacher or building performance now or even if student tests can be accurately used to evaluate teachers and schools.
There’s a lot of information here. Let’s look at each finding in turn.
1) PA Educators Hate Standardized Tests
When it comes to the PSSAs, 67% of principals and 76% of teachers said the tests were ineffective indicators of student achievement.
There was slightly more support for the Keystone Exams. This time 45% of principals said the test was an ineffective indicator of student achievement (with 27% saying the tests were effective). Meanwhile, 60% of teachers said the test was ineffective.
Both principals and teachers said their curriculum had been narrowed to prepare students for PSSAs and Keystone Exams. Instead of going into more depth on regular classwork or learning new skills, the focus shifts to teaching to the tests.
Most principals (approximately 80-90%) said that students are taught test-taking skills, and their schools administered practice tests, bench-mark tests, and/or diagnostic tests to prepare students for the PSSA exams. This held for teachers, too, with 81-88% saying they teach test-taking skills and administer practice tests. Principals also said the costs of this additional test prep varied from $200 to more than $100,000.
Taking the tests also eats up valuable class time. Administering the assessments takes between 5.7 to 8 days for each kind of test – the PSSA and the Keystone Exams, according to Principals.
In addition, the report details the cost of giving these tests. In fiscal year 2017-2018, PDE paid $42.17 million for these tests. This is part of a national trend:
“Standardized tests and test preparation have subsequently become big business and that multibillion dollar business continued to grow since the enactment of NCLB and the subsequent enactment of ESSA. According to the Pew Center on the States, annual state spending on standardized tests increased from $423 million before the NCLB (enacted in 2002) to upwards of $1.1 billion in 2008 (to put this in perspective this reflects a 160 percent increase compared to a 19.22 percent increase in inflation during the same time period). A more recent study by the Brown Center on Education Policy at Brooking put the cost at upwards of $1.7 billion in 2011 related to state spending on standardized tests.”
2) Opt Outs on the Rise
Many states allow parents to opt their children out of standardized testing. Some do so in cases of a physical disability, medical reasons, or emergencies. A few allow opt-outs based on religious objection – like Pennsylvania. Some states allow opt-outs for any reason whatsoever.
The religious exemption is not used widespread throughout the state and most schools meet the 95% participation rate required by the federal government. However, use of the religious exemption is definitely on the rise – enough so that the authors of the report find it alarming:
“Meanwhile, as previously indicated in this section, schools throughout the country are experiencing and grappling with an increase in the number of parents seeking to have their children opt-out of standardized testing now that new state assessments have been implemented pursuant to the federal requirements. Pennsylvania is no exception to this trend and is also experiencing an increase in the number of parents utilizing the religious opt-out.”
For the PSSA tests, opt outs increased from 2013-14 to 2016-17. However, total numbers in school year 2017-18 decreased sightly.
Opt outs went from 1,886 to 6,425 to 15,644 to 19,012 to 16,961.
During the same time period for the Keystone Exams, opt outs steadily increased each year but were at lower overall rates.
For the high school test, opt outs went from 382 to 666 to 1,000 to 1,313 to 1,633.
These are vitally important figures because opt out data is rarely tabulated and released to the public. Many media accounts actually state the opposite of the data in this report – in particular that opting out has decreased since Congress passed the ESSA in 2015. Apparently the media got this one wrong.
Though the religious opt-out is the only reason specifically allowed in state Chapter 4 regulations, PDE reports there are five additional ways that students end up not taking the tests:
1) Other Parental Request – parents simply refusing to let their kids participate but not objecting based on specifically religious reasons.
2) No Attempt and No Exclusion Marked – students who are given the test but do not answer enough questions to receive a score.
3) No Test – no test record on file for unknown reasons.
4) Extended Absence – a student missed the testing and make-up window due to absences.
5) Other – does not fit any of the other categories.
Federal law – in particular No Child Left Behind (NCLB) and subsequent reauthorizations of that legislation – requires states to use student participation in standardized testing as a factor in a state’s accountability system. According to the report, any district with less than 95% of students taking the test should be “addressed.”
The report does not go into any further detail about what this means, other than to say that falling under 95% can:
“…ultimately result in a reduced achievement/proficiency measure… If the student participation rate falls below 95 percent, states are required to calculate student achievement/proficiency by dividing the number of students scoring proficiently by no less than 95 percent of the total students (which effectively assigns a score of “0” to all nonparticipants once the participation rate has fallen below 95 percent).”
In effect, the district gets a bad mark on a piece of paper. So what?
Under NCLB, schools with poor performance could receive sanctions like state takeover or lower funding. However, this is extremely unlikely – especially since the passing of ESSA. This newest reauthorization of the law gives states leeway in designing their accountability systems. It leaves the enforcement of this 95% participation rate up to the states, requiring them to develop an accountability plan in the event that a school or district fails to meet this standard.
So a school would only be punished if the state decided to do so. If a state legislature decided to allow parents to opt their children out for any reason at all, they would not have to take any punitive measures. Since the ESSA, the buck stops at the state house door on this one. California, for example, takes note of low participation rates but these rates are not factored into a school’s rating. On the other hand, Florida mandates direct intervention from the state’s Department of Education until participation rates are met.
3) Impact of New Reasons to Opt Out
This is where things get a bit sticky.
The report mentions the idea of expanding the options for opting-out of statewide assessments (e.g., PSSA and Keystone Exams) to include objections based on philosophical grounds or due to health issues.
On the one hand, the authors write “The impact of adding opt-out categories may be minimal.” They don’t know if more people would use the expanded options or if the same numbers who use the religious exemption today might simply divide themselves up among all three options.
The authors worry, however, that new pathways to opt out may increase the total number of people refusing the tests for their children and would reduce Pennsylvania’s participation rate in standardized testing.
This is a particularly troubling paragraph:
“The existence of opt-outs (religious or otherwise) has the potential to negatively impact a state’s participation rates and may potentially impact a state’s [Local Education Agency (LEAs)] and schools achievement/proficiency rate and ultimately the ability of a state to be in compliance with federally required assessments and accountability measures. Furthermore, providing opt-outs and giving parents notice of such has the potential to conflict with the message about the importance of standardized testing. Ultimately placing the state departments of education and local school districts in the potentially awkward position of having to explain why it is important for students to participate in testing (given the federal requirements), while also giving and notifying parents of the opt-out options for their children. In 2015, US Department of Education sent out letters to a dozen states flagging their low participation rates (statewide, or at the district or subgroup level) on the 2014-15 school year assessments and indicated that they needed to create a plan to reduce opt-outs due to low participation rates.”
This seems to be the order of the day at PDE. It’s why earlier this year, school administrators were advised by state officials to crack down on parents opting their children out of standardized tests.
And all of it is based on a cowardly and incomplete understanding of federal law. If Commonwealth schools fall below 95% participation in the test and get a bad mark on a worthless metric, it doest have to matter. No matter how many letters the federal government sends to the state legislature or PDE, the law is clear. The state is in charge here. Our legislature can choose to side with taxpayers, residents, and citizens or with civil servants and strongly worded letters.
4) Replacing the Keystone Exams
There’s not much more to add to this than the initial finding.
The authors of the report say there would be no problem with replacing the Keystone Exams with the ACT or SAT because these national tests would properly assess students’ college and career readiness.
The report is actually pretty shoddy in this regard not really examining the claims of the College Board which makes both tests. The authors just pretty much accepted the College Board’s word wholesale. Nor was their any evaluation of what teachers and principals thought about these tests like there had been for the PSSA and Keystone Exams.
However, the report does make a good point about test reduction. Many students already take the SAT or ACT test, so eliminating the Keystones would reduce the over all amount of tests they had to take.
Also the authors deserve credit for writing about how using student test scores to evaluate teachers and schools is seriously bad practice.
According to the report, 77 percent of principals and 93 percent of teachers said PSSA tests were not effective indicators for teacher evaluations, with similar figures for the Keystone Exams and building performance evaluations.
While everything in the report may not be 100% accurate, it includes important information that should be wider known.
In addition, the report has trustworthy data about opt outs throughout the Commonwealth. Unlike what has been reported in the media, opting out is not on the decline, it is on the rise.
The Keystone Exams should be thrown in the trash, because that’s what they are – trash. At very least they should be replaced with the SAT or ACT. Even better to remove any requirement for standardized tests wholesale – and that includes the PSSA.
The ESSA allows states a lot of leeway about how and what accountability system they use. There is no need to worry about some imperial federal power invading Pennsylvania to force our hand with standardized testing. We should call their bluff on this. I’ll bet that if we did so, many other states would do the same.
Standardized testing is another failed education policy. Our legislators would do good to read this report and make up their own minds about it.
Though a few years and disasters have happened since its publication, it doesn’t deserve to be forgotten by the very people who ordered it to be written in the first place.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
If you’ve stumbled across this article on social media, congratulations!
You’re one of the few people allowed to read it!
This blog, Gadfly on the Wall, used to be read by at least half a million people each year. Now it’s seen by barely 100,000.
The reason? Poor writing? Flagging interest in the subject?
I don’t think so.
Education is still as important today as it was in 2014 when I started this venture. And as to my writing ability, it’s no worse now than it was 8 years ago.
So this year, my blog had the fewest hits since I started – 124,984 in 2022. By comparison, last year I had 222,414.
I’d write an article, post it on social media and see it reposted again and again. You’d think that would mean it was popular, but no. The people who saw it liked it enough to suggest it to others, but it went little further. With each share, fewer people saw it. Like someone put up a wall in front of it.
In truth, I’m lucky as many people had the opportunity to read my work as did.
The question is where do I go from here?
Should I continue, knowing only a select few will get to hear me? Should I try paying the billionaire tech bros to let more readers in?
My work isn’t a product and no one is paying me to do it.
Oh well…
In any case, here’s a look back at my most popular articles from the year that was and one honorable mention:
Description: My school’s football team is mostly black. They played a mostly white football team and were greeted by racial slurs and an allegedly intentional injury to one of our players. However, the Western Pennsylvania Interscholastic Athletic League (WPIAL) blamed both sides for the incident.
Fun Fact: It’s one of those decidedly local stories that community newspapers used to cover before almost all went bankrupt or were sold to the media giants. Having this platform allowed me to call out an injustice when most voices were silenced. The injured player’s mother thanked me for doing so. Stories like this keep me going.
Description: At the beginning of the summer, governments were so shaken by the exodus of teachers from the classroom that they were discussing raising our salaries or giving us bonuses. Parents were so adamantly against distance learning they demanding in-person classes with real, live human teachers. What a shock to the super elite education “experts” who had been pushing ways to eliminate teachers for decades and ignoring our consistent march out of the field under these conditions.
Description: Charter schools are inequitable because they have charters. These are special agreements that they don’t have to follow all the rules other authentic public schools funded by tax dollars must follow. That’s unfair and it applies to EVERY charter school because every one has a charter. Hence, the name.
Fun Fact: Criticism of charter schools in general usually degrades to defense of individual charter schools avoiding whatever general criticism is leveled against the industry. The argument in this article has the benefit of avoiding any such evasion. All charter schools are guilty of this (and many are guilty of much more). All of them.
Description: Just a list of many things classroom teachers know about schools and education but that the general public often ignores. These are the kinds of things missing from the education debate because we rarely include teachers in the discussion about the field where they are the experts.
Fun Fact: For a few hours people were talking about this article far and wide. And then – boom – it got shut down with a bang. This one was so universal it should have been popular for weeks. But it just disappeared.
Description: Charter schools are colonial enterprises. They loot and pillage the local tax base but without having to be governed by school boards made up of community members – otherwise known as local taxpayers. They can be run by appointed boards often made up of people who do not come from the community in question. They are outsiders come merely for personal profit. These invaders are quite literally taking local, community resources and liquidating them for their own use – the maximization of personal profit. The public is removed from the decision-making process about how its own resources are utilized and/or spent.
Fun Fact: It’s an argument from consistency. If we’re against the colonial enterprise, we must be against charter schools, too. I’m particularly proud of the graphic (above) I created to go with this article.
Description: Dr. Mark Holtzman, the Superintendent from the district where I live, left under strange circumstances. He resigned and took a new contract in a matter of hours so he could get a raise from a lame duck school board without having to wait for the people the community elected to decide the matter to take office first. Then when it all came to light, he left the district for greener pastures.
Fun Fact: More than any other news source, I documented what happened in detail. Without a series of articles I wrote on this, most people would have had very little idea what happened. It would have just been rumors. This is why we need local journalism. It shouldn’t be left to bloggers like me.
Description: This was social media’s latest crackdown on edu-bloggers and other truth tellers. I used to get 1,000 readers a week. Now I’m lucky to get a few hundred. There’s a strict algorithm that determines what people get to see on their Facebook pages. And if it says you’re invisible, then POOF! You’re gone and the people who would most enjoy your writing and want to pass it on don’t get the chance. It’s undemocratic in the extreme but totally legal because Facebook is a for-profit company, not a public service. Money wins over free exchange of ideas.
Fun Fact: There used to be so many other education bloggers like me out there. Now there are just a handful. This is why.
Description: Standardized tests were supposed to improve our public schools. They were supposed to ensure all students were getting the proper resources. They were supposed to ensure all teachers were doing their best for their students. But after more than four decades, these assessments have not fulfilled a single one of these promises. In fact, all they’ve done is make things worse at public schools while creating a lucrative market for testing companies and school privatization concerns.
Fun Fact: Pundits still talk about standardized testing as if it were innovative. It’s not. It’s the status quo. Time to end this failed experiment.
Description: Let’s examine some charter school propaganda – one piece at a time – and see if there’s any truth to these marketing claims. Charter schools are actually not public schools in the same way as other taxpayer funded schools. They do not save money – they waste it. Their students do not outperform authentic public school students. They are not innovative – they are regressive. They do not protect children’s civil rights – they violate them.
Fun Fact: I designed the title and picture to trick readers into thinking this was a pro-charter school article. So many people were butt hurt when they read it! I just hope it helped clarify the matter to those who were undecided.
Description: The Measures of Academic Progress (MAP) test is an assessment made by Northwest Evaluation Association (NWEA), a so-called non-profit organization out of Portland, Oregon. Some states require the MAP as part of their standardized testing machinery. However, in my home state of Pennsylvania, the MAP is used as a pre-test or practice assessment by districts that elect to pay for it. What a waste! Why do we need a test BEFORE the test? The assessment’s job is to show how our students are doing in Reading, Math and Science compared with an average test taker. How does that help? I don’t teach average test takers. I teach human beings. Students learn at their own rates – sometimes faster, sometimes slower. We don’t quicken the timescale with needless comparisons.
Fun Fact: I think this article was as popular as it was because people could relate. So many teachers told me how relieved they were to hear someone else expressing all the frustrations they were experiencing in their own districts with the MAP and other tests like it. If administrators and school boards would just listen to teachers! If they’d even bother asking them!
Description: When it comes to dumb ideas that just won’t go away, there is a special place in the underworld for the demand that teachers post their learning objectives prominently in the classroom. It presupposes that teachers control everything their students learn in the classroom and can offer it to them on a silver platter. It’s not just a useless waste of time but a dangerous misunderstanding of what actually happens in the learning process.
Fun Fact: This isn’t exactly news, but teachers were relieved to hear their truth finally given voice. So many of us still have to abide by this nonsense when we could be doing something that actually makes a difference. It’s nice to have your sanity and frustration confirmed. If only administrators could admit they were wrong and stop demanding this crap!
Gadfly’s Other Year End Round Ups
This wasn’t the first year I’ve done a countdown of the year’s greatest hits. I usually write one counting down my most popular articles and one listing articles that I thought deserved a second look. Here are all my end of the year articles since I began my blog in 2014:
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
This almost always concludes with two types of plan.
First, there is the serious venture made up of things like increasing spending to meet student need, wraparound services, early intervention, reducing class size, redistributive justice and cultural competence – a plan that looks the reality in the face and makes bold attempts to come to terms with it.
Then there is the cheap knockoff proposition – a buzzword-laden scheme where someone is trying to convince you their half hearted proposal is actually a solution to the very real problem of educational inequality.
And the number one thing you can use to tell the difference between the two is this – standardized test scores.
The first plan that is centered around actually fixing disparities makes no mention of test scores – or at least relegates them to obstacles. The second is built all around them – as an essential component of the overall scheme.
This is because the second feel-good-accomplish-nothing plan is essentially performative.
Therefore, it is constructed around standardized test scores as a metric of success.
Planners think: We’re going to do A, B and C to make our schools more equitable. And how will we know we’re doing it right? We’ll use our standardized test scores!
And anyone who understands the history of these types of assessments and how they still work will know that this mirage is built at the cost of genuine equity.
So constructing your plan to fix this problem around one of its root causes is like claiming you can fix a sinking ship by drilling more holes in its hull.
At best, it’s naive. At worst, it’s self-defeating and disingenuous.
Both types of assessment are supposed to measure what students have learned. But not all learning is equal.
For example, a beginning chef needs to know how to use the stove, have good knife skills and how to chop an onion. But if you give her a standardized test, it instead might focus on how to make foie gras – something that would only come in handy at a high end French restaurant.
The fact of the matter is that standardized tests do NOT necessarily focus on the most important aspects of a given task. They focus on obscurities – things that most students don’t know.
This is implicit in the design of these exams and is very different from the kinds of tests designed by classroom teachers.
When a teacher makes a test for her students, she’s focused on the individuals in her classes. She asks primarily about the most essential aspects of the subject and in such a way that her students will best understand. There may be a few obscure questions, but the focus is on whether the test takers have learned the material or not.
When psychometricians design a standardized test, on the other hand, they aren’t centered on the student. They aren’t trying to find out if the test taker knows the most important facts or has the most essential skills in each field. Instead, there is a tendency to eliminate the most important test questions so that the test – not the student – will be better equipped to make comparisons between students based on a small set of questions. After all, a standardize test isn’t designed for a few classes – it is one size fits all.
New questions are field tested. They are placed randomly on an active test but don’t count toward the final score. Test takers aren’t told which questions they’ll be graded on and which are just practice questions being tried out on students for the first time. So students presumably give their best effort to both types. Then when the test is scored, the results of the field test questions determine if they’ll be used again as graded questions on a subsequent test.
If 40 to 60 percent of test takers answer the question correctly, it might make it onto the test. But questions that are answered correctly by 80 percent or more of test takers are usually jettisoned.
“As a consequence of the quest for score variance in a standardized achievement test, items on which students perform well are often excluded. However, items on which students perform well often cover the content that, because of its importance, teachers stress. Thus, the better the job that teachers do in teaching important knowledge and/or skills, the less likely it is that there will be items on a standardized achievement test measuring such knowledge and/or skills.”
If we are guided in large part by standardized test scores, we aren’t guided by authentic learning. We’re guided by a false picture of learning. Therefore, the most effective way – perhaps the only practical way – of raising test scores is to teach directly to a specific test. And not only the test, but the specific version of the test being given that year.
So if we do somehow manage to raise test scores, we haven’t improved academics at all but a mere semblance of it. And thus the equity we might celebrate in such a situation would be just as false.
You got a good score on the MAP test. Hurrah! But that doesn’t mean you know anything of real value except how to take this particular MAP test which, itself, will change after the next round of questions are field tested.
And if we insist on evaluating the equity of our schools on these test scores, we will only make things that much worse.
We end up chasing the psychometricians. We try to guess which aspects of a subject they think most students don’t know and then we teach our students that to the exclusion of more important information. And since what students don’t know changes, we end up having to change our instructional focus every few years based on the few bread crumbs surreptitiously left for us by the state and the testing corporations.
That is not a good way to teach someone anything. It’s like teaching your child how to ride a bike based on what the neighbor kid doesn’t know.
It’s an endless game of catch up that only benefits the testing industry because they cash in at every level. They get paid to give the tests, to grade the tests and when students fail, they get paid to sell us this year’s remediation material before kids take the test again, and – you guessed it – the testing companies get another check!
It’s a dangerous feedback loop, a cycle that promotes artificially prized snippets of knowledge over constructive wholes.
But we can no longer continue feeding that beast and pretending that we can somehow provide equity to our underserved children, too.
We have to choose – equity or testing. Fairness or unrestrained capitalism.
Do not believe anyone who tells you to support a plan built on both.
It does not exist.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
These assessments were based on explicitly eugenicist foundations – the idea that certain races were distinctly superior to others.
Colleague Lewis Terman made the goal clear in his book, The Measurement of Intelligence, that these “experimental” tests will show “enormously significant racial differences in general intelligence, differences which cannot be wiped out by any scheme of mental culture.”
In 1923, another psychologist, Carl Brigham, took these ideas further in his seminal workA Study of American Intelligence. In it, he used data gathered from these IQ tests to argue the following:
“The decline of American intelligence will be more rapid than the decline of the intelligence of European national groups, owing to the presence here of the negro. These are the plain, if somewhat ugly, facts that our study shows. The deterioration of American intelligence is not inevitable, however, if public action can be aroused to prevent it.”
The practice was even upheld by the US Supreme Court in the 1927 Buck v. Bell decision. Justices decided that mandatory sterilization of “feeble-minded” individuals was, in fact, Constitutional.
Of the ruling, which has never been explicitly overturned, Justice Oliver Wendell Holmes wrote, “It is better for all the world, if instead of waiting to execute degenerate offspring for crime, or to let them starve for their imbecility, society can prevent those who are manifestly unfit from continuing their kind…. Three generations of imbeciles are enough.”
Eventually Brigham took his experience with Army IQ tests to create a new assessment for the College Board – the Scholastic Aptitude Test – now known as the Scholastic Assessment Test or SAT. It was first given to high school students in 1926 as a gatekeeper. Just as the Army intelligence tests were designed to distinguish the superior from the inferior, the SAT was designed to predict which students would do well in college and which would not. It was meant to show which students should be given the chance at a higher education and which should be left behind.
And unsurprisingly it has always – and continues to – privilege white students over children of color. The same as nearly every standardized test still does.
HAS IT CHANGED?
None of this can be challenged. These are historical facts. They are simply what happened justified in the words of the people who perpetrated them.
In 1964, a Department of Education report found that the average black high school senior scored below 87% of white seniors (in the 13 percentile) on standardized assessments. Fifty years later, the National Assessment of Educational Progress (NAEP) found that black seniors had narrowed the gap until they were merely behind 81% of white seniors (scoring in the 19th percentile).
Is that really the kind of progress you want to champion?
The reason for the disparity has nothing to do with the learning students of color (and the poor whose scores are similar) achieve nor their worth as human beings.
Discrimination is purposefully built in to the standardization process, according to W. James Popham, PhD, Professor Emeritus at the University of California at Los Angeles and former test maker. He explained in an interview with Frontline:
“Traditionally constructed standardized achievements, the kinds that we’ve used in this country for a long while, are intended chiefly to discriminate among students … to say that someone was in the 83rd percentile and someone is at 43rd percentile. And the reason you do that is so you can make judgments among these kids. But in order to do so, you have to make sure that the test has in fact a spread of scores. One of the ways to have that test create a spread of scores is to limit items in the test to socioeconomic variables, because socioeconomic status is a nicely spread out distribution, and that distribution does in fact spread kids’ scores out on a test.”
The scores have to fall into categories – Below Basic, Basic, Proficient and Advanced, for instance. If too many students cluster in the middle, the results are invalid. We need the scores spread out – even if we must resort to non-educational factors to get there.
Family income is not something the tests ignore, says Popham. It is an essential component specifically tested for in question construction. In fact, he claims that between 15-80% of the questions (depending on the subject area) on norm-referenced exams are linked to socio-economic status (SES).
Thus minorities with higher percentages of impoverished people are selected against. Not because of any explicit racist ideology – but to get the pretty bell curve standardized assessments require.
“Too often, test designers rely on questions which assume background knowledge more often held by White, middle-class students. It’s not just that the designers have unconscious racial bias; the standardized testing industry depends on these kinds of biased questions in order to create a wide range of scores.”
For example, Choi recalled a 10th grade student in his class asking him about a standardized test question. “With a puzzled look, she pointed to the prompt asking students to write about the qualities of someone who would deserve a “key to the city.” Many of my students, nearly all of whom qualified for free and reduced lunch, were not familiar with the idea of a ‘key to the city.’”
So when they get such a question wrong, it isn’t necessarily because they don’t know the concept being tested, but they don’t understand what was being asked in the first place.
“Compare two 1998 SAT verbal [section] sentence-completion items with similar themes: The item correctly answered by more blacks than whites was discarded by the Educational Testing Service, whereas the item that has a higher disparate impact against blacks became part of the actual SAT. On one of the items, which was of medium difficulty, 62% of whites and 38% of African-Americans answered correctly, resulting in a large impact of 24%…On this second item, 8% more African-Americans than whites answered correctly…”
In other words, the criteria for whether a question is chosen for future tests is if it replicates the outcomes of previous exams – specifically tests where students of color score lower than white children. And this is still the criteria test makers use to determine which questions to use on future editions of nearly every assessment in wide use in the US.
We no longer profess eugenicist ideas of racial purity embedded in our assessments as self evident or based on science. But they’re there none-the-less.
Today, the very concept of intelligence being quantifiable remains in question.
In “The Mismeasurement of Man,” evolutionary biologist Stephen Jay Gould challenged many of these ideas – in particular those of Terman. Rather than see intelligence as a genetic trait, Gould envisioned it more abstractly and thus shattered the idea that any mere number could capture human value.
We can’t just give a faulty bridge a new coat of paint. We must demolish it and rebuild an entirely new structure to carry us into the future.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
He takes notes in class, does all his homework and participates in discussions.
He writes insightful essays and demonstrates a mastery of spelling and grammar.
He reads aloud with fluency and inflection. He asks deep questions about the literature and aces nearly all of his classroom reading comprehension tests.
But after reluctantly subjecting my classes to the MAP and being instructed to analyze the results with my colleagues, we noticed this contradiction.
In many cases, scores did not match up with teacher expectations for our students.
In about 60-80% of cases, students who had demonstrated high skills in the subject were given scores below the 50th percentile – many below the 25th percentile.
These were kids with average to high grades who the MAP scored as if they were in the bottom half of their peers across the state.
Heck! A third of my students are in the advanced class this year – but the MAP test would tell me most of them need remediation!
If we look at that data dispassionately, there are possible explanations. For one, students may not have taken the test seriously.
And to some degree this is certainly the case. The MAP times student responses and when they are input fast and furious, it stops the test taker until the teacher can unlock the test after warning them against rapid guessing.
However, the sheer number of mislabeled students is far too great to be accounted for in this way. Maybe five of my students got the slow down sloth graphic. Yet so many more were mislabeled as failures despite strong classroom academics.
Nice for them. However, I am not some rube reading this in the paper. I am not examining some spreadsheet for which I have no other data. I am IN the classroom every day observing these very same kids. I’ve been right there for almost an entire grading period of lessons and assessments – formative and summative. I have many strong indications of what these kids can do, what they know and what they don’t know.
I am a Nationally Board Certified Teacher with more than two decades experience. But Northwest Evaluation Association (NWEA), a testing company out of Portland, Oregon, wants me to believe that after 90 minutes it knows my students better than I do after six weeks!
Time to admit the MAP is a faulty product.
But it’s not just that one standardized test. We find the same disparity with the PSSA and other like assessments.
Nationally, classroom grades are better than these test scores.
By contrast, teacher-created tests are just the opposite. They are designed almost exclusively to assess whether learning has taken place and to what degree. Comparability isn’t really something we do. That’s the province of administrators and other support staff.
The primary job of teaching is just that – the transfer of knowledge, offering opportunities and a conducive environment for students to learn.
You can’t give a person a blood transfusion if you can’t accurately measure how much blood you’re giving her. And comparing how much blood was given to a national average of transfusions is not helpful.
You need to know how much THIS PERSON needs. You need to know what would help her particular needs.
When good students get bad test scores, it invariably means you have a bad test.
Until society accepts this obvious truth, we will never be able to provide our students with the education they deserve.
Good students will continue to be mislabeled for the sake of a standardized testing industry that is too big to fail.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
To answer that question, we have to understand how we got here in the first place – where these kinds of assessments came from in the US and how they became the guiding policy of our public schools.
From the very start, it had serious consequences for public policy. The results were used to rationalize the forced sterilization of 60,000 to 70,000 people from groups with low test scores, thus preventing them from “polluting” the gene pool.
However, Brigham’s greatest claim to fame was the creation of the Scholastic Assessment Test (SAT) to keep such undesirables out of higher education. These tests were not central to school curriculum and mainly used as gatekeepers with the SAT in particular still in wide use today.
The problem then – as now – is that standardized tests aren’t very good assessments. They work okay for really simple things like rudimentary math. However, the more complex a skill you’re assessing, the more inadequate the tests. For example, imagine just trying to have a conversation with someone where your only choices of reply were limited to four canned responses. That’s a multiple-choice assessment. The result is a testing system that selects against the poor and minorities. At best, it reproduces the economic and racial disparities of society. At worst, it ensures those disparities will continue into the next generation.
That isn’t to say the system went unchallenged. By the 1960s, the junk science and leaps of logic behind standardized testing were obvious and people began fighting back in court. Black plaintiffs began winning innumerable lawsuits against the testing industry.
Perhaps the most famous case is Hobson v. Hansen in 1967, which was filed on behalf of a group of Black students in Washington, DC. The court ruled that the policy of using tests to assign students to tracks was racially biased because the tests were standardized to a White, middleclass group.
So in the 1980s, the Reagan administration published “A Nation at Risk,” a campfire tale about how America’s public schools were failing. Thus, the authors argued we needed standardized testing to make American children competitive in a global marketplace.
For instance, it concluded that average student test scores had decreased but didn’t take into account that scores had actually increased in every demographic group. It compared two decades worth of test scores, but failed to mention that more students took the test at the end of that period than at the beginning, and many of the newer students were disadvantaged. In other words, it compared test scores between an unrepresentative group at the beginning of the comparison with a more representative group at the end and concluded that these oranges were nothing like the apples they started with. Well, duh.
Most people weren’t convinced by the disaster capitalism at work here, but the report marks a significant moment in the standardization movement. In fact, this is really where our modern era began.
It took an additional two decades, until 2001, for President George W. Bush’s No Child Left Behind (NCLB) legislation to require standardized testing in ALL public schools.
With bipartisan support, Bush tied federal funding of schools to standardized test performance and annual academic progress. And from then on, the die was cast. This policy has been upheld through both Republican and Democratic regimes.
From the 1980s to 2022 we’ve had wide scale standardized testing in our schools. That’s roughly 40 years where the entirety of what is done in public school has been organized around these assessments. They drive the curriculum and are the ultimate benchmark by which success or failure is judged. If this policy was ever going to work, it would have done so by now.
However, it has achieved NONE of its stated goals.
NCLB specifically stated that all children would be proficient in reading and math by 2014. That has not happened. Despite spending billions of dollars on remediation and completely reorganizing our schools around the assessments, test scores have remained mostly static or even decreased.
NCLB also championed the idea that competing for test scores would result in better teachers. However, that didn’t happen either. Instead, educators were forced to narrow the curriculum to cover mostly what was assessed, reduce creativity and critical thinking, and teachers who served poor and minority students were even punished for doing so.
If the purpose of standardized testing was all the things the law purported, then it was a decades long failure. It is the policy equivalent of slamming your head into a wall repeatedly and wondering why you aren’t moving forward. (And where did this headache come from?)
If, however, the purpose of standardized testing was to fulfill Friedman’s privatization dreams, then it was a resounding success. Public schools still persist, but they have been drained, weakened and in many ways subverted.
Look at the evidence.
Standardized testing has grown from a $423 million industry before 2001 to a multi-billion dollar one today. If we add in test prep, new text books, software, and consultancy, that figure easily tops the trillion dollar mark.
Huge corporations make the tests, grade the tests and then sell remediation materials when students fail. It’s a huge scam.
But that’s not the only business created by this policy. Test and punish opened entirely new markets that hadn’t existed before. The emphasis on test scores and the “failing schools” narrative stoked unwarranted distrust in the public school system and a demand for more privatized alternatives.
Chief among these was charter schools.
The first charter school law was passed in 1991 in Minnesota. It allowed for the creation of new schools that would have special agreements (or charters) with states or districts to run without having to abide by all the usual regulations. Thus, the school could go without an elected board, pocket public money as private profit, etc. The bill was quickly copied and spread to legislatures across the country by the American Legislative Exchange Council (ALEC).
However, charter schools are rife with fraud and malfeasance. For instance, more than a quarter of charter schools close within 5 years of opening. By year 15, roughly 50% of charter schools close. That’s not a stable model of public education. It’s a get rich quick scheme. And since these types of schools are free from the kinds of regulations, democratic governance and/or transparency that keeps authentic public schools in check, another charter school scandal pops up almost every day.
But let’s not forget school vouchers. Before high stakes testing, the idea of using public money to pay for private or parochial schools was widely considered unconstitutional. Now about 4% of US students go to private and parochial schools some of which are funded with school vouchers. This is an option in 32 states and the District of Columbia, and more than 600,000 students participated in a voucher, scholarship tax credit or education savings account program last school year, according to EdChoice, a pro-voucher and school choice group.
In short, they’re subsidies for wealthier kids at the expense of the middle class and disadvantaged.
Without standardized testing, it is impossible to imagine such an increase in privatization.
High stakes testing is a Trojan horse. It is a way to secretly undermine and weaken public schools so that testing corporations, charter schools and voucher schools can thrive.
Judged by its own metrics of success, standardized testing is an abject failure. Judged by the metric of business and school privatization it is a rousing success.
And that’s why it has been so hard to discontinue.
This is corporate welfare at its finest, and the people getting rich off our tax dollars won’t allow us to turn off the flow of funding without a fight.
On the right, policymakers are often boldly honest about their goals to bolster privatization over public schools. On the left, policymakers still cling to the failed measures of success testing has not been able to meet time-and-again.
However, both groups support the same system. They only give different reasons.
It is past time to wake up and smell the flowers.
If we want to ensure education dollars go to education and not profiteers, we need to end standardized testing.
If we want to help students learn to the best of their abilities, we need to stop gaslighting them with faulty measures of success or failure.
If we want to allow teachers to do the best for their students, we need to stop holding them back with antiquated eugenicist shackles.
And if we truly want to save our public school system, we have to stop propping up privatization.
In short, we need to end standardized testing.
The sooner, the better.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
Students as young as Kindergarten (and sometimes younger) are asked to read a text aloud in a given time and each mispronunciation is recorded and marked against them.
To my knowledge none are actually used on the DIBELS test but they give you an idea of what an adult version might be like if given to people our age and not just the littles.
Can you imagine being a child, feeling the pressure of a test and being presented with something that looked like those passages!?
The fear! The sense of urgency to say something before the time runs out! The feeling of inadequacy and confusion as you finished knowing you got it wrong!
Today I was forced to leave my class of 8th grade students with a sub so an “expert”from the Allegheny Intermediate Unit could lecture me and the rest of my school’s English department in using DIBELS as a gatekeeper assessment for all students.
That way we can group the students more easily based on their reading deficiencies.
I literally had to stop teaching for THAT.
I was bopping around the classroom, reading students’ writing, helping them organize it, helping them fix their explanations and craft sophisticated essays on a short story.
And the rest of the department with similar experience and education. In the group was also the holder of a doctorate in education. Almost all of us at least held a masters degree.
However, you’d need a classroom teacher to explain that to you. And these are more business types. Administrators and number crunchers who may have stood in front of a classroom a long time ago but escaped at the first opportunity.
They look at a class full of students and don’t see human children. They see numbers, data.
Unfortunately there’s a whole world of reality up here above ground that they’re ignoring. And up here continuing with their willful fantasy is doing real harm.
When I look at my classes of students, I don’t see overwhelming academic deficiencies.
Even their test scores don’t justify that myth.
According to the Pennsylvania System of School Assessments (PSSA), they’re pretty much where I’d expect them any other year.
My students are desperate for attention – any kind of attention – and will do almost anything to get it.
They’d prefer to be respected, but they don’t understand how to treat each other respectfully. So they aim for any kind of response.
To a large extent this is due to a disruption in the social and emotional development they would have received at school. But robbed of good role models and adequate consequences, they’re somewhat at sea.
We have to recognize the reality teachers, students and parents are living through.
And we have to make decisions based on that reality, not the same old preconceptions that have never gotten us anywhere.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
Schools could barely keep their doors open and conduct in-person classes. Many educators were still teaching their students on-line or both on-line and in-person at the same time. Hundreds of teachers died from the virus. Thousands of students have lost parents, relatives or became sick, themselves.
And if increasing cases of the even more contagious Delta Variant continue to spread in 2021-22 while the last 30% of American adults are reluctant to get the vaccine, the situation could be even worse this spring.
In their paper, Bruno and Goldhaber looked at last year’s waiver requests asking for permission to cancel or modify statewide exams in 11 states and the District of Columbia.
“We speculate that there was concern that even temporarily waiving statewide tests would give momentum to those advocating for the elimination of testing all together. That is, [the US Department of Education] USDOE (and perhaps states that did not request that common assessments be waived) may be less interested in what happens with testing this year than worried about a slippery slope toward increasingly lax testing requirements.” [Emphasis mine]
Speaking of which, the report then becomes focused on advice for standardized testing advocates to combat mounting pressure to end these mandated federal assessments.
If the public doesn’t see the value in the tests, Bruno and Goldhaber say, policymakers must explain why the tests are important, and not just in generalities. They must explicitly show how standardized test scores improve education and help specific students.
“We encourage policymakers to think carefully, explicitly, and publicly about how they have tailored their standardized testing policies to achieve various diagnostic, research, and accountability objectives. This will help to ensure that standardized tests have benefits for more schools and students and will bolster fragile political support for statewide tests.”
However, nowhere in the entire paper do Bruno and Goldhaber actually do this, themselves.
How do standardized tests help students?
That’s exactly the question at stake here.
In short, I would argue as I have countless times before that they DO NOT help students.
If the authors had good counterarguments, now would have been a good time.
The authors do say that standardized test scores are predictive of latter student outcomes but they ignore whether other assessments or factors are MORE predictive.
Yes, students with high test scores often graduate, excel in college or trade schools, etc. However, the same can be said with classroom grades. In fact, classroom grades are evenmore accurate.
This just makes sense. Classroom grades are based on at least 180 days of formal and informal assessment. Standardized tests are merely a snapshot of a few days work.
However, even more predictive is child poverty. The rich kids usually do much better than the poor kids. Same with race, class and the funding each student receives at his or her school.
If you want to help students, that’s where you need to begin – equitable resource allocation. Make sure all students have what they need to succeed, and realize that the more poverty you have, the greater the need, the greater the resources necessary.
Standardized tests literally were invented to justify bias. They were designed to prove that higher income, higher class, white people were entitled to more than poorer, lower class, brown people. Any defense of the assessments today must explain how the contemporary variety escapes the essential racist assumptions the entire project is based on.
And there is no better image to prove that than forcing kids to take a meaningless test during a global pandemic.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
The Covid-19 pandemic closed school rooms across the nation for various lengths of time. Some are still closed. Some are beginning to close again.
Many classes were conducted remotely through conferencing software like Zoom and file sharing programs like Google Classroom. Others were conducted through a hybrid model combining in-person instruction and cyber instruction. While still others met in-person with numerous mitigation efforts like masks, social distancing and air purifiers.
Many students were absent, struggled to learn and experienced countless traumas due to the isolation, sickness and deaths.
That’s more than Americans who died in the attack on Pear Harbor (2,403), the 9/11 terrorists attacks (3,000), WWI (116,000) or WWII (405,000).
Only the Civil War (600,000 – 850,000) has a larger death toll. For now.
As of April 1, nearly 3.47 million children have tested positive for COVID-19, most with mild symptoms, according to the American Academy of Pediatrics. A few hundred have died, mostly children of color. Many more kids probably contracted the virus but were asymptomatic spreaders of the disease to adults.
As a result, between 37,000 and 43,000 children in the United States have lost at least one parent to COVID-19, according to USC research.
How do you sort through all these tragedies and traumas and say THIS was caused by a lack of standardized testing?
But in 2001 we created an industry. Huge corporations write the tests, grade the tests and provide the remediation for the tests. Billions of dollars in taxes are funneled into this captive market which creates monetary incentives for our lawmakers to keep the system going.
Yes, some civil rights organizations have waffled back and forth over this as big donors who value the tests make or withhold contributions. Meanwhile, many other more grassroots civil rights organizations such as Journey for Justice Alliance (JJA), a group made up of 38 organizations of Black and Brown parents and students in 23 states, have continuously called for the abolition of high stakes testing.
A scientific mind would be empirical about this. It would examine the results as much as possible and determine whether moving forward made any sense.
This is especially true as the pandemic health crisis continues to make the act of giving the tests difficult at best and dangerous at worst.
There is no way a logical mind can look at the situation and not come to the conclusion that the status quo on testing is a triumph of capitalism over science and reason.
In a month or so, the year without testing will be just that – a single year.
We shall go on to the end. We shall test during Covid, we shall test in the classes and on-line, we shall test with growing confidence and growing strength wearing masks, we shall defend our industry, whatever the cost may be. We shall test in the homes, we shall fill in bubbles on sanitized desks, we shall test in the fields and in the streets, we shall test in the hospitals; we shall never surrender!
Do you live in Allegheny County, Pennsylvania? I’m running for County Council in District 9
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
It’s absurd, like paramedics arriving at a car crash, finding one person in a pool of blood and another completely unscathed – but before they know which person needs first aid, they have to take everyone’s blood pressure.
If the only logical way to make decisions is based on test scores, you should provide those scores to the greatest decision-making body in the country: voters.
After all, student tests are designed to favor answers from privileged white people. Most of these lawmakers are the target demographic already. They passed a standardized test (or paid someone to pass the test for them) as a smokescreen getting into whichever prep school or ivy league college where they were legacy enrollments, anyway.
I’m talking about a new series of standardized tests designed to show how much these lawmakers adhere to the principles of their respective political parties.
So there’d be two versions – one for Republicans and one for Democrats.
A high score means the test taker is a bona fide example of their party’s ideals. A low score means they should probably be booted out on their butts.
For example, a question for Democrats might be:
Which policy is progressive?
A) School privatization B) Fracking on native lands C) Drone strikes D) Universal healthcare
And an example for Republicans:
Which policy is fiscally responsible?
A) School privatization B) Tax cuts for billionaires C) More unnecessary wars D) Investing in infrastructure
The answers are both D and that’s because this test would be in high De-mand! Get it?
They could say, “Vote for Sam Smith. He got an Advanced Score on the Democratic System of Statesperson Assessments (DSSA).”
Or “Don’t vote for Megan Mission. She only scored a Satisfactory on the Partnership for Assessment of Republicanism for Congress or Klan (PARCK).”
What an improvement that would be!
Finally, we wouldn’t have to rely on a politician’s voting record or campaign contributions or platform…. We could just look at the score and vote accordingly.
But who would we get to make and grade the tests?
It couldn’t be the politicians, themselves, or even their respective political parties. That wouldn’t be standardized somehow.
If we can’t let teachers create tests for their own students, we certainly can’t trust politicians to do the same for their fellow campaigners.
I guess we could task the testing corporations with making these assessments, but that’s a conflict of interests. We should instead rely on the educational experts, people with the credentials and the most experience actually giving standardized tests.
And since the teachers would get to grade the assessments, they could make sure the scores are curved so only a very limited number pass each year. We can’t have grade inflation, after all.
What would the teachers do with this money, I wonder?
And all of this just because of standardized testing!
Maybe the lawmakers have the right idea in prioritizing high stakes testing!
Or maybe they understand the value of benefiting from the testing industrial complex and not being subjected to it.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
gadflyonthewallblog 