The majority of teachers and principals in Pennsylvania hate standardized tests.
An increasing number of parents are refusing to allow their kids to take the tests.
And there may be better alternatives to the state’s Keystone Exams.
These were just some of the key findings of a blockbuster report from June 2019 by the state General Assembly’s Legislative Budget and Finance Committee.
However, now that things are returning to some semblance of normalcy, it seems that bureaucrats from the state Department of Education (PDE) are taking the wrong lessons from the report while the legislature seems to have forgotten it entirely.
The report was conducted because of legislation written by state Sen. Ryan P. Aument (R-Lancaster County). It directed the Legislative Budget and Finance Committee to “study the effectiveness of standardized testing, including the Keystone Exams and SATs, and their use as indicators of student academic achievement, school building performance, and educator effectiveness.”
The key findings are as follows:
1)The majority of principals and teachers disapprove of the state’s standardized tests – both the Pennsylvania System of School Assessment (PSSA) tests given in grades 3-8 and Keystone Exams given in high school. They think these tests are ineffective, expensive and harmful to district curriculum and students.
2) State law allows parents to opt their children out of testing for one reason only – religious grounds. Parents are using this religious exemption in increasing numbers. This puts districts in danger of violating federal participation and accountability standards.
3) It has been suggested that the state allow two additional reasons for parents to opt their children out of testing – philosophical grounds and health concerns. It is unclear whether doing so would increase overall opt outs or not.
4) The federal Every Student Succeeds Act (ESSA) passed in 2015 allows the use of the SAT and/or ACT test to take the place of high school standardized testing. It has been suggested the Commonwealth replace the Keystone Exams with these tests. The report finds the ACT and/or SAT would successfully determine college readiness and reduce the overall amount of standardized testing. However, this would not allow other uses of current state tests like evaluating teacher effectiveness and school building performance.This may not matter though because the report also casts doubt on whether the current tests (PSSA and Keystone Exams) do an adequate job of assessing teacher or building performance now or even if student tests can be accurately used to evaluate teachers and schools.
There’s a lot of information here. Let’s look at each finding in turn.
1) PA Educators Hate Standardized Tests
When it comes to the PSSAs, 67% of principals and 76% of teachers said the tests were ineffective indicators of student achievement.
There was slightly more support for the Keystone Exams. This time 45% of principals said the test was an ineffective indicator of student achievement (with 27% saying the tests were effective). Meanwhile, 60% of teachers said the test was ineffective.
Both principals and teachers said their curriculum had been narrowed to prepare students for PSSAs and Keystone Exams. Instead of going into more depth on regular classwork or learning new skills, the focus shifts to teaching to the tests.
Most principals (approximately 80-90%) said that students are taught test-taking skills, and their schools administered practice tests, bench-mark tests, and/or diagnostic tests to prepare students for the PSSA exams. This held for teachers, too, with 81-88% saying they teach test-taking skills and administer practice tests. Principals also said the costs of this additional test prep varied from $200 to more than $100,000.
Taking the tests also eats up valuable class time. Administering the assessments takes between 5.7 to 8 days for each kind of test – the PSSA and the Keystone Exams, according to Principals.
In addition, the report details the cost of giving these tests. In fiscal year 2017-2018, PDE paid $42.17 million for these tests. This is part of a national trend:
“Standardized tests and test preparation have subsequently become big business and that multibillion dollar business continued to grow since the enactment of NCLB and the subsequent enactment of ESSA. According to the Pew Center on the States, annual state spending on standardized tests increased from $423 million before the NCLB (enacted in 2002) to upwards of $1.1 billion in 2008 (to put this in perspective this reflects a 160 percent increase compared to a 19.22 percent increase in inflation during the same time period). A more recent study by the Brown Center on Education Policy at Brooking put the cost at upwards of $1.7 billion in 2011 related to state spending on standardized tests.”
2) Opt Outs on the Rise
Many states allow parents to opt their children out of standardized testing. Some do so in cases of a physical disability, medical reasons, or emergencies. A few allow opt-outs based on religious objection – like Pennsylvania. Some states allow opt-outs for any reason whatsoever.
The religious exemption is not used widespread throughout the state and most schools meet the 95% participation rate required by the federal government. However, use of the religious exemption is definitely on the rise – enough so that the authors of the report find it alarming:
“Meanwhile, as previously indicated in this section, schools throughout the country are experiencing and grappling with an increase in the number of parents seeking to have their children opt-out of standardized testing now that new state assessments have been implemented pursuant to the federal requirements. Pennsylvania is no exception to this trend and is also experiencing an increase in the number of parents utilizing the religious opt-out.”
For the PSSA tests, opt outs increased from 2013-14 to 2016-17. However, total numbers in school year 2017-18 decreased sightly.
Opt outs went from 1,886 to 6,425 to 15,644 to 19,012 to 16,961.
During the same time period for the Keystone Exams, opt outs steadily increased each year but were at lower overall rates.
For the high school test, opt outs went from 382 to 666 to 1,000 to 1,313 to 1,633.
These are vitally important figures because opt out data is rarely tabulated and released to the public. Many media accounts actually state the opposite of the data in this report – in particular that opting out has decreased since Congress passed the ESSA in 2015. Apparently the media got this one wrong.
Though the religious opt-out is the only reason specifically allowed in state Chapter 4 regulations, PDE reports there are five additional ways that students end up not taking the tests:
1) Other Parental Request – parents simply refusing to let their kids participate but not objecting based on specifically religious reasons.
2) No Attempt and No Exclusion Marked – students who are given the test but do not answer enough questions to receive a score.
3) No Test – no test record on file for unknown reasons.
4) Extended Absence – a student missed the testing and make-up window due to absences.
5) Other – does not fit any of the other categories.
Federal law – in particular No Child Left Behind (NCLB) and subsequent reauthorizations of that legislation – requires states to use student participation in standardized testing as a factor in a state’s accountability system. According to the report, any district with less than 95% of students taking the test should be “addressed.”
The report does not go into any further detail about what this means, other than to say that falling under 95% can:
“…ultimately result in a reduced achievement/proficiency measure… If the student participation rate falls below 95 percent, states are required to calculate student achievement/proficiency by dividing the number of students scoring proficiently by no less than 95 percent of the total students (which effectively assigns a score of “0” to all nonparticipants once the participation rate has fallen below 95 percent).”
In effect, the district gets a bad mark on a piece of paper. So what?
Under NCLB, schools with poor performance could receive sanctions like state takeover or lower funding. However, this is extremely unlikely – especially since the passing of ESSA. This newest reauthorization of the law gives states leeway in designing their accountability systems. It leaves the enforcement of this 95% participation rate up to the states, requiring them to develop an accountability plan in the event that a school or district fails to meet this standard.
So a school would only be punished if the state decided to do so. If a state legislature decided to allow parents to opt their children out for any reason at all, they would not have to take any punitive measures. Since the ESSA, the buck stops at the state house door on this one. California, for example, takes note of low participation rates but these rates are not factored into a school’s rating. On the other hand, Florida mandates direct intervention from the state’s Department of Education until participation rates are met.
3) Impact of New Reasons to Opt Out
This is where things get a bit sticky.
The report mentions the idea of expanding the options for opting-out of statewide assessments (e.g., PSSA and Keystone Exams) to include objections based on philosophical grounds or due to health issues.
On the one hand, the authors write “The impact of adding opt-out categories may be minimal.” They don’t know if more people would use the expanded options or if the same numbers who use the religious exemption today might simply divide themselves up among all three options.
The authors worry, however, that new pathways to opt out may increase the total number of people refusing the tests for their children and would reduce Pennsylvania’s participation rate in standardized testing.
This is a particularly troubling paragraph:
“The existence of opt-outs (religious or otherwise) has the potential to negatively impact a state’s participation rates and may potentially impact a state’s [Local Education Agency (LEAs)] and schools achievement/proficiency rate and ultimately the ability of a state to be in compliance with federally required assessments and accountability measures. Furthermore, providing opt-outs and giving parents notice of such has the potential to conflict with the message about the importance of standardized testing. Ultimately placing the state departments of education and local school districts in the potentially awkward position of having to explain why it is important for students to participate in testing (given the federal requirements), while also giving and notifying parents of the opt-out options for their children. In 2015, US Department of Education sent out letters to a dozen states flagging their low participation rates (statewide, or at the district or subgroup level) on the 2014-15 school year assessments and indicated that they needed to create a plan to reduce opt-outs due to low participation rates.”
This seems to be the order of the day at PDE. It’s why earlier this year, school administrators were advised by state officials to crack down on parents opting their children out of standardized tests.
And all of it is based on a cowardly and incomplete understanding of federal law. If Commonwealth schools fall below 95% participation in the test and get a bad mark on a worthless metric, it doest have to matter. No matter how many letters the federal government sends to the state legislature or PDE, the law is clear. The state is in charge here. Our legislature can choose to side with taxpayers, residents, and citizens or with civil servants and strongly worded letters.
4) Replacing the Keystone Exams
There’s not much more to add to this than the initial finding.
The authors of the report say there would be no problem with replacing the Keystone Exams with the ACT or SAT because these national tests would properly assess students’ college and career readiness.
The report is actually pretty shoddy in this regard not really examining the claims of the College Board which makes both tests. The authors just pretty much accepted the College Board’s word wholesale. Nor was their any evaluation of what teachers and principals thought about these tests like there had been for the PSSA and Keystone Exams.
However, the report does make a good point about test reduction. Many students already take the SAT or ACT test, so eliminating the Keystones would reduce the over all amount of tests they had to take.
Also the authors deserve credit for writing about how using student test scores to evaluate teachers and schools is seriously bad practice.
According to the report, 77 percent of principals and 93 percent of teachers said PSSA tests were not effective indicators for teacher evaluations, with similar figures for the Keystone Exams and building performance evaluations.
While everything in the report may not be 100% accurate, it includes important information that should be wider known.
In addition, the report has trustworthy data about opt outs throughout the Commonwealth. Unlike what has been reported in the media, opting out is not on the decline, it is on the rise.
The Keystone Exams should be thrown in the trash, because that’s what they are – trash. At very least they should be replaced with the SAT or ACT. Even better to remove any requirement for standardized tests wholesale – and that includes the PSSA.
The ESSA allows states a lot of leeway about how and what accountability system they use. There is no need to worry about some imperial federal power invading Pennsylvania to force our hand with standardized testing. We should call their bluff on this. I’ll bet that if we did so, many other states would do the same.
Standardized testing is another failed education policy. Our legislators would do good to read this report and make up their own minds about it.
Though a few years and disasters have happened since its publication, it doesn’t deserve to be forgotten by the very people who ordered it to be written in the first place.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
Let me put that in perspective – a scandal-ridden investment firm that made billions in the wars in Iraq and Afghanistan bought one of standardized testing’s big four and then added the Measures of Academic Progress (MAP) test to its arsenal.
This almost certainly means the cost of state testing is going to increase since the providers of the tests are shrinking.
“It used to be if you put out a [Request for Proposal] RFP for state assessment, you get five, six, 10 bidders,” said Scott Marion, executive director of the Center for Assessment. “Now you’re lucky to get three. When you’re doing that, there’s maybe not as much expertise and certainly the cost will go up” (emphasis mine).
You can just imagine how this will affect the marketplace.
NWEA serves about 10,000 school districts and HMH estimates it works with more than 50 million students and 4 million educators in 150 countries, according to a press release about the proposed acquisition.
So we can expect districts and even entire states which rely heavily on the MAP test to be encouraged to buy as much HMH curriculum as possible. That way they can teach directly what is on their standardized tests.
That is assuming, of course, the acquisition agreement is approved after a 90-day regulatory review period.
To be honest, I would be surprised if there are any objections.
Such cozy relationships already exist with other education companies. For example, Curriculum Associates provides the aforementioned curriculum for its own i-Ready assessment.
After all, if these sorts of assessments can be gamed by increased access to materials created by the same corporate entities that create and grade the tests, are we really assessing knowledge? Aren’t we just giving students a score based on how many books and software packages their districts bought from the parent company? Is that really education?
Then in February of 2022, New York-based private-equity firm Veritas Capital acquired HMH at a price of $21 per share, or about $2.8 Billion. And under Veritas, HMH acquired NWEA and the two companies will work together to do many of the things that HMH used to do by itself – like a golden dragon perched atop the standardized testing treasure trove.
All for the benefit of Veritas Capital.
Make no mistake, the investment firm wouldn’t have become involved if it couldn’t make a profit off the situation. That’s what it does – through scandal after scandal.
Founded in 1992 by the late investment banker Robert McKeon (who died by suicide after mounting improprieties came to light), Veritas Capital began its life buying up government contractors and forming close ties with former senior government officials. Of the company’s many defense-related investments, the most infamous was its 2005 purchase of DynCorp International, a shady company involved in the US’s Iraq and Afghanistan wars.
In 2006, the firm acquired MZM Inc., an intelligence contractor, which was investigated for providing bribes to Rep. Duke Cunningham, R-Calif., in exchange for help obtaining Pentagon contracts.
Throughout its history, Veritas has fostered close ties to government officials. Campaign finance records show executives at the investment firm have given over $100,000 to various politicians, mostly Republicans. In 2014, Veritas paid Bill Clinton $250,000 for a speech.
The New York Times reported in 2001 that numerous retired generals were on Veritas’ payroll and the company used such ties to the Pentagon and frequent appearances in the media to boost Veritas-owned military contractors, including DynCorp.
Musallam focuses on technology companies like HMH that operate in sectors dominated by the US federal government such as standardized testing. After all, the only reason public schools throughout the country have to give these assessments is federal law. It’s a captive market paid for by tax dollars.
America spends $6.8 trillion a year on defense, health care and education – markets dominated by the government.
“These are government-influenced markets, no doubt about it, and being close to how the government thinks about those markets enables us to understand how we can best invest,” Musallam said.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
This almost always concludes with two types of plan.
First, there is the serious venture made up of things like increasing spending to meet student need, wraparound services, early intervention, reducing class size, redistributive justice and cultural competence – a plan that looks the reality in the face and makes bold attempts to come to terms with it.
Then there is the cheap knockoff proposition – a buzzword-laden scheme where someone is trying to convince you their half hearted proposal is actually a solution to the very real problem of educational inequality.
And the number one thing you can use to tell the difference between the two is this – standardized test scores.
The first plan that is centered around actually fixing disparities makes no mention of test scores – or at least relegates them to obstacles. The second is built all around them – as an essential component of the overall scheme.
This is because the second feel-good-accomplish-nothing plan is essentially performative.
Therefore, it is constructed around standardized test scores as a metric of success.
Planners think: We’re going to do A, B and C to make our schools more equitable. And how will we know we’re doing it right? We’ll use our standardized test scores!
And anyone who understands the history of these types of assessments and how they still work will know that this mirage is built at the cost of genuine equity.
So constructing your plan to fix this problem around one of its root causes is like claiming you can fix a sinking ship by drilling more holes in its hull.
At best, it’s naive. At worst, it’s self-defeating and disingenuous.
Both types of assessment are supposed to measure what students have learned. But not all learning is equal.
For example, a beginning chef needs to know how to use the stove, have good knife skills and how to chop an onion. But if you give her a standardized test, it instead might focus on how to make foie gras – something that would only come in handy at a high end French restaurant.
The fact of the matter is that standardized tests do NOT necessarily focus on the most important aspects of a given task. They focus on obscurities – things that most students don’t know.
This is implicit in the design of these exams and is very different from the kinds of tests designed by classroom teachers.
When a teacher makes a test for her students, she’s focused on the individuals in her classes. She asks primarily about the most essential aspects of the subject and in such a way that her students will best understand. There may be a few obscure questions, but the focus is on whether the test takers have learned the material or not.
When psychometricians design a standardized test, on the other hand, they aren’t centered on the student. They aren’t trying to find out if the test taker knows the most important facts or has the most essential skills in each field. Instead, there is a tendency to eliminate the most important test questions so that the test – not the student – will be better equipped to make comparisons between students based on a small set of questions. After all, a standardize test isn’t designed for a few classes – it is one size fits all.
New questions are field tested. They are placed randomly on an active test but don’t count toward the final score. Test takers aren’t told which questions they’ll be graded on and which are just practice questions being tried out on students for the first time. So students presumably give their best effort to both types. Then when the test is scored, the results of the field test questions determine if they’ll be used again as graded questions on a subsequent test.
If 40 to 60 percent of test takers answer the question correctly, it might make it onto the test. But questions that are answered correctly by 80 percent or more of test takers are usually jettisoned.
“As a consequence of the quest for score variance in a standardized achievement test, items on which students perform well are often excluded. However, items on which students perform well often cover the content that, because of its importance, teachers stress. Thus, the better the job that teachers do in teaching important knowledge and/or skills, the less likely it is that there will be items on a standardized achievement test measuring such knowledge and/or skills.”
If we are guided in large part by standardized test scores, we aren’t guided by authentic learning. We’re guided by a false picture of learning. Therefore, the most effective way – perhaps the only practical way – of raising test scores is to teach directly to a specific test. And not only the test, but the specific version of the test being given that year.
So if we do somehow manage to raise test scores, we haven’t improved academics at all but a mere semblance of it. And thus the equity we might celebrate in such a situation would be just as false.
You got a good score on the MAP test. Hurrah! But that doesn’t mean you know anything of real value except how to take this particular MAP test which, itself, will change after the next round of questions are field tested.
And if we insist on evaluating the equity of our schools on these test scores, we will only make things that much worse.
We end up chasing the psychometricians. We try to guess which aspects of a subject they think most students don’t know and then we teach our students that to the exclusion of more important information. And since what students don’t know changes, we end up having to change our instructional focus every few years based on the few bread crumbs surreptitiously left for us by the state and the testing corporations.
That is not a good way to teach someone anything. It’s like teaching your child how to ride a bike based on what the neighbor kid doesn’t know.
It’s an endless game of catch up that only benefits the testing industry because they cash in at every level. They get paid to give the tests, to grade the tests and when students fail, they get paid to sell us this year’s remediation material before kids take the test again, and – you guessed it – the testing companies get another check!
It’s a dangerous feedback loop, a cycle that promotes artificially prized snippets of knowledge over constructive wholes.
But we can no longer continue feeding that beast and pretending that we can somehow provide equity to our underserved children, too.
We have to choose – equity or testing. Fairness or unrestrained capitalism.
Do not believe anyone who tells you to support a plan built on both.
It does not exist.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
These assessments were based on explicitly eugenicist foundations – the idea that certain races were distinctly superior to others.
Colleague Lewis Terman made the goal clear in his book, The Measurement of Intelligence, that these “experimental” tests will show “enormously significant racial differences in general intelligence, differences which cannot be wiped out by any scheme of mental culture.”
In 1923, another psychologist, Carl Brigham, took these ideas further in his seminal workA Study of American Intelligence. In it, he used data gathered from these IQ tests to argue the following:
“The decline of American intelligence will be more rapid than the decline of the intelligence of European national groups, owing to the presence here of the negro. These are the plain, if somewhat ugly, facts that our study shows. The deterioration of American intelligence is not inevitable, however, if public action can be aroused to prevent it.”
The practice was even upheld by the US Supreme Court in the 1927 Buck v. Bell decision. Justices decided that mandatory sterilization of “feeble-minded” individuals was, in fact, Constitutional.
Of the ruling, which has never been explicitly overturned, Justice Oliver Wendell Holmes wrote, “It is better for all the world, if instead of waiting to execute degenerate offspring for crime, or to let them starve for their imbecility, society can prevent those who are manifestly unfit from continuing their kind…. Three generations of imbeciles are enough.”
Eventually Brigham took his experience with Army IQ tests to create a new assessment for the College Board – the Scholastic Aptitude Test – now known as the Scholastic Assessment Test or SAT. It was first given to high school students in 1926 as a gatekeeper. Just as the Army intelligence tests were designed to distinguish the superior from the inferior, the SAT was designed to predict which students would do well in college and which would not. It was meant to show which students should be given the chance at a higher education and which should be left behind.
And unsurprisingly it has always – and continues to – privilege white students over children of color. The same as nearly every standardized test still does.
HAS IT CHANGED?
None of this can be challenged. These are historical facts. They are simply what happened justified in the words of the people who perpetrated them.
In 1964, a Department of Education report found that the average black high school senior scored below 87% of white seniors (in the 13 percentile) on standardized assessments. Fifty years later, the National Assessment of Educational Progress (NAEP) found that black seniors had narrowed the gap until they were merely behind 81% of white seniors (scoring in the 19th percentile).
Is that really the kind of progress you want to champion?
The reason for the disparity has nothing to do with the learning students of color (and the poor whose scores are similar) achieve nor their worth as human beings.
Discrimination is purposefully built in to the standardization process, according to W. James Popham, PhD, Professor Emeritus at the University of California at Los Angeles and former test maker. He explained in an interview with Frontline:
“Traditionally constructed standardized achievements, the kinds that we’ve used in this country for a long while, are intended chiefly to discriminate among students … to say that someone was in the 83rd percentile and someone is at 43rd percentile. And the reason you do that is so you can make judgments among these kids. But in order to do so, you have to make sure that the test has in fact a spread of scores. One of the ways to have that test create a spread of scores is to limit items in the test to socioeconomic variables, because socioeconomic status is a nicely spread out distribution, and that distribution does in fact spread kids’ scores out on a test.”
The scores have to fall into categories – Below Basic, Basic, Proficient and Advanced, for instance. If too many students cluster in the middle, the results are invalid. We need the scores spread out – even if we must resort to non-educational factors to get there.
Family income is not something the tests ignore, says Popham. It is an essential component specifically tested for in question construction. In fact, he claims that between 15-80% of the questions (depending on the subject area) on norm-referenced exams are linked to socio-economic status (SES).
Thus minorities with higher percentages of impoverished people are selected against. Not because of any explicit racist ideology – but to get the pretty bell curve standardized assessments require.
“Too often, test designers rely on questions which assume background knowledge more often held by White, middle-class students. It’s not just that the designers have unconscious racial bias; the standardized testing industry depends on these kinds of biased questions in order to create a wide range of scores.”
For example, Choi recalled a 10th grade student in his class asking him about a standardized test question. “With a puzzled look, she pointed to the prompt asking students to write about the qualities of someone who would deserve a “key to the city.” Many of my students, nearly all of whom qualified for free and reduced lunch, were not familiar with the idea of a ‘key to the city.’”
So when they get such a question wrong, it isn’t necessarily because they don’t know the concept being tested, but they don’t understand what was being asked in the first place.
“Compare two 1998 SAT verbal [section] sentence-completion items with similar themes: The item correctly answered by more blacks than whites was discarded by the Educational Testing Service, whereas the item that has a higher disparate impact against blacks became part of the actual SAT. On one of the items, which was of medium difficulty, 62% of whites and 38% of African-Americans answered correctly, resulting in a large impact of 24%…On this second item, 8% more African-Americans than whites answered correctly…”
In other words, the criteria for whether a question is chosen for future tests is if it replicates the outcomes of previous exams – specifically tests where students of color score lower than white children. And this is still the criteria test makers use to determine which questions to use on future editions of nearly every assessment in wide use in the US.
We no longer profess eugenicist ideas of racial purity embedded in our assessments as self evident or based on science. But they’re there none-the-less.
Today, the very concept of intelligence being quantifiable remains in question.
In “The Mismeasurement of Man,” evolutionary biologist Stephen Jay Gould challenged many of these ideas – in particular those of Terman. Rather than see intelligence as a genetic trait, Gould envisioned it more abstractly and thus shattered the idea that any mere number could capture human value.
We can’t just give a faulty bridge a new coat of paint. We must demolish it and rebuild an entirely new structure to carry us into the future.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
He takes notes in class, does all his homework and participates in discussions.
He writes insightful essays and demonstrates a mastery of spelling and grammar.
He reads aloud with fluency and inflection. He asks deep questions about the literature and aces nearly all of his classroom reading comprehension tests.
But after reluctantly subjecting my classes to the MAP and being instructed to analyze the results with my colleagues, we noticed this contradiction.
In many cases, scores did not match up with teacher expectations for our students.
In about 60-80% of cases, students who had demonstrated high skills in the subject were given scores below the 50th percentile – many below the 25th percentile.
These were kids with average to high grades who the MAP scored as if they were in the bottom half of their peers across the state.
Heck! A third of my students are in the advanced class this year – but the MAP test would tell me most of them need remediation!
If we look at that data dispassionately, there are possible explanations. For one, students may not have taken the test seriously.
And to some degree this is certainly the case. The MAP times student responses and when they are input fast and furious, it stops the test taker until the teacher can unlock the test after warning them against rapid guessing.
However, the sheer number of mislabeled students is far too great to be accounted for in this way. Maybe five of my students got the slow down sloth graphic. Yet so many more were mislabeled as failures despite strong classroom academics.
Nice for them. However, I am not some rube reading this in the paper. I am not examining some spreadsheet for which I have no other data. I am IN the classroom every day observing these very same kids. I’ve been right there for almost an entire grading period of lessons and assessments – formative and summative. I have many strong indications of what these kids can do, what they know and what they don’t know.
I am a Nationally Board Certified Teacher with more than two decades experience. But Northwest Evaluation Association (NWEA), a testing company out of Portland, Oregon, wants me to believe that after 90 minutes it knows my students better than I do after six weeks!
Time to admit the MAP is a faulty product.
But it’s not just that one standardized test. We find the same disparity with the PSSA and other like assessments.
Nationally, classroom grades are better than these test scores.
By contrast, teacher-created tests are just the opposite. They are designed almost exclusively to assess whether learning has taken place and to what degree. Comparability isn’t really something we do. That’s the province of administrators and other support staff.
The primary job of teaching is just that – the transfer of knowledge, offering opportunities and a conducive environment for students to learn.
You can’t give a person a blood transfusion if you can’t accurately measure how much blood you’re giving her. And comparing how much blood was given to a national average of transfusions is not helpful.
You need to know how much THIS PERSON needs. You need to know what would help her particular needs.
When good students get bad test scores, it invariably means you have a bad test.
Until society accepts this obvious truth, we will never be able to provide our students with the education they deserve.
Good students will continue to be mislabeled for the sake of a standardized testing industry that is too big to fail.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
She was in her 90s and had been unwell since before COVID. But she was also our matriarch, the point around which so much of our interrelations orbited and met.
After the funeral, I found myself at my uncle’s house somehow tasked with watching over several young cousins who had had just about enough of sitting around quietly in itchy suits and dresses.
To get a moment to myself, I set them a task: go downstairs among the assorted relatives and ask them to tell you a story about Ce Ce. Best story wins.
They went off like an explosion. And when they came back, they each had a touching tale about Ce Ce.
One was about how she defended a niece who wanted to marry someone of another faith. Another story was a fond recollection of the sweet and sour spaghetti sauce she used to make, the recipe of which is lost forever.
I was even surprised to hear some stories I had never known like that after my grandfather died, a semi-famous painter had asked Ce Ce on a date!
When my little cousins’ recitations were done, they were united in one thing – wanting to know who won.
I stumbled. I stammered.
I really had no way of judging such a thing.
They had all brought back such wonderful stories. Who won? We were ALL enriched by hearing them.
And the results of these tests are used to make high stakes decisions about which classes the students can enroll in, which enrichments, field trips or remediation they require, and even how much funding will be given or withheld from the schools and districts where they attend.
What if we took classroom grades as seriously as we take standardized test scores?
What if we valued them MORE?
The world would be a very different place.
The entire narrative of failing students and failing schools would turn on its head. After all, graduation rates have steadily increased over the last decade.
Students are completing more courses and more difficult courses. And students are even getting higher grades in these classes!
The new analysis comes from the U.S. Department of Education, and tracks transcripts of a representative sample of high school graduates in 1990, 2000, 2009, and 2019.
It does not include scores from 2020 and 2022 when both classroom grades and national test scores fell. But that’s clearly because of the pandemic and the fact that most students educations and testing schedules were disrupted.
Before COVID, students increasingly were taking higher-level courses, and their Grade Point Averages (GPAs) were steadily rising — from an average of 2.68 in 1990 to 2.94 in 2000, 3.0 in 2009, and 3.11 in 2019.
This is true of students from all backgrounds, but disparities still existed. On average, white and Asian students had higher GPAs than Black and Hispanic students. Though girls, overall, had higher GPAs than boys.
However, on the National Assessment of Educational Progress (NAEP), given to a sample of students across the country, test scores during the same period did not show a similar increase. Math and reading scores in 2019 were slightly lower than in 2009 and unchanged from 2005. Science scores haven’t budged since 2009.
It doesn’t take much to show why classroom grades are better at assessing student learning. Compare them with standardized test scores.
Students earn grades based on a wide range of assessments, activities, and behaviors – quizzes, class participation, oral and written reports, group assignments, homework, and in-class work.
Standardized tests, on the other hand, are not assigned on such a multifaceted range of factors. Instead, they are designed to obtain a measure of student proficiency on a specified set of knowledge and skills within limited academic areas, such as mathematics or reading.
Classroom grades are tapestries sown from many patches showing a year’s worth of progress. Standardized tests are at best snapshots of a moment in time.
So the biggest difference isn’t a matter of validity, it is pragmatism. Test scores can be used to rate students from all over the country or the world. They can be used to sort kids into a hierarchy of best to worst. Though why anyone would want to do that is beyond me. The purpose of education is not like the National Football League (NFL). It’s to encourage learning, not competition based on a simulation of learning.
And there is evidence that classroom grades are more valid than standardized test scores.
Kids with perfect scores on the SAT or American College Testing (ACT) tests don’t achieve more than kids who received lower scores or never took the tests in the first place.
Researchers followed more than 123,000 students who attended universities that don’t require applicants to take these tests as a prerequisite for admission. They concluded that SAT and ACT test scores do not correlate with how well a student does in college.
However, classroom grades do have predictive value – especially when compared to standardized tests. Students with high grades in high school but middling test scores do better in college than students with higher test scores and lower grades.
Why? Because grades are based on something other than the ability to take one test. They demonstrate a daily commitment to work hard. They are based on 180 days (in Pennsylvania) of classroom endeavors, whereas standardized tests are based on the labor of an afternoon or a few days.
Classroom grades would not have such consistent predictive value if they were nothing but the result of grade inflation or lenient teachers.
In fact, of the two assessments – classroom grades and standardized tests – one is far more essential to the daily learning of students than the other.
We could abolish all standardized testing without any damage to student learning. In fact, the vacuum created by the loss of these high stakes tests would probably result in much less teaching to the test. Days, weeks, months of additional class time would suddenly appear and much more learning would probably take place.
Academic decisions about which classes students can enroll in or what remediation is necessary could just as easily be made based on classroom grades and teacher observations. And funding decisions for schools and districts could be made based on need and equity – not the political football of standardized testing.
However, getting rid of classroom grades would be much more disruptive. Parents and students would have few measures by which to determine if students had learned the material. Teachers would have fewer tools to encourage children to complete assignments. And if only test scores remained, the curriculum would narrow to a degree unheard of – constant, daily test prep with no engagement to ones life, critical thinking or creativity.
To be fair, there are mastery-based learning programs that try to do without grades, but they are much more experimental and require a complete shift in how we view learning. This is a more holistic system that requires students to demonstrate learning at one level before moving ahead to the next. However, it is incredibly labor intensive for teachers and often relies heavily on edtech solutions to make it viable.
I’m not saying this is an impossible system or even taking a stance on its value. But a large scale shift away from classroom grades would be chaotic, confusing and probably a failure without serious support, scaffolding and parental, teacher and student buy-in.
At the end of the day, classroom grades are the best tool we have to determine whether learning has taken place and to what degree. We should do everything we can to change the way policymakers prefer the standardized approach to the personalized one.
‘It would be nice if all of the data which sociologists require could be enumerated because then we could run them through IBM machines and draw charts as the economists do. However, not everything that can be counted counts, and not everything that counts can be counted.”
Thus, the urge to quantify student learning seems predicated on the popular maxim: If you can’t measure it, you can’t manage it.
Classroom grades are actually concerned with the project at hand – assessment of learning.
Which brings me back to my little cousins.
When I told them I couldn’t possibly pick a winner between them based on their stories, there were lots of groans of annoyance.
They viewed the whole project as a competition and they wanted to win.
I hope on reflection they’ll see that we all won.
Everything isn’t a contest. We are not all opponents.
If they can grasp that, it would be the greatest lesson I could teach.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
I’m a public school teacher in western Pennsylvania, and when I returned to school this week before classes started, I noticed my stapler was irreparably jammed from last year.
The secretary lead me to a closet full of brand new Swingline staplers.
I thanked her, took one back to my room and started stapling.
Three staples in, it was irreparably jammed.
When I returned home that evening and complained to my family about the woes of the day, my sweet 13-year-old daughter offered me a stapler we had around the house.
When I brought it to school, it worked like a dream.
It wasn’t some top of the line model. It was another basic Swingline stapler. It was slightly less boxy and more modern than the kind I got from the office. But it worked. That’s the important difference.
So why did the office have a closet full of faulty staplers?
Those staplers – that many staplers – probably add up to hundreds of dollars.
And they don’t even work!
Sadly, the full extent of the waste district-wide is much farther reaching than just the staplers.
Later that very day, teachers in my building were forced to sit through a virtual training on the MAP test.
This is an assessment made by Northwest Evaluation Association (NWEA), a so-called non-profit organization out of Portland, Oregon.
The company claims its assessments are used by over 9,500 schools and districts in 145 countries – but none is more popular than the MAP.
Some states even require the MAP as part of their standardized testing machinery. However, in the Commonwealth, the MAP is used as a pre-test or practice assessment by districts that elect to pay for it.
My building – the middle school – used a variety of different assessments throughout the years for this purpose – IXL, CDT, etc.
My district had used the MAP consistently for years at the elementary schools, so someone in administration thought it made sense to bring it to the middle school now and eventually institute it in the high school, as well.
Do we really need an assessment BEFORE the state mandated assessments?
Heck no!
Classroom teachers give enough assignments and tests of their own to know where their students are academically throughout the year. We grade them after all. What do you think that’s based on – guessing?
But certain administrators just love these pre-tests. They love looking at spreadsheets of student data and comparing one grading period to another. They think if the numbers go higher, it will be proof they’re good principals and functionaries.
It’s pathetic to be honest. What a waste of taxpayer dollars that could be used for actual learning! What a waste of class time that could be used for actual teaching!
And what a negative impact these assessment actually have on students and their learning!
For instance, at the MAP training, teachers were told the assessment’s job was to show how our students were doing in Reading, Math and Science compared with an average test taker.
How is constantly comparing them to a norm going to help them improve?
If I went on a diet and stepped on the scale, learning that my weight loss wasn’t as high as an average dieter would not help me stay away from sweets. If anything, it would inspire me to go on a binge in the snack drawer.
When learning a skill, it doesn’t help to know how well others are or are not learning that same skill. It matters how much you are learning in comparison to yourself. Yesterday I knew THIS. Today I know a bit MORE. Who cares what the so-called average learner can do!?
But no matter how many times I say such things to administrators or paid trainers from NWEA, they just don’t get it.
At this training, the instructor actually wanted to know what “elevator speech” teachers were going to give to parents about why the MAP was important!
It’s bad enough we’re being forced to give this crappy assessment, but now you want us to spout propaganda to the very people paying our salaries!?
Having trusted teachers sooth community worry with corporate propaganda would be a big win for the testing company.
However, I’ll give the trainer one thing – she understood that the MAP assessment scores would not be useful unless students could be encouraged to take the test seriously. Nobody tries their best at something they think is unimportant.
Her solution was two-fold. First, NWEA has produced several propaganda videos to show students why the test is important.
Teachers are supposed to monitor all this on a screen and intervene when it occurs. We’re supposed to counsel kids not to just guess and then allow them back on the test. If the algorithm still thinks students are guessing, we’re supposed to suspend their test and make them take it all over again.
Teachers throughout the state have to take on-line classes every year about what we are and are not allowed to do during the PSSA test. Stopping students who seem to be guessing, is not allowed. I’m not even allowed to point out if a student skipped a question on the test!
I certainly can’t scrap a PSSA test that I think a student didn’t give his best effort on and make him do it again!
So how exactly is this MAP test a practice for the real thing!?
Even under the best of circumstances, it’s an artificial environment where scores are massaged to give an unrealistic picture of how students will do on the PSSA.
Of course, administration at my school has one more trick up its sleeve to get students to take the MAP test seriously.
These people are making lots of money off this standardized testing baloney!
According to a 2015 brochure from NWEA about the MAP test, it costs $13.50 per student to take the test every year. And that’s just for the Reading and Math. It costs an additional $2.50 per pupil for the Science test.
So if we estimate 1000 students at the elementary and middle school level, that’s roughly $16,000 a year to take the test.
And that doesn’t include the price of trainings like the one I had to sit through this week.
According to that same brochure, the cost for a single days training is $4,000, though sometimes it can be reduced to $3,500 if you buy the right package.
Trainings can go up to $40,000 for multiple days and an in-person trainer.
I wonder how much money my district flushed down the toilet on this garbage.
We need a test of basic decency for decision makers.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
Schools could barely keep their doors open and conduct in-person classes. Many educators were still teaching their students on-line or both on-line and in-person at the same time. Hundreds of teachers died from the virus. Thousands of students have lost parents, relatives or became sick, themselves.
And if increasing cases of the even more contagious Delta Variant continue to spread in 2021-22 while the last 30% of American adults are reluctant to get the vaccine, the situation could be even worse this spring.
In their paper, Bruno and Goldhaber looked at last year’s waiver requests asking for permission to cancel or modify statewide exams in 11 states and the District of Columbia.
“We speculate that there was concern that even temporarily waiving statewide tests would give momentum to those advocating for the elimination of testing all together. That is, [the US Department of Education] USDOE (and perhaps states that did not request that common assessments be waived) may be less interested in what happens with testing this year than worried about a slippery slope toward increasingly lax testing requirements.” [Emphasis mine]
Speaking of which, the report then becomes focused on advice for standardized testing advocates to combat mounting pressure to end these mandated federal assessments.
If the public doesn’t see the value in the tests, Bruno and Goldhaber say, policymakers must explain why the tests are important, and not just in generalities. They must explicitly show how standardized test scores improve education and help specific students.
“We encourage policymakers to think carefully, explicitly, and publicly about how they have tailored their standardized testing policies to achieve various diagnostic, research, and accountability objectives. This will help to ensure that standardized tests have benefits for more schools and students and will bolster fragile political support for statewide tests.”
However, nowhere in the entire paper do Bruno and Goldhaber actually do this, themselves.
How do standardized tests help students?
That’s exactly the question at stake here.
In short, I would argue as I have countless times before that they DO NOT help students.
If the authors had good counterarguments, now would have been a good time.
The authors do say that standardized test scores are predictive of latter student outcomes but they ignore whether other assessments or factors are MORE predictive.
Yes, students with high test scores often graduate, excel in college or trade schools, etc. However, the same can be said with classroom grades. In fact, classroom grades are evenmore accurate.
This just makes sense. Classroom grades are based on at least 180 days of formal and informal assessment. Standardized tests are merely a snapshot of a few days work.
However, even more predictive is child poverty. The rich kids usually do much better than the poor kids. Same with race, class and the funding each student receives at his or her school.
If you want to help students, that’s where you need to begin – equitable resource allocation. Make sure all students have what they need to succeed, and realize that the more poverty you have, the greater the need, the greater the resources necessary.
Standardized tests literally were invented to justify bias. They were designed to prove that higher income, higher class, white people were entitled to more than poorer, lower class, brown people. Any defense of the assessments today must explain how the contemporary variety escapes the essential racist assumptions the entire project is based on.
And there is no better image to prove that than forcing kids to take a meaningless test during a global pandemic.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
It’s as if they’re starving to learn something, and this play is nourishing their hearts and minds.
I laugh because my first thought was to come down on the shouting out and side commenting until a deeper part of me realized this was all okay. They were on-task, if unrestrained.
Comprehension is not a race. There is no one ahead or behind. Everyone goes at their own pace. And if you try to force someone to go more quickly than is best for them, they’ll stumble and fall.
But let’s be honest, these learning loss champions are not really worried about children. They’re representatives of the standardized testing industry.
So as the school year rapidly comes to a close, I have a suggestion to make.
I know I’m not qualified to do so.
I’m just a public school teacher with 17 years experience. I’ve never sat on any think tank boards. No testing corporation has ever paid me a dime to hawk one of their high quality remediation products.
I know that’s controversial, but I believe it to be true.
As such, they need down time.
They need time to regroup and recharge.
This pandemic has been hard on everyone.
As of April 1, nearly 3.47 million children have tested positive for COVID-19, most with mild symptoms, according to the American Academy of Pediatrics. A few hundred have died, mostly children of color. Many more kids probably contracted the virus but were asymptomatic spreaders of the disease to adults.
As a result, between 37,000 and 43,000 children in the United States have lost at least one parent to COVID-19, according to USC research.
They have suffered through changes in routine, disruptions in learning, breaks in the continuity of their healthcare, missed significant life events like birthday parties, vacations and graduations. But worst of all they have suffered the loss of safety and security.
We should not be demanding they work harder at a time like this.
We should be providing them with kindness, empathy and love.
In the classroom, I no longer have a thing called “Late Work.”
If a student hands in an assignment passed the due date, there is no penalty. I just grade it. And if it isn’t done correctly, I give them a chance to redo it.
As many chances as they need.
I remediate. I tutor. I offer advice, counseling, a sympathetic ear.
It’s not that much different than any other year, except in how often children need it now.
Kids AND their parents.
I can’t tell you how many adults I’ve counseled in the last several months.
So when the last day of school arrives, I will close my books.
There will be no assignments over the summer from me.
No homework. No requirements. No demands.
The best things kids can do is go out and play.
Have fun.
Recharge.
The corporate testing drones will tell you that’s a waste of time. Our kids are getting behind doing things like that.
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
The Covid-19 pandemic closed school rooms across the nation for various lengths of time. Some are still closed. Some are beginning to close again.
Many classes were conducted remotely through conferencing software like Zoom and file sharing programs like Google Classroom. Others were conducted through a hybrid model combining in-person instruction and cyber instruction. While still others met in-person with numerous mitigation efforts like masks, social distancing and air purifiers.
Many students were absent, struggled to learn and experienced countless traumas due to the isolation, sickness and deaths.
That’s more than Americans who died in the attack on Pear Harbor (2,403), the 9/11 terrorists attacks (3,000), WWI (116,000) or WWII (405,000).
Only the Civil War (600,000 – 850,000) has a larger death toll. For now.
As of April 1, nearly 3.47 million children have tested positive for COVID-19, most with mild symptoms, according to the American Academy of Pediatrics. A few hundred have died, mostly children of color. Many more kids probably contracted the virus but were asymptomatic spreaders of the disease to adults.
As a result, between 37,000 and 43,000 children in the United States have lost at least one parent to COVID-19, according to USC research.
How do you sort through all these tragedies and traumas and say THIS was caused by a lack of standardized testing?
But in 2001 we created an industry. Huge corporations write the tests, grade the tests and provide the remediation for the tests. Billions of dollars in taxes are funneled into this captive market which creates monetary incentives for our lawmakers to keep the system going.
Yes, some civil rights organizations have waffled back and forth over this as big donors who value the tests make or withhold contributions. Meanwhile, many other more grassroots civil rights organizations such as Journey for Justice Alliance (JJA), a group made up of 38 organizations of Black and Brown parents and students in 23 states, have continuously called for the abolition of high stakes testing.
A scientific mind would be empirical about this. It would examine the results as much as possible and determine whether moving forward made any sense.
This is especially true as the pandemic health crisis continues to make the act of giving the tests difficult at best and dangerous at worst.
There is no way a logical mind can look at the situation and not come to the conclusion that the status quo on testing is a triumph of capitalism over science and reason.
In a month or so, the year without testing will be just that – a single year.
We shall go on to the end. We shall test during Covid, we shall test in the classes and on-line, we shall test with growing confidence and growing strength wearing masks, we shall defend our industry, whatever the cost may be. We shall test in the homes, we shall fill in bubbles on sanitized desks, we shall test in the fields and in the streets, we shall test in the hospitals; we shall never surrender!
Do you live in Allegheny County, Pennsylvania? I’m running for County Council in District 9
Like this post? You might want to consider becoming a Patreon subscriber. This helps me continue to keep the blog going and get on with this difficult and challenging work.
gadflyonthewallblog 