The Six Biggest Problems with Data-Driven Instruction

0

 

 

“On the dangers of being data-driven: Imagine driving from A to B ignoring the road, the weather, the traffic around you… only staring at the gauges on the dashboard.”

 – Educator Dan McConnell

 

 

“Make your instruction data-driven.”

 

If you’re a public school teacher, you’ve probably heard this a hundred times.

 

In the last week.

 

Principals and administrators use that word – “data-driven” – as if it were inscribed over the front doors of the schoolhouse in stone.

 

The idea goes like this: All lessons should be based on test scores.

 

Students take the federally mandated standardized test. Your job is to make sure they get the best possible score. Your class is nothing but a way station between standardized tests.

 

Pretest your students and then instruct them in such a way that when they take the test again, they’ll get the best possible score.

 

It’s total nonsense. And it doesn’t take much to see why.

 

No teacher should ever be data-driven. Every teacher should be student-driven.

 

You should base your instruction around what’s best for your students – what motivates them, inspires them, gets them ready and interested in learning.

 

To be sure, you should be data-informed – you should know what their test scores are and that should factor into your lessons in one way or another – but test scores should not be the driving force behind your instruction, especially since standardized test scores are incredibly poor indicators of student knowledge.

 

No one really believes that the Be All and End All of student knowledge is children’s ability to choose the “correct” answer on a multiple-choice test. No one sits back in awe at Albert Einstein’s test scores – it’s what he was able to do with the knowledge he had. Indeed, his understanding of the universe could not be adequately captured in a simple choice between four possible answers.

 

As I see it, there are at least six major problems with this dependence on student data at the heart of the data-driven movement.

 

So without further ado, here is a sextet of major flaws in the theory of data-driven instruction:

 

 

 

  1. The Data is Unscientific

    When we talk about student data, we’re talking about statistics. We’re talking about a quantity computed from a sample or a random variable.

    As such, it needs to be a measure of something specific, something clearly defined and agreed upon.

    For instance, you could measure the brightness of a star or its position in space.

    However, when dealing with student knowledge, we leave the hard sciences and enter the realm of psychology. The focus of study is not and cannot be as clearly defined. What, after all, are we measuring when we give a standardized test? What are the units we’re using to measure it?

    We find ourselves in the same sticky situation as those trying to measure intelligence. What is this thing we’re trying to quantify and how exactly do we go about quantifying it?

    The result is intensely subjective. Sure we throw numbers up there to represent our assumptions, but – make no mistake – these are not the same numbers that measure distances on the globe or the density of an atomic nucleus.

    These are approximations made up by human beings to justify deeply subjective assumptions about human nature.

    It looks like statistics. It looks like math. But it is neither of these things.

    We just get tricked by the numbers. We see them and mistake what we’re seeing for the hard sciences. We fall victim to the cult of numerology. That’s what data-driven instruction really is – the deepest type of mysticism passed off as science.

    The idea that high stakes test scores are the best way to assess learning and that instruction should center around them is essentially a faith based initiative.

    Before we can go any further, we must understand that.

  2. It Has Never Been Proven Effective

    Administrators and principals want teachers to base their instruction around test scores.

    Has that ever been proven an effective strategy for teachers planning lessons or the allocation of resources? Can we prove a direct line from data to better instruction to better test scores?

    The answer is an unequivocal NO.

    In a 2007 study from Gina Schuyler Ikemoto and Julie A. Marsh published in the Yearbook for the National Society for the Study of Education, data driven instruction actually was found to have harmful effects on educator planning and, ultimately, student learning.

    Researchers looked at 36 instances of data use in two districts, where 15 teachers used annual tests to target weaknesses in professional development or to schedule double periods of language arts for English language learners. The result was fewer instances of collective, sustained, and deeper inquiry by groups of teachers and administrators using multiple data sources – test scores, district surveys, and interviews – to reallocate funds for reading specialists or start an overhaul of district high schools.

    Teachers found the data less useful if it was not timely – standardized test scores are usually a year old by the time they get to educators. Moreover, the data was of less value if it did not come with district support and if instructors did not already buy into its essential worth.

    In short, researchers admitted they could not connect student achievement to the 36 instances of basic to complex data-driven decisions in these two districts.

    But that’s just one study.

    In 2009, the federal government published a report (IES Expert Panel) examining 490 studies where schools used data to make instructional decisions.

    Of these studies, the report could only find 64 that used experimental or quasi-experimental designs. Of these it could find only six – yes, six – that met the Institute of Education Sciences standard for making causal claims about data-driven decisions to improve student achievement.

    And when examining these six studies, the panel found “low evidence” to support data-driven instruction. They concluded that the theory that data-driven instructional decisions improve student test scores has not been proven in any way, shape or form.

  3. It’s Harmful – The Stereotype Threat and Motivation

    Data-driven instruction essentially involves grouping students based on their performance on standardized tests.

    You put the low scorers HERE, the students on the bubble who almost reached the next level HERE, and the advanced students HERE. That way you can easily differentiate instruction and help meet their needs.

    However, there is a mountain of psychological research showing that this practice is harmful to student learning. Even if you don’t put students with different test scores in different classes, simply informing them that they belong to one group or another has intense cognitive effects.

    Simply being told that you are in a group with lower test scores depresses your academic outcomes. This is known as the stereotype threat.

    When you focus on test scores and inform students of where they fall on the continuum down to the percentile – of how far below average they are – you can trigger this threat. Simply tracking students in this way can actually make their scores worse.

    It can create negative feelings about school, threatening students’ sense of belonging, which is key to academic motivation.

    But it’s not just the low scorers who are harmed. Even the so-called “advanced” students can come to depend on their privileged status. They define themselves by their achievement, collecting prizes, virtual badges and stickers. These extrinsic rewards then transform their motivation from being driven by the learning and the satisfaction of their curiosity to depending on what high achievement gets them, researchers have found.

    In short, organizing all academics around tests scores is a sure way to lower them.

  4. The Data Doesn’t Capture Important Factors

    Data-driven instruction is only as good as the data being used. But no data system can be all inclusive.

    When we put blinders on and say only these sorts of factors count, we exclude important information.

    For instance, two students do the same long-term project and receive the same grade. However, one student overcame her natural tendency to procrastinate and learned more than in past projects. The other did not put forth his best effort and achieved lower than his usual.

    If we only look at the data, both appear the same. However, good teachers can see the difference.

    Almost every year I have a few students who are chronically tardy to class. A good teacher finds out why – if this is because they aren’t making the best use of the class interval or if they have a greater distance to travel than other students. However, if we judge solely on the data, we’re supposed to penalize students without considering mitigating factors. That’s being data-driven – a poor way to be a fair teacher.

    It has been demonstrated repeatedly that student test scores are highly correlated with parental income. Students from wealthier parents score well and those from more impoverished families score badly. That does not mean one group is smarter or even more motivated than the other. Living in poverty comes with its own challenges. Students who have to take care of their siblings at home, for instance, have less time for homework than those who have nothing but free time.

    A focus solely on the data ignores these factors. When we’re admonished to focus on the data, we’re actually being told to ignore the totality of our students.

  5. It’s Dehumanizing

    No one wants to be reduced to a number or a series of statistics.

    It is extremely insulting to insist that the best way for teachers to behave is to treat their students as anything other than human beings.

    They are people with unique needs, characteristics, and qualities, and should be treated accordingly.

    When one of my students does an amazing job on an assignment or project, my first impulse is not to reduce what they’ve done to a letter grade or a number. I speak my approbation aloud. I write extensive comments on their papers or conference with them about what they’ve done.

    Certainly, I have to assign them a grade, but that is merely one thing educators do. To reduce the relationship to that – and only that – is extremely reductive. If all you do is grade the learner, you jeopardize the learning.

    Every good teacher knows the importance of relationships. Data-driven instruction asks us to ignore these lessons in favor of a mechanistic approach.

    I’m sorry. My students are not widgets and I refuse to treat them as such.

    I am so sick of going to conferences or faculty meetings where we focus exclusively on how to get better grades or test scores from our students. We should, instead, focus on how to see the genius that is already there! We should find ways to help students self-actualize, not turn them into what we think they should be.

    At this point, someone inevitably says that life isn’t fair. Our students will have to deal with standardized tests and data-driven initiatives when they get older. We have to prepare them for it.

    What baloney!

    If the real world is unfair, I don’t want my students to adjust to that. I want to make it better for them.

    Imagine telling a rape victim that that’s just the way the world is. Imagine telling a person brutalized by the police that the world is unfair and you just have to get used to it.

    This is a complete abdication not just of our job as teachers but our position as ethical human beings.

    Schools are nothing without students. We should do everything we can to meet their needs. Period.

  6. It’s Contradictory – It’s Not How We Determine Value in Other Areas

    Finally, there is an inherent contradiction that all instruction must be justified by data.

    We don’t require this same standard for so many aspects of schooling.

    Look around any school and ask yourself if everything you see is necessarily based on statistics.

    Does the athletic program exist because it increases student test scores? Does each student lunch correlate with optimum grades? Do you have computers and iPads because they have a measureable impact on achievement?

    Some administrators and principals DO try to justify these sorts of things by reference to test scores. But it’s a retroactive process.

    They are trying to connect data with things they already do. And it’s completely bogus.

    They don’t suddenly believe in football because they think it will make the team get advanced scores. They don’t abruptly support technology in the classroom because they think it will make the school achieve adequate yearly progress.

    They already have good reasons to think athletics helps students learn. They’ve seen participation in sports help students remain focused and motivated – sometimes by reference to their own lives. Likewise, they’ve seen the value of technology in the classroom. They’ve seen how some students turn on like someone flipped a switch when a lesson has a technological component.

    These aren’t necessarily quantifiable. They don’t count as data but they are based on evidence.

    We come to education with certain beliefs already in place about what a school should do and others are formed based on the empiricism of being there, day-in, day-out. “Data” rarely comes into the decision making process as anything but a justification after the fact.


    And so we can firmly put the insistence on data-driven instruction in the trash bin of bad ideas.

    It is unscientific, unproven, harmful, reductive, dehumanizing and contradictory.

    The next time you hear an administrator or principal pull out this chestnut, take out one of these counterarguments and roast it on an open fire.

    No more data-driven instruction.

    Focus instead on student-driven learning.

 

Don’t let them co-opt you into the cult of numerology. Remain a difference-maker. Remain a teacher.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-3

Public School is Not For Profit. It is For Children.

Screen Shot 2018-08-28 at 4.14.57 PM

 

Betsy DeVos doesn’t get it.

 

But neither did Arne Duncan.

 

Whether right or left or somewhere in between, the person sitting at cabinet level tasked with advising the President on education matters invariably knows nothing about the purpose of public schools.

 

Duncan thought it had something to do with canned academic standards and standardized tests.

 

DeVos thinks it involves vouchers to religious or private schools.

 

But they’re both as wrong as two left shoes.

 

Public schools exist for one reason and one reason only – to meet the needs of children.

 

They aren’t there to enrich the private sector or even provide the job market with future employees.

 

They exist to teach, to counsel, to inspire, to heal.

 

And all these other schemes favored by Dunce Duncan and Batty Betsy that purport to meet kids needs while somehow enjoying the totally unintended side effect of enriching wealthy investors completely misses the point.

 

Public schools serve one purpose – to help the kids enrolled in them.

 

That’s all.

 

If someone is getting rich off that, there’s a huge problem somewhere.

 

Unfortunately, the Secretaries of Education of Donald Trump and Barack Obama aren’t the only ones to get it wrong. Policymakers on both sides of the aisle have lost sight of this fact.

 

So have pundits and media personalities on Fox News, CNN and MSNBC. So have CEOs and tech entrepreneurs and economists and anyone – really – whom our society seems to take seriously.

 

Don’t believe me?

 

Take the latest pronouncement from DeVos, our Secretary of Education.

 

She announced recently that she was looking into using federal funds to buy guns for teachers to better protect their students from school shooters.

 

It doesn’t take a genius to see that this is not in the best interests of children.

 

Teachers with guns mean a MORE dangerous environment for children, not less.

 

It means escalating the chance of friendly fire much more than boosting the possibility of a kindergarten teacher turning into an action hero.

 

It means heightening the chance of children getting their hands on these firearms and doing themselves or others harm.

 

And given the disproportionate murders of people of color even at the hands of trained professionals in the police force, it means children of color being legitimately terrified of their mostly white educators – or worse.

 

The reason given by DeVos may be to make children safer. But the measure she’s proposing really has nothing to do with them at all.

 

It’s a boondoggle for private industry – one private industry in particular – gun manufacturers.

 

Instead of sensible regulations on a product that’s at least as dangerous as items that are much more heavily controlled – such as cold medicine and automobiles – DeVos is doing the only thing she can to protect what she really cares about – corporate profits.

 

She is using money earmarked “safety” to increase danger.

 

Or as she sees it – she’s using a government apparatus that could harm the gun industry to instead pad its pockets.

 

You’ll hear some progressives and moderates decry this move with passion and fervor – and for good reason – but what many fail to realize is that it’s not new.

 

It’s really just a continuation of a sickness that has crept into our society about how we conceptualize the very idea of school.

 

We have moved away from the proposition that everything must be done in the student’s best interest and have replaced it with an imperative to benefit business and industry.

 

After all, what is the push for academic accountability through standardized tests and Common Core but corporate welfare for the testing and publishing industry?

 

What is the push for charter and voucher schools but government subsidies for school privatization?

 

High stakes standardized testing isn’t about helping students learn. Neither is Common Core, value-added measures or a host of top-down corporate policies championed by lions of the left and supply-side patriots.

 

They are about creating a problem where one doesn’t exist: accountability.

 

“How do we make sure students receive a quality education?” As if this has ever been hard to determine.

 

In general, the schools with greater needs than funding are where students struggle. The schools where everyone has more than they need is where they excel.

 

But they try to sweep the issue of inequitable funding and resources under the rug by framing the question entirely about teachers and schools.

 

In short, instead of asking about an obvious inequality, they hide a preconceived answer in the question: “How do we make sure teachers and schools are actually educating kids?”

 

Wrong question. But here’s the answer, anyway: Administrators observe teachers and determine if they’re doing their jobs. And school boards evaluate administrators.

 

In general, the staff isn’t the problem. It’s the lack of resources we give them to work with – everything from crumbling buildings, large classes, narrowed curriculum to a lack of wraparound social services.

 

It doesn’t take much to see we’re shortchanging our neediest students.

 

You don’t need standardized tests to tell you that. You don’t need new academic standards. You don’t need to evaluate educators on things beyond their control.

 

But doing so creates a new market, a need that can be filled by corporate interests unrestrained by the conviction that public schools are not supposed to be a profit-making venture.

 

People providing services for schools are supposed to make a living – not a killing – off the public’s dime.

 

The same can be said for school privatization.

 

Public schools are in no way inferior to institutions that are privately managed. Tax dollars administered by duly-elected representatives in the light of day are in no way less effective or more corrupt than the alternative – letting bureaucrats behind closed doors dole out the money however they choose even into their own pockets.

 

In fact, just the opposite!

 

Nor have charter or voucher schools ever been shown to increase student learning without also selecting only the best academic students and shunning those most difficult to teach, providing fewer resources for students and/or operating with greater funding.

 

But pretending that privatization is a better alternative to democratic rule creates a market, it opens the door so the system can be gamed for profit at the expense of student learning and wellbeing.

 

That’s why we look in awe at LeBron James, an athlete who uses his fortune to open a school providing all the things society refuses for students of color. A basketball player who refuses to usurp the public’s leadership role in administering that fully public school.

 

He’s a shinning example of actual philanthropy in an age of bogus philanthrocapitalism. But he’s also proof that his solution is not reproducible large scale.

 

The rich – even if they are well intentioned – cannot save us. Only the public can support all public schools.

 

And to do that, we must understand the purpose behind these institutions.

 

Otherwise, we’ll continue to be trapped on a runaway train where the conductor seems to possess no sense of urgency about slowing down.

 

We would never have been in this situation – and in fact could right the course even now – if we just took the time to clarify what we were doing and why we were doing it.

 

We could save generations of children if we stopped cashing in on public schools and realized the reason for their existence.

 

We could ensure both our present and our posterity.

 

If only we remembered that one thing.

 

Public schools are not for profit.

 

They are for children.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-4

Public Schools Best Fulfill Dr. King’s “Purpose of Education”

CZCqp_RUkAE-ZHI

 

What is the purpose of education?

 

Is it to train the next generation of workers?

 

Or is it to empower the next generation of citizens?

 

Is it to give children the skills necessary to meet the needs of business and industry?

 

Or is it to provide them the tools to self-actualize and become the best people they can be?

 

In today’s world, our leaders continue to insist that the answer to the question is the former corporate training model. Knowledge is only valuable if it translates to a job and thus a salary.

 

But we didn’t always think that way.

 

As another Martin Luther King Day is about to dawn this week, I’m reminded of the man behind the myth, a person who clearly would deny this materialistic view of learning.

 

When we think of Dr. Martin Luther King, Jr., we usually think of the towering figure of the Civil Rights Movement who gave the “I have a dream” speech during the March on Washington in 1963.

 

However, as a teacher, I find myself turning to something he wrote in 1947 when he was just an 18-year-old student at Morehouse College.

 

While finishing his undergraduate studies in sociology, he published an essay in the student paper called “The Purpose of Education.”

 

Two sections immediately jump off the page. The first is this:

 

“We must remember that intelligence is not enough. Intelligence plus character–that is the goal of true education. The complete education gives one not only power of concentration, but worthy objectives upon which to concentrate. The broad education will, therefore, transmit to one not only the accumulated knowledge of the race but also the accumulated experience of social living.”

 

So for King it wasn’t enough for schools to teach facts. It wasn’t enough to teach skills, math, writing, reading, history and science. The schools are also responsible for teaching children character – how to be good people, how to get along with each other.

 

It’s a worthy goal.

 

But 2018 contains a far different educational landscape than 1947.

 

When King wrote, there were basically two kinds of school – public and private. Today there is a whole spectrum of public and private each with its own degree of self-governance, fiscal accountability and academic freedom.

 

On the one side we have traditional public schools. On the other we have fully private schools. And in the middle we have charter, voucher and home schools.

 

So which schools today are best equipped to meet King’s ideal?

 

Private schools are by their very nature exclusionary. They attract and accept only certain students. These may be those with the highest academics, parental legacies, religious beliefs, or – most often – families that can afford the high tuition. As such, their student bodies are mostly white and affluent.

 

That is not King’s ideal. That is not the best environment to form character, the best environment in which to learn about people who are different than you and to develop mutual understanding.

 

Voucher schools are the same. They are, in fact, nothing but private schools that are subsidized in part by public tax dollars.

 

Charter schools model themselves on private schools so they are likewise discriminatory. The businesses who run these institutions – often for a profit – don’t have to enroll whoever applies. Even though they are fully funded by public tax dollars, they can choose who to let in and who to turn away. Often this is done behind the cloak of a lottery, but with no transparency and no one checking to ensure it is done fairly, there is no reason to believe operators are doing anything but selecting the easiest (read: cheapest) students to educate.

 

Charter schools have been shown to increase segregation having student bodies that are more monochrome than those districts from which they cherry pick students. This is clearly not King’s ideal.

 

Homeschooling is hard to generalize. There is such a wide variety of experiences that can be described under this moniker. However, they often include this feature – children are taught at home by their parent or parents. They may or may not interact with their academic peers and the degree to which they meet and understand different cultures is variable to say the least. They may meet King’s ideal, but frankly the majority of them probably do not.

 

So we’re left with traditional public schools. Do they instill “intelligence plus character”?

 

Answer: it depends.

 

There are many public schools where children of different races, nationalities, religions, and creeds meet, interact and learn together side-by-side.

 

Students wearing hajibs learn next to those wearing yarmulkes. Students with black skin and white skin partner with each other to complete class projects. Students with parents who emigrated to this country as refugees become friends with those whose parents can trace their ancestors back to the Revolutionary War.

 

These schools are true melting pots where children learn to become adults who value each other because of their differences not fear each other due to them. These are children who not only learn their academics as well – if not often better – than those at competing kinds of schools, but they also learn the true face of America and they learn to cherish it.

 

This is the true purpose of education. This is the realization of King’s academic ideal and his civil rights dream.

 

However, this is not the case at every public school.

 

While there are many like this, there are too many that are increasingly segregated. In fact, in some areas our schools today are more segregated than they were at the time of Dr. King’s assassination.

 

These are schools that get the lion’s share of resources, that have the newest facilities, the widest curriculum, the most affluent clientele.

 

So, no, not even all public schools meet this ideal. But those that don’t at least contain the possibility of change.

 

We could integrate all public schools. We could never integrate our charter, voucher and private schools. That goes against their essential mission. They are schools made to discriminate. Public schools are meant to be all inclusive. Every one could meet King’s ideal, if we only cared enough to do it.

 

Which brings me to the second section of King’s early essay that pops off the page:

 

“The function of education, therefore, is to teach one to think intensively and to think critically. But education which stops with efficiency may prove the greatest menace to society. The most dangerous criminal may be the man gifted with reason, but with no morals.”

 

Seventy one years ago, King was warning us about the situation we suffer today.

 

When we allow academics to be distinct from character and understanding, we put ourselves at the mercy of leaders with “reason, but with no morals.”

 

We put ourselves and our posterity in the hands of those like President Donald Trump, the fruit of a fully private education.

 

Racism and privilege become the defining characteristics of a class without character, in King’s sense.

 

If we want to reclaim what it means to be an American, if we want to redefine ourselves as those who celebrate difference and defend civil rights, that begins with understanding the purpose of education.

 

It demands we defend public schools against privatization. And it demands that we transform our public schools into the integrated, equitable institutions we dreamed they could all be.