The Six Biggest Problems with Data-Driven Instruction

0

 

 

“On the dangers of being data-driven: Imagine driving from A to B ignoring the road, the weather, the traffic around you… only staring at the gauges on the dashboard.”

 – Educator Dan McConnell

 

 

“Make your instruction data-driven.”

 

If you’re a public school teacher, you’ve probably heard this a hundred times.

 

In the last week.

 

Principals and administrators use that word – “data-driven” – as if it were inscribed over the front doors of the schoolhouse in stone.

 

The idea goes like this: All lessons should be based on test scores.

 

Students take the federally mandated standardized test. Your job is to make sure they get the best possible score. Your class is nothing but a way station between standardized tests.

 

Pretest your students and then instruct them in such a way that when they take the test again, they’ll get the best possible score.

 

It’s total nonsense. And it doesn’t take much to see why.

 

No teacher should ever be data-driven. Every teacher should be student-driven.

 

You should base your instruction around what’s best for your students – what motivates them, inspires them, gets them ready and interested in learning.

 

To be sure, you should be data-informed – you should know what their test scores are and that should factor into your lessons in one way or another – but test scores should not be the driving force behind your instruction, especially since standardized test scores are incredibly poor indicators of student knowledge.

 

No one really believes that the Be All and End All of student knowledge is children’s ability to choose the “correct” answer on a multiple-choice test. No one sits back in awe at Albert Einstein’s test scores – it’s what he was able to do with the knowledge he had. Indeed, his understanding of the universe could not be adequately captured in a simple choice between four possible answers.

 

As I see it, there are at least six major problems with this dependence on student data at the heart of the data-driven movement.

 

So without further ado, here is a sextet of major flaws in the theory of data-driven instruction:

 

 

 

  1. The Data is Unscientific

    When we talk about student data, we’re talking about statistics. We’re talking about a quantity computed from a sample or a random variable.

    As such, it needs to be a measure of something specific, something clearly defined and agreed upon.

    For instance, you could measure the brightness of a star or its position in space.

    However, when dealing with student knowledge, we leave the hard sciences and enter the realm of psychology. The focus of study is not and cannot be as clearly defined. What, after all, are we measuring when we give a standardized test? What are the units we’re using to measure it?

    We find ourselves in the same sticky situation as those trying to measure intelligence. What is this thing we’re trying to quantify and how exactly do we go about quantifying it?

    The result is intensely subjective. Sure we throw numbers up there to represent our assumptions, but – make no mistake – these are not the same numbers that measure distances on the globe or the density of an atomic nucleus.

    These are approximations made up by human beings to justify deeply subjective assumptions about human nature.

    It looks like statistics. It looks like math. But it is neither of these things.

    We just get tricked by the numbers. We see them and mistake what we’re seeing for the hard sciences. We fall victim to the cult of numerology. That’s what data-driven instruction really is – the deepest type of mysticism passed off as science.

    The idea that high stakes test scores are the best way to assess learning and that instruction should center around them is essentially a faith based initiative.

    Before we can go any further, we must understand that.

  2. It Has Never Been Proven Effective

    Administrators and principals want teachers to base their instruction around test scores.

    Has that ever been proven an effective strategy for teachers planning lessons or the allocation of resources? Can we prove a direct line from data to better instruction to better test scores?

    The answer is an unequivocal NO.

    In a 2007 study from Gina Schuyler Ikemoto and Julie A. Marsh published in the Yearbook for the National Society for the Study of Education, data driven instruction actually was found to have harmful effects on educator planning and, ultimately, student learning.

    Researchers looked at 36 instances of data use in two districts, where 15 teachers used annual tests to target weaknesses in professional development or to schedule double periods of language arts for English language learners. The result was fewer instances of collective, sustained, and deeper inquiry by groups of teachers and administrators using multiple data sources – test scores, district surveys, and interviews – to reallocate funds for reading specialists or start an overhaul of district high schools.

    Teachers found the data less useful if it was not timely – standardized test scores are usually a year old by the time they get to educators. Moreover, the data was of less value if it did not come with district support and if instructors did not already buy into its essential worth.

    In short, researchers admitted they could not connect student achievement to the 36 instances of basic to complex data-driven decisions in these two districts.

    But that’s just one study.

    In 2009, the federal government published a report (IES Expert Panel) examining 490 studies where schools used data to make instructional decisions.

    Of these studies, the report could only find 64 that used experimental or quasi-experimental designs. Of these it could find only six – yes, six – that met the Institute of Education Sciences standard for making causal claims about data-driven decisions to improve student achievement.

    And when examining these six studies, the panel found “low evidence” to support data-driven instruction. They concluded that the theory that data-driven instructional decisions improve student test scores has not been proven in any way, shape or form.

  3. It’s Harmful – The Stereotype Threat and Motivation

    Data-driven instruction essentially involves grouping students based on their performance on standardized tests.

    You put the low scorers HERE, the students on the bubble who almost reached the next level HERE, and the advanced students HERE. That way you can easily differentiate instruction and help meet their needs.

    However, there is a mountain of psychological research showing that this practice is harmful to student learning. Even if you don’t put students with different test scores in different classes, simply informing them that they belong to one group or another has intense cognitive effects.

    Simply being told that you are in a group with lower test scores depresses your academic outcomes. This is known as the stereotype threat.

    When you focus on test scores and inform students of where they fall on the continuum down to the percentile – of how far below average they are – you can trigger this threat. Simply tracking students in this way can actually make their scores worse.

    It can create negative feelings about school, threatening students’ sense of belonging, which is key to academic motivation.

    But it’s not just the low scorers who are harmed. Even the so-called “advanced” students can come to depend on their privileged status. They define themselves by their achievement, collecting prizes, virtual badges and stickers. These extrinsic rewards then transform their motivation from being driven by the learning and the satisfaction of their curiosity to depending on what high achievement gets them, researchers have found.

    In short, organizing all academics around tests scores is a sure way to lower them.

  4. The Data Doesn’t Capture Important Factors

    Data-driven instruction is only as good as the data being used. But no data system can be all inclusive.

    When we put blinders on and say only these sorts of factors count, we exclude important information.

    For instance, two students do the same long-term project and receive the same grade. However, one student overcame her natural tendency to procrastinate and learned more than in past projects. The other did not put forth his best effort and achieved lower than his usual.

    If we only look at the data, both appear the same. However, good teachers can see the difference.

    Almost every year I have a few students who are chronically tardy to class. A good teacher finds out why – if this is because they aren’t making the best use of the class interval or if they have a greater distance to travel than other students. However, if we judge solely on the data, we’re supposed to penalize students without considering mitigating factors. That’s being data-driven – a poor way to be a fair teacher.

    It has been demonstrated repeatedly that student test scores are highly correlated with parental income. Students from wealthier parents score well and those from more impoverished families score badly. That does not mean one group is smarter or even more motivated than the other. Living in poverty comes with its own challenges. Students who have to take care of their siblings at home, for instance, have less time for homework than those who have nothing but free time.

    A focus solely on the data ignores these factors. When we’re admonished to focus on the data, we’re actually being told to ignore the totality of our students.

  5. It’s Dehumanizing

    No one wants to be reduced to a number or a series of statistics.

    It is extremely insulting to insist that the best way for teachers to behave is to treat their students as anything other than human beings.

    They are people with unique needs, characteristics, and qualities, and should be treated accordingly.

    When one of my students does an amazing job on an assignment or project, my first impulse is not to reduce what they’ve done to a letter grade or a number. I speak my approbation aloud. I write extensive comments on their papers or conference with them about what they’ve done.

    Certainly, I have to assign them a grade, but that is merely one thing educators do. To reduce the relationship to that – and only that – is extremely reductive. If all you do is grade the learner, you jeopardize the learning.

    Every good teacher knows the importance of relationships. Data-driven instruction asks us to ignore these lessons in favor of a mechanistic approach.

    I’m sorry. My students are not widgets and I refuse to treat them as such.

    I am so sick of going to conferences or faculty meetings where we focus exclusively on how to get better grades or test scores from our students. We should, instead, focus on how to see the genius that is already there! We should find ways to help students self-actualize, not turn them into what we think they should be.

    At this point, someone inevitably says that life isn’t fair. Our students will have to deal with standardized tests and data-driven initiatives when they get older. We have to prepare them for it.

    What baloney!

    If the real world is unfair, I don’t want my students to adjust to that. I want to make it better for them.

    Imagine telling a rape victim that that’s just the way the world is. Imagine telling a person brutalized by the police that the world is unfair and you just have to get used to it.

    This is a complete abdication not just of our job as teachers but our position as ethical human beings.

    Schools are nothing without students. We should do everything we can to meet their needs. Period.

  6. It’s Contradictory – It’s Not How We Determine Value in Other Areas

    Finally, there is an inherent contradiction that all instruction must be justified by data.

    We don’t require this same standard for so many aspects of schooling.

    Look around any school and ask yourself if everything you see is necessarily based on statistics.

    Does the athletic program exist because it increases student test scores? Does each student lunch correlate with optimum grades? Do you have computers and iPads because they have a measureable impact on achievement?

    Some administrators and principals DO try to justify these sorts of things by reference to test scores. But it’s a retroactive process.

    They are trying to connect data with things they already do. And it’s completely bogus.

    They don’t suddenly believe in football because they think it will make the team get advanced scores. They don’t abruptly support technology in the classroom because they think it will make the school achieve adequate yearly progress.

    They already have good reasons to think athletics helps students learn. They’ve seen participation in sports help students remain focused and motivated – sometimes by reference to their own lives. Likewise, they’ve seen the value of technology in the classroom. They’ve seen how some students turn on like someone flipped a switch when a lesson has a technological component.

    These aren’t necessarily quantifiable. They don’t count as data but they are based on evidence.

    We come to education with certain beliefs already in place about what a school should do and others are formed based on the empiricism of being there, day-in, day-out. “Data” rarely comes into the decision making process as anything but a justification after the fact.


    And so we can firmly put the insistence on data-driven instruction in the trash bin of bad ideas.

    It is unscientific, unproven, harmful, reductive, dehumanizing and contradictory.

    The next time you hear an administrator or principal pull out this chestnut, take out one of these counterarguments and roast it on an open fire.

    No more data-driven instruction.

    Focus instead on student-driven learning.

 

Don’t let them co-opt you into the cult of numerology. Remain a difference-maker. Remain a teacher.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-3

FBI Warns EdTech Puts Student Safety at Risk

students-99506_960_720

 

Pandora’s box may not be a heavy wooden chest.

 

It might just be a laptop or an iPad.

 

The Federal Bureau of Investigations (FBI) issued a warning late last week about the proliferation of educational technologies (EdTech) in schools and the dangers they pose to student safety and privacy.

 

The devices and applications singled out by US domestic intelligence services all involve software that asks for and saves student input.

 

This includes classroom management tools like Class Dojo and eBackpack as well as student testing and remediation applications like Classroom Diagnostic Tools and StudyIsland.

 

The alert cautions schools and parents that the widespread collection of student data involved in these applications could cause safety concerns if the information is compromised or exploited.

 

It’s just such technologies that have proliferated at a breakneck pace in districts throughout the country while legislation, cybersecurity and even awareness of the danger has lagged behind.

 

The Bureau is concerned about EdTech services because many are “adaptive, personalized learning experiences” or “administrative platforms for tracking academics, disciplinary issues, student information systems, and classroom management programs.”

 

The Bureau clarified that use of EdTech applications was not, in itself, the problem. They are a tool, and like any tool, they can be used for good or ill.

 

Advocates say when used correctly, these systems offer the potential for student collaboration, and for teachers to collect and compare information about students to help design better lessons. On the other hand, critics warn that the problems with most EdTech is essential to the way it operates and that it is designed more for the purposes of student data mining and profitization by corporations than with academic success for children in mind.

 

In either case, the dangers posed by such devices are indisputable.

 

The amount and kinds of data being collected about students has not been well publicized or understood. Many school boards, administrators, teachers, parents and students may be using EdTech without a full appreciation of how it often surreptitiously collects data on children.

 

Analysts listed several types of student particulars being stockpiled:

 

  • “personally identifiable information (PII);

 

  • biometric data;

 

  • academic progress;

 

  • behavioral, disciplinary, and medical information;

 

  • Web browsing history;

 

  • students’ geolocation;

 

  • IP addresses used by students; and

 

  • classroom activities.”

 

This is highly sensitive information that could be extremely harmful to students if it fell into the wrong hands.

 

Pedophiles could use this data to find and abduct children. Criminals could use it to blackmail them. It could even be sold to unscrupulous corporations or exploited by other children to bully and harass classmates.

 

These concerns are not merely academic. Data breaches have already occurred.

 

According to the FBI:

 

“…in late 2017, cyber actors exploited school information technology (IT) systems by hacking into multiple school district servers across the United States. They accessed student contact information, education plans, homework assignments, medical records, and counselor reports, and then used that information to contact, extort, and threaten students with physical violence and release of their personal information. The actors sent text messages to parents and local law enforcement, publicized students’ private information, posted student PII on social media, and stated how the release of such information could help child predators identify new targets.”

 

Though the Bureau did not mention them by name, it cited two specific cybersecurity failures in 2017 at two large EdTech companies where millions of student’s private information became publicly available.

 

The FBI seems to be referring to Edmodo and Schoolzilla. Hackers stole 77 million user accounts from Edmodo last year and sold that data to a “dark web” marketplace. Likewise, Schoolzilla stored student information on a publicly accessible server, according to a security researcher, thereby exposing the private information of approximately 1.3 million children.

 

It was these data breaches that prompted the Department of Education to issue a Cyber Advisory alert in October of 2017 warning that cyber criminals were targeting schools that had inadequate data security. Criminals were trying to gain access to this sensitive material about students to “shame, bully, and threaten children.”

 

In February, another cybercrime was reported by the Internal Revenue Service. The agency released an “urgent alert” about scammers targeting school districts with W-2 phishing schemes.

 

Of particular concern to the FBI are school devices connected directly to the Internet. They offer hackers an immediate link to private student information. This is true even if a school device is in the student’s home.

 

Tablets, laptops or monitoring devices such as cameras or microphones could be exploitable by tech savvy criminals – especially since many EdTech programs allow remote-access capabilities without the user even being aware of what is happening.

 

Indeed, some EdTech companies collect information on students’ feelings; become student data brokers and use voice-activated speakers in the classroom.

 

 

The FBI listed several recommendations for parents and families:

 

  • “Research existing student and child privacy protections of the Family Educational Rights and Privacy Act (FERPA), the Protection of Pupil Rights Amendment (PPRA), the Children’s Online Privacy Protection Act (COPPA), and state laws as they apply to EdTech services.

  • Discuss with their local districts about what and how EdTech technologies and programs are used in their schools.

 

  • Conduct research on parent coalition and information-sharing organizations which are available online for those looking for support and additional resources.

 

  • Research school-related cyber breaches which can further inform families of student data vulnerabilities.

 

  • Consider credit or identity theft monitoring to check for any fraudulent use of their children’s identity.

 

  • Conduct regular Internet searches of children’s information to help identify the exposure and spread of their information on the Internet.”

 

The warning concluded by asking those who have information about cybercrimes or who have been victimized by data breaches to file a complaint with the Internet Crime Complaint Center at www.ic3.gov.

 

For some, the FBI’s warning highlights the need for greater education funding for cybersecurity.

 

“While other industries are investing in greater IT security to protect against cyber threats, many schools are facing budget constraints that result in declining resources for IT security programs,” staff members from the Future of Privacy Forum wrote on their blog. “Schools across the country lack funding to provide and maintain adequate security,” they wrote.

 

Others see the problem as one of antiquated legislation not keeping up with changing technologies.

 

The FBI memo highlights a need for Congress to update federal student privacy laws, said Rachael Stickland, co-founder and co-chair of the Parent Coalition for Student Privacy.

 

At present, these issues are covered haphazardly from state to state, a situation Strickland finds intolerably inadequate.

 

The main federal law safeguarding student data privacy, The Family Educational Rights and Privacy Act, should be updated, she said. It must address exactly which ways districts should be allowed to share data with EdTech companies.

 

“What we need is a comprehensive approach at the federal level to address these outdated federal laws,” she said. “We really need these modernized to address not only the cybersecurity threats but also the potential misuse of this data.”

 

However, some critics are skeptical of the very use of EdTech in the classroom, at all.

 

They see the potential dangers being so massive and difficult to guard against that any remedies to fix the problem would be counterproductive.

 

Instead of spending millions or billions more on cybersecurity measures, we should use any additional funding for more essential budget items like teachers, tutors, nurses and counselors.

 

The goal is to provide a quality education – not to provide a quality education with increasingly complex technologies. If the former can be done with the latter, that’s fine. But if we have to choose between the two, we should go with proven methods instead of the latest fads.

 

EdTech corporations are making staggering profits off public schools while almost every other area goes lacking. In the nexus of business and industry, we may be losing sight of what’s best for students.

 

The question is have we already opened Pandora’s box, and if we haven’t, can we stop ourselves from doing so?

 


Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-3

STEM Education Severs the Arts from the Sciences

right-brain-left-brain

  

What’s the most effective way to dumb down a nation?

 

 

Focus on How without Why.

 

 

That’s really the biggest problem with the pedagogical fad of STEM education.

 

 

There’s nothing objectively wrong with teaching science, technology, engineering and math – the disciplines that make up STEM.

 

 

In many cases, doing so is essential to a well-rounded education.

 

 

But therein lies the problem – you can’t have a well-rounded education if you purposely leave out some of the most vital aspects of knowledge.

 

 

Where’s the art? Where’s the literature? Where’s the social studies, government, citizenship, drawing, painting, music – heck! Where’s the philosophical understanding of life, itself?

 

 

STEM initiatives often involve creating two tiers of school subjects. You have the serious disciplines that will earn you respect and a job. And you have the soft, mamby pamby humanities that are no good to anyone.

 

 

The problem is one of focus not content.

 

 

Corporate-minded bureaucrats who know nothing of human psychology, child development or education look solely at standardized test scores and get hysterical.

 

 

The U.S. is falling behind other nations – especially in science and math, they say. So we must do whatever we can to bring those test scores up, Up, UP!

 

 

Yet they have never bothered to see that our student test scores have never been at the top of the pack for all the decades we’ve been making international comparisons.

 

 

We started contrasting multiple choice assessment results for 13-year-olds in a dozen countries back in 1964. And ever since, America has always been right in the middle.

 

Yet for those five decades we’ve dominated the world in science, technology, research and innovation.

 

 

In that time we sent the first people to the moon, mapped the human genome, and invented the Internet – all while getting middling test scores.

 

 

In short, standardized assessments are a fantastically unreliable indicator of national success, just as they are poor indicators of individual learning.

 

 

We’ve never been a nation content with picking our answers from four options – A,B,C,D. We blaze new paths!

 

 

But number obsessed fools have convinced a public blinded by sports statistics that these tests mean our kids are deficient. And the only cure is to put on blinders and focus almost exclusively on those subjects most featured on the tests.

 

Even reading and writing are only valuable if they let us guess what a normalized reader is supposed to comprehend from a given passage and if they allow us to express ourselves in the most rudimentary and generic ways.

 

 

This is exactly what they do in countries with the highest test scores – countries that are LESS innovative than the U.S.

 

 

Asian countries from Singapore to South Korea to India are not blind to this irony. While we are trying to imitate them, they are trying to imitate the kind of broad liberal arts education in which we used to pride ourselves.

 

“Many painters learn by having fun,” said Jack Ma, founder of one of China’s biggest Internet companies Alibaba.

 

“Many works of art and literature are the products of having fun. So, our entrepreneurs need to learn how to have fun, too.”

 

Ma worries that his country is not as innovative as those in the West because China’s educational system focuses too much on the basics and does not foster a student’s complete intelligence, allowing him or her to experiment and enjoy the learning process.

 

In other words, no matter how good you are at math and science, you still need to know how to learn, think and express yourself.

 

To be fair, these criticisms of STEM are not new.

 

Even global pundits like Fareed Zakaria have made similar arguments.

 

The result has been a hasty addition – change STEM to STEAM by adding in the arts.

 

Unfortunately, this hasn’t always worked out for the best.

 

Most of the time, the arts component is either an after thought or merely a sweetener to get students interested in beginning the journey – a journey that is all STEM all the time.

 

There is still an education hierarchy with the sciences and math at the top and the humanities and social studies at the bottom.

 

This is extremely unfortunate and will cause long-term detrimental effects to our society.

 

For instance, we pride ourselves in being democratically ruled. Political power does not come from authority, it comes from the consent of the governed.

 

This requires a public that knows how to do more than just add and subtract. Voters need to understand the mechanisms of government so they grasp their rights. They need a knowledge of history so they don’t repeat the mistakes of the past. They need to grasp human psychology, anthropology, and sociology to understand how people work in groups and individually.

 

Moreover, as human beings, they need the humanities. People have thoughts and feelings. They need to know how to express those thoughts and feelings and not just by writing a five-paragraph essay. They need to be able to create works of art. They need to be able to write a story or poem. They need to be able to manipulate images. They need to understand and create music.

 

Without these things, it can be difficult to become fully actualized people.

 

That used to be the goal of education. Provide students with the tools to become the best version of themselves.

 

But this focus on STEM and STEAM only endeavors to make them the best cogs the workforce needs.

 

We have relinquished our commitment to students and replaced it with a commitment to business and industry.

 

The idea is that schools owe the job market workers. That could not be further from the truth. We owe our students the tools that will help them live the best lives. And employment is only one small facet of that goal.

 

I’m not saying we shouldn’t teach math and science. We should – we MUST. But those can’t be prioritized over and above other essential human endeavors.

 

We need to fund and encourage a broad liberal arts education for all students. As they get older and move on to post-secondary studies including industrial arts they will inevitably specialize in areas that they find most interesting.

 

But until then, it is our job to give them every opportunity to learn – not to mold them into future wage slaves or boost national pride with arbitrary and meaningless test scores.


 

Like this post? I’ve written a book, “Gadfly on the Wall: A Public School Teacher Speaks Out on Racism and Reform,” now available from Garn Press. Ten percent of the proceeds go to the Badass Teachers Association. Check it out!

WANT A SIGNED COPY?

Click here to order one directly from me to your door!

book-2