Noonan: Big data misses the mark and messes with policy

Most polls and big data analysts missed the target election night. After at least a year and a half of taking the country’s presidential preference temperature and analyzing voter behavior using complicated algorithms and finger crossing and concluding that Hillary Clinton would win, Donald Trump took the night.
“If ‘big data’ is not that useful for predicting an election, then how much should we be relying on it for predicting civil uprisings in countries where we have an interest or predicting future terror attacks?” asked Patrick Tucker, the author of “The Naked Future: What Happens in a World That Anticipates Your Every Move?”
Let’s add: how much should we rely on it to predict how our students are going to “perform” as they move through their complicated lives?
CMAS/PARCC multiple choice test scores form the basis of many education policy decisions in Colorado that haven’t dented the generally flat results over a decade. In elementary grades at the state level between 2015 and 2016, scores are roughly the same, a percentage up or down across all subjects and grades, with 20 percent to 39 percent of children proficient in math at different levels and 37 percent to 43 percent of children proficient in English/Language Arts (ELA) in different grades.
In middle schools in Cherry Creek School District, ELA scores range from 25 percent to 66 percent proficient in subject matter. The Challenge Academy, its gifted and talented magnet school, achieves 100 percent proficiency in eighth grade ELA only. The Cherry Creek Charter Academy middle school runs from 66 percent to 82 percent proficient.
These “big data” numbers suggest that at CCSD’s comprehensive middle schools, from its most affluent areas to its most low-income, somewhere between 75 percent of children to 34 percent of children can’t read or perform other language tasks very well. It’s time to test that data-based conclusion against what’s in front of our own eyes and ears. Does big data based on small bore multiple choice tests give us information that is particularly useful?
Does all this CMAS data mean that the Cherry Creek/Arapahoe County geography is going to be deficient in people capable of doing work in about 10 years when these children become taxpaying (we hope) adults? Has all the money the state poured into collecting this data, and has all the money the state poured into computer-based – and thus data-based learning – helped our children learn better or teachers teach more meaningfully?
The Advanced Placement Government course taught in Cherry Creek School District covers such complicated and pertinent subjects as political parties, interest groups, lobbyists and political money. Students are given a list of terms and short questions to study. They take a multiple-choice test to determine their knowledge. That’s big-data collection at its worst.
Polls rely on “yes/no” or multiple choice answers to questions that have no “yes/no” or multiple choice answers. When the Legislature and education policy-makers base their decisions on similar data collection, this election’s big-data failure shows how unreliable the big-data basis for conclusions can be.
