Finalized CMAS standardized test results and performance ratings for 2023 will come out this month. School and district performance ratings ostensibly tell the public how our schools and districts are performing based on criteria established by the state legislature and the state Board of Education.

People buy houses based on these ratings. Parents want their children to go to “good schools” in “good districts.” School and district performance ratings are not trivial.

It’s critical, then, to ensure our standardized CMAS testing system delivers accurate, valid information people can rely on to make these kinds of important decisions. Do our CMAS tests deliver on that expectation? Not really.

A deep dive into 2022 CMAS testing outcomes creates lots of questions and no good answers yet. The legislature’s accountability task force that’s supposed to examine our accountability and performance rating system has a lot of work to do. The work should start with how many students actually participate in the testing program. The details of participation at the district level, which reflects the aggregation of CMAS results from test takers in third through eighth grades, reveal a mess.

Stay up to speed: Sign-up for daily opinion in your inbox Monday-Friday

The information here comes from a review of 111 school districts that report enough data to complete performance ratings by the Colorado Department of Education (CDE). Each district is rated on its CMAS testing level that must meet 95% participation of eligible students.

Pressure from parents who do not want their children tested created a two-tier participation calculation: accountability participation and total participation. Accountability participation subtracts parent-excused students from the number of eligible test takers. Total participation calculates the percent of eligible test takers to the total number of students taking the tests.

If the “Accountability” percent is 95% or above, the school or district “Meets” participation and passes that portion of the performance-rating program. But that number doesn’t reflect what actually happens on the ground in schools and districts. It can mislead parents and policy-makers into thinking the CMAS testing system is better than it is.

The most egregious example of how this part of the system doesn’t work occurred in 2022 with Education Re-Envisioned BOCES that offers mostly online education as an alternative learning program. This entity achieved 99.6% Accountability participation based on only 44.7% of students who actually took the tests. Out of 2,963 students who were eligible for testing, 1,325 took the tests. More than 1,600 students received official parent excusals, so by Colorado law, those students are not included in the Accountability calculation. That’s how 44.7% actual student participation turns into 99.6% Meets participation.

In contrast, Cripple Creek-Victor RE-1 received an 85% Accountability participation count and an 85% “Total Participation” count. Those results put Cripple Creek out of compliance with the 95% Meets Participation even though 40% more of its students actually took the tests than with Education Re-Envisioned BOCES.

A bigger example exists when comparing results from Cherry Creek School District and Denver Public Schools. Cherry Creek has a 96.2% Accountability rating based on 76.9% of students who actually took the CMAS tests. The roughly 20% difference between the Accountability and Total calculations is due to parental excusal letters authorizing students to sit out the tests. Denver has an 88.9% Total Participation rate. But not enough parents signed excusal documents so Denver’s Accountability rating is 94.3%, just under what it needed to Meet the standard. Consequently, Denver received a “Low Participation” score.

Mostly, districts do not meet participation levels using the Total Participation numbers. In the “Distinction” category, all 11 districts meet the Accountability target. But by Total Participation, six districts miss the mark leaving about 3,000 students out of the CMAS assessment pool. That number of students exceeds the populations of eight Distinction school districts and almost matches the 3,665 population of the highest-ranking district, Cheyenne Mountain.

Of 49 Accredited districts, only one misses the 95% Meets Accountability participation mark. Only 13 hit the 95% threshold in Total Participation, with the two lowest under 80%. Counting only Accredited districts with 13,000 or more students, 30,390 students didn’t take the tests. Only five of the 49 Accredited districts have more population than the total of non-test-takers. The 30,390 number does not include students who missed the tests in districts with fewer than 13,000 students or students without excusals.

Perhaps the most interesting fact is how participation plays in the district Improvement rating category. These districts are rated the lowest in overall performance in 2022. That is, these are the districts the state labels not good.

Out of 50 districts for which data was available with district populations above 13,000 (Adams 14 and Mapleton were combined and counted as one), 11,467 students didn’t take the tests. Though the number of districts that “failed” the “Meets” standard vastly exceeded the number in Accredited districts, Improvement districts’ overall test-taking student participation was much higher than with Accredited districts.

The difference in ratings occurred because Improvement districts, in general, have more low-income, English Language Learning students whose parents simply didn’t sign parent excusals. In sum, students in districts with Accredited ratings were more frequent test-ditchers than students in Improvement districts.

The total number of students who didn’t take the tests, including all parent-excused non-test-takers in the Distinction category and all non-test-takers with district populations above 13,000 with Accredited and Improvement ratings, is 44,669. Someone must know what the full total number of non-test-takers is, but that information is not easily available on the CDE website. A roughly 44,000-plus student non-test-taking number is large. Only three school districts are larger.

The number is sufficiently high to cast doubt on the validity of using CMAS test results as an accurate measure of how schools and districts are performing. The total of parent excusals is a poll that shows at least 44,000 adults think the testing system is not working.

This data undercuts the foundational premise of rating and ranking districts and perhaps a foundational premise of why people buy houses where they do.

Paula Noonan owns Colorado Capitol Watch, the state’s premier legislature tracking platform.