If you were to design a system to hold public schools accountable to parents of those attending and the citizens whose taxes support the schools, you would only design it the way that Illinois has if you were trying to hide something. Over the past few weeks, Illinois public schools are giving parents of children who were in 3rd through 8th grade in March 2013 the results of their Illinois State Achievement Tests (ISAT) scores. These parents will receive the score their child received in math and reading (and science for grades 4 and 7). In the past, they also received the national percentile rank — how their child ranked compared to the national sample of the children.
Unfortunately, the percentile comes from an independent section of the ISAT, a subset of the SAT10 (no the college application exam, but a different test), and it is not connected to the rest of the ISAT from which the scores that determine whether children exceed, meet, or are below state standards are derived. Indeed, many parents are puzzled as to how their child could fail to meet state standards on the scored ISAT but rank well in the SAT10 percentile. Rather than explain why this would be the case, the Illinois State Board of Education chose not to print forms with the percentile on it. Indeed, one should have grave doubt about the validity of a test that says you fail to meet standards if you score in the 60th percentile or higher nationally.
ISBE has an evasive answer. The ISAT cut-scores — the scores that determine whether a student exceeds, meets, or is below the standards — were raised in 2013. Why? Because ISBE thought they should be recalibrated to reflect how ISBE believes students will perform when tests that are designed to assess how well students meet the Common Core State Standards are developed and administered. Yes, you read that sentence correctly: ISBE believed it knew how students would do on a test that has yet to be completed, let alone taken, and based on that belief, changed the scores for the test that the students actually took.
Now, ISBE will claim that it can predict how well students in grades 3-8 will do on a non-existent test because it knows how well students who took the ISAT in grade 8 in 2008 did on the Praire State Achievement Exam (PSAE) as 11th graders in 2011. And since only 51% of students in 2011 met the “college and career readiness,” something had to be wrong with the cut-scores on the 2008 ISAT in which 82% of the 8th graders met state standards. It never crossed anyone’s mind at ISBE, it seems, that:
1) the set of skills and knowledge that enable 8th grader to meet ISAT standards are not the same set needed to meet standards on the PSAE.;
2) the content of the PSAE, especially the ACT part, is more difficult than the ISATs
3) the CCSS-based tests will not be equatable with the PSAE tests.
4) the instruction in high school does not prepare them for the PSAEs
So ISBE changed cut scores so that a 2008 8th grader who had met the standards on the 2008 ISAT but did not met expectations on the PSAE in 2011 would no longer meet standards on the 2008 ISAT. It used that rescaling to change the cut scores for 2013. The statistical technique that ISBE used to rescale the scores guaranteed that the more students would fall below state standards and fewer students would exceed them. Indeed, the statistical method used — equipercentile equating — is intended to be use on tests that measure the same underlying dimensions (for example, comparing the ACT with the SAT), not comparing dissimilar tests three years apart.
So what is a parent to do with the results of the 2013 ISATs? Not very much. At best you can compare the scores and sub-scores of your child v. the school, district, and state averages. This doesn’t tell you much: you are either lower, equal, or higher than the averages.What data would help? Stay tuned for part II.