25 May 2014

Is Illinois Over-Taxed?

With the 5% income tax rate about to expire, many politicians are complaining about how over-taxed Illinois residents are. Illinois residents are unhappy, and more people leave the state than move to it. But when asked why they plan to leave, only 8% cite taxes. Illinois tax collection lags behind most other states, including its neighbors, compared to the size of its economy. When examining all state taxes, not just income, as a percentage of state gross domestic product (GDP), Illinois is behind its neighbors; I added Massachusetts ,which is another flat-tax state.

Screen Shot 2014-05-20 at 12.11.21 PM

As one can see, for some time Illinois lagged significantly behind other states in state tax collection as a percentage of GDP. The Illinois fiscal problems reflect persistent  under-taxation given the size of its economy.


11 April 2014

2016 Election: No Clinton?

While the buzz is that Hilary Clinton will run for the Democratic nomination in 2016, I would not be doing this if I were planning to run:

Since Mrs. Clinton left the State Department in February 2013, she has delivered a steady string of speeches to industry groups that pay her around $250,000 per event.

Usually this is what ex-politicians do after they have left public life, not when they plan to continue in it.  A rival for the nomination will certainly make this an issue.

9 April 2014

Common Sense in NYC

NYC is about to adopt a new promotion policy in K12 schooling that no longer uses state accountability tests as a decisive factor in student promotion from one grade to another. The tests will still be a factor, but a poor test score alone will no longer block advancement to the next grade. Instead, the test score will be one element in a larger portfolio of student work that teachers and principals will review to decide whether a student is ready for promotion to the next grade.

NYC had allowed students with low test scores to use a portfolio to appeal being held back on the basis of test score, but the policy required approval by the teacher, the principal, and the NYC schools chancellor’s designee.

If only Chicago had such a policy in place. Since 1996, Chicago Public Schools has used standardized and later state test scores as a decisive factor in deciding promotions in grades 3, 6, and 8. If the standardized test score was too low, regardless of grades, students had to attend summer school. There was no opportunity to retake the test before summer school, against the recommendation of the Standards for Educational and Psychological Testing standard 13.6:

Students who must demonstrate mastery of certain skills or knowledge before being promoted or granted a diploma should have a reasonable number of opportunities to succeed on equivalent forms of the test or be provided with construct-equivalent testing alternatives of equal difficulty to demonstrate skills or knowledge.



29 March 2014

An Empty Core

Field testing for the multi-state assessments for the Common Core State Standards began last week. Chris Minnich, the executive director of the Council of Chief State School Officers, which oversaw the drafting of the CCSS with the National Governor’s Association, heralded the events in  the Huffington Post:

The promise of Common Core standards, now being implemented in more than 40 states, is that they set a higher bar for what students should know and understand at each grade level. What hasn’t changed is that even under these new standards, individual teachers and schools will continue to decide how best to teach their students to meet them. But because the Common Core standards are more rigorous expectations, carefully researched and pegged to meet or exceed what students in other high-performing countries are learning, they will discourage learning-by-memorizing drills and “teaching to the test.”

The aim is noble, but good intentions do not make good policy. Minnich says that the standards are more rigorous and “require students to develop and demonstrate stronger mastery of core subject matter.” The CCSS do not set out a curriculum for meeting them, though many publishers are offering such curricula ostensibly based on the standards. For those of us who have read them, it is by no means clear that a “higher bar” is set or that they enable “stronger mastery of core subject matter.” Instead, they often ask students to explain how they multiplied after multiplying 5 x 4, to which my daughter answers “by multiplying, duh!” Or asking for “evidence” from a story to explain why one character felt they way he or she did, and by evidence they mean quoting back a line from the text.

But let’s stipulate that the standards do require stronger mastery and higher learning. How then will schools and teachers revise their curricula to meet these new standards? Developing state-wide assessments should be the last step in rolling out the CCSS but instead it is the first step. Since the standards seek a progression from kindergarten—yes, now a half-day kindergarten must meet the same standards as a full-day kindergarten—how are teachers supposed to bring a 6th, 7th, or 8th grader up these higher standards when they were taught under lesser standards in their K-5 years? Either the standards are not substantively different and so this will take only a little extra time and effort,  or the standards are in fact significantly more rigorous and improving student instruction will take much more time and effort. If the latter, then the assessments come too soon. We know that many children do not meet the bar for old standards.

For once, an athletic analogy is appropriate for educational achievement. If an athlete cannot leap over a low hurdle, you do not improve his or her performance by raising the height of the hurdle.

19 February 2014

Common Core Math Questions

In November, PARCC, one of the two multi-state consortia developing Common Core assessments, released some more sample questions. CCSSI Mathematics has  good analysis of them.  As they right note, this is an exercise in factoring, not in multiplication. The 10 x  10 grid prohibits a student from arranging the tiles in a 2 x 24 or 4 x 12 grid. Only the 6 x 8 or 8 x 6 rectangular shape is possible.

Screen Shot 2014-02-19 at 9.17.09 AM

Screen Shot 2014-02-19 at 9.17.23 AM


I have one more criticism to add for his math problem: the user interface is awful. To answer the question, the students need to click on at least 48 squares. This is a tedious task. With a pencil, the students could easily shade them in broad strokes across columns or rows. The user interface does not permit them to select tiles by click-and-drag or using the keyboard. To un-select the tiles, the student must click each one. So an error in selecting them, could require a child to make more than 48 mouse-clicks.

Then there is part C:

Screen Shot 2014-02-19 at 9.37.15 AM


Again, CCSSI Mathematics has a good criticism of the age-inappropriate nature of these CC standards. Setting that aside, as the item is written, the answer range is quite broad. Consider this:Screen Shot 2014-02-19 at 9.41.18 AM

This is a multiplication question that Andy could have written. It doesn’t represent the problem in C, but the item-writers did not require the test subject to do so. All they asked for was an multiplication equation that used ? to represent the number of row. Is this a correct answer? Or this:

Screen Shot 2014-02-19 at 9.50.56 AM

Neither demonstrates any understanding of the problem posed in C, but they are mathematically accurate multiplication equations that Andy could have written. And quite sophisticated ones indeed.

6 November 2013

Christie’s Shallow Victory

The press is proclaiming NJ Gov. Chris Christie to be the candidate that can reach beyond the GOP in 2016 based on his landslide victory on Tuesday. But this is a weak analysis of what actually occurred. Christie won with 1,242,568 votes, a small gain over his 2009 win with 1,174,445 votes, or 48.5% of the ballots cast. On Tuesday, he won 60.5% of the vote. Compare this to the 2012 presidential election in NJ —Mitt Romney lost NJ with 1,477,568 votes, more than Christie earned in either election. In short, Christie’s victory is due less to his appeal to a broad spectrum of voters and more to a lack of serious opposition.

The USA Today story buries a key finding:

Exit polls on Tuesday, with the same voters who handed Christie his epic win, showed that in a matchup with Democrat Hillary Clinton, Christie would lose New Jersey 48%-44%.

Considering that Democratic turnout was low, this is a surprising result. While reporter Martha T. Moore considers this to be due to NJ’s larger Democratic base, it also reflects the limits of Christie’s appeal compared to the alternatives.


20 September 2013

On Interpreting the ISATs, Part I

If you were to design a system to hold public schools accountable to parents of those attending and the citizens whose taxes support the schools, you would only design it the way that Illinois has if you were trying to hide something. Over the past few weeks, Illinois public schools are giving parents of children who were in 3rd through 8th grade in March 2013 the results of their Illinois State Achievement Tests (ISAT) scores. These parents will receive the score their child received in math and reading (and science for grades 4 and 7). In the past, they also received the national percentile rank — how their child ranked compared to the national sample of the children.

Unfortunately, the percentile comes from an independent section of the ISAT, a subset of the SAT10 (no the college application exam, but a different test), and it is not connected to the rest of the ISAT from which the scores that determine whether children exceed, meet, or are below state standards are derived. Indeed, many parents are puzzled as to how their child could fail to meet state standards on the scored ISAT but rank well in the SAT10 percentile. Rather than explain why this would be the case, the Illinois State Board of Education chose not to print forms with the percentile on it. Indeed, one should have grave doubt about the validity of a test that says you fail to meet standards if you score in the 60th percentile or higher nationally.

ISBE has an evasive answer. The ISAT cut-scores — the scores that determine whether a student exceeds, meets, or is below the standards — were raised in 2013. Why? Because ISBE thought they should be recalibrated to reflect how ISBE believes students will perform when tests that are designed to assess how well students meet the Common Core State Standards are developed and administered. Yes, you read that sentence correctly: ISBE believed it knew how students would do on a test that has yet to be completed, let alone taken, and based on that belief, changed the scores for the test that the students actually took.

Now, ISBE will claim that it can predict how well students in grades 3-8 will do on a non-existent test because it knows how well students who took the ISAT in grade 8 in 2008 did on the Praire State Achievement Exam (PSAE) as 11th graders in 2011. And since only 51% of students in 2011 met the “college and career readiness,” something had to be wrong with the cut-scores on the 2008 ISAT in which 82% of the 8th graders met state standards. It never crossed anyone’s mind at ISBE, it seems, that:
1) the set of skills and knowledge that enable 8th grader to meet ISAT standards are not the same set needed to meet standards on the PSAE.;
2) the content of the PSAE, especially the ACT part, is more difficult than the ISATs
3) the CCSS-based tests will not be equatable with the PSAE tests.
4) the instruction in high school does not prepare them for the PSAEs

So ISBE changed cut scores so that a 2008 8th grader who had met the standards on the 2008 ISAT but did not met expectations on the PSAE in 2011 would no longer meet standards on the 2008 ISAT. It used that rescaling to change the cut scores for 2013. The statistical technique that ISBE used to rescale the scores guaranteed that the more students would fall below state standards and fewer students would exceed them. Indeed, the statistical method used — equipercentile equating — is intended to be use on tests that measure the same underlying dimensions (for example, comparing the ACT with the SAT), not comparing dissimilar tests three years apart.

So what is a parent to do with the results of the 2013 ISATs? Not very much. At best you can compare the scores and sub-scores of your child v. the school, district, and state averages. This doesn’t tell you much: you are either lower, equal, or higher than the averages.What data would help? Stay tuned for part II.


4 September 2013

Accountability means not telling you later

Correction: In examining prior cohort data, in turns out that what ISBE and the IIRC call cohort data is not what I describe it as below. In fact, the IIRC “cohort” just compares the scores of all 3rd graders in year to all 4th graders in year t+1. The trouble with doing this is that not all 3rd graders in year t stay in the same school for year t+1 and some new students arrive from other schools. Now CPS did distinguish in this more accurate way as part of its value-added scores; I had assumed they drew on the same data that went into the cohort study. I was wrong. CPS assembled this dataset itself (CPS has not issued value-added scores for 2013 ISATs, probably because they are moving over to NWEA MAP to measure teacher and principal performance). Nevertheless, I do not see why the IIRC site cannot be updated publicly when the data has been there since July. Now parents have to download a bulky Excel file to find out how their school performed.

The 2013 ISAT scores for Illinois schools have been out since July 2013. But the more helpful data, how cohorts of students have performed, has only been available to school officials. The public has no access to it. And the Illinois State Board of Education refuses to release it to the public until the last moment possible under state law, 31 Oct. That date is the deadline by which ISBE must release the data. But the board members apparently don’t know what a deadline is. They have the data, but they have chosen to deny it to the public until the last legal moment.

But, you might object, the 2013 ISAT results at the school and grade level were released in July. True, but that data is not all that useful. Cohort data tells us, for example, how 3rd graders who took the test in 2012 did as 4th graders in 2013.  [It should do this but the IIRC data does not.] The data we have available compares, for example, the set of 3rd graders in 2012 with the set of 3rd graders in 2013. That doesn’t tell us much about the school or the teachers; it tells us about the 3rd graders.


31 August 2013

Democracy and Foreign Policy

This is how US foreign policy is supposed to work when it comes to using force when there is not an imminent danger: the president asks Congress for authorization. Of course, like Presidents George H. and George W. Bush, President Obama denied that he needs this authorization to use force under the Constitution, but what he says he needs and what he and his recent predecessors actually did are two different things. His decision to seek Congressional approval for the use of force reinforces the legal position that the president does not have unilateral authority to use force. It won’t resolve the debate, but it is a step in the right direction.


31 July 2013

Journalist Bites Leakers

There is stark debate between Jeff Toobin, the CNN legal analyst, and Glenn Greenwald, blogger and Guardian columnist over Bradley Manning’s transfer of State Dept. cables to Wikileaks. What makes Cablegate different from other leak cases is that the ship of state did not leak from the top, but from the bottom. It doesn’t get much lower than a private first class.

As Greenwald points out quite rightly, Manning leaked cables marked “Secret” or less (the classification goes confidential > secret > top secret). He did not disclose “Top Secret” documents and certainly not “Secure Compartmentalized Information,” secrets so secret that only specific persons are given access regardless of their clearance level. By contrast, Bob Woodward and scores of other Washington journalists frequently disclose such information when it is leaked to them (though in many, but not all cases, the underlying documents are not available to the public).

What makes the Manning case different from these leaks is that we have most of the documents available. This is similar to the Pentagon Papers, which the NY Times and then the Washington Post published. Like Manning, Daniel Ellsberg did not give the  newspapers the most sensitive documents, the four volumes of diplomatic records. Unlike the Manning case, Ellsberg had been a senior civilian officer in the Defense Dept.

Toobin’s position seems to be that when senior officials leak to establishment journalists, it is OK. But when junior officials leak to upstart journalists, it is a bad thing. Greenwald calls him on this. What Toobin omits in his quasi-defense of leaking is that even senior officials have no authority to leak documents. They cannot foretell what damage their actions might cause when they leak more highly classified knowledge. Particularly sensitive is the National Security Agency’s capacities to monitor electronic communications. Although there is nothing particularly secret about how the NSA is doing what it has done, an inadvertent disclosure of what else the NSA can or cannot do regardless of legal authority could be incredibly damaging to US intelligence efforts in other areas.

Greenwald makes a crucial point: the damage from Cablegate is quite possibly nil. No one has tied any cable that was leaked to any harm. Government officials have backed off assertions that serious damage would occur. Of course, the counterpoint is that the fortunate outcome in this case does not mean that widespread leaking would be more benign in other cases.

The broader issue is how document should be classified. As Greenwald notes — as have many government officials — the classification process is frequently abused to hide the truth and that many documents are classified with no solid grounds for secrecy. Despite repeated calls to reform the classification rules, the process remains convoluted.

Toobin’s support for the Manning verdict is bizarre. Legally, it may be the correct verdict, but Toobin’s derision of Manning is odd for a practicing journalist.  Toobin seems to have forgotten his roots. His first book was a memoir of his work for the Iran-Contra special prosecutor Lawrence Walsh, a job he took shortly out of law school. Toobin had to submit his manuscript to Walsh for review under his terms of employment, and a legal battle broke out when Walsh refused to approve it. Toobin took criticism for seemingly taking the low-level job in order to publish a tell-all book. To now turn and denounce a low-level leaker is the height of chutzpah.