Wisconsin vs non-collective bargaining states

In the last few weeks, the post below and ones like it have been circulating around Facebook and the internet (see eg: here, in the comments here: , and here ):

Only 5 states do not have collective bargaining for educators and have deemed it illegal. Their ranking on ACT/SAT scores:

South Carolina - 50th
North Carolina - 49th
Georgia - 48th
Texas - 47th
Virginia - 44th

Wisconsin is currently ranked 2nd. Welcome to the race to the bottom.

The statistics have become so widespread that they've been used by the Economist and Paul Krugman at the NYTimes has pulled similar data.

Rebuttals have pointed out that the SAT is a really bad way to judge educational achievement, since in Wisconsin in particular only a small portion of students—those heading for selective colleges—ever take it:

As it turns out, these ratings are bogus. For each state it adds the ranking for SAT scores to the ranking for ACT scores (and it's not even clear what year the data comes from), but it doesn't take into consideration the percentage of the population who take either test. The College Boards specifically warn against doing state-to-state comparisons for the SAT, because in some states all students are required to take the text, while in others only the best students do. Only 4% of Wisconsin students took the SAT in 2010, and since they tend to be the cream of the crop it's not surprising that Wisconsin does well (but third in the nation, not first). On the other hand, 69% of Wisconsin seniors took the ACT in 2010, and Wisconsin comes in 17th in terms of composite ACT scores.

A more thorough debunking of these statistics may be found here; among the revelations is the fact that the data is from 1999. The owner of this site is actually an advocate of "student organizing"; it is to his credit that he has the intellectual honesty to challenge claims that purport to back up his side of the argument.

So, since I like data, playing with data, and living in data, I decided to look at a much better measure of educational achievement—the “Nation’s Report Card.”

For a long time now, the US Department of Education has compiled NEAP scores. These are the results of exams given nationwide to a massive number of students. They show trends, and apples to apples comparisons can be made easily. The exams are given in Mathematics, Reading, Science, Technology and Engineering, Arts, Civics, Economics, Foreign Language, Geography, US History, World History, and Writing. The exams are given to both 4th and 8th graders. A pilot program has started to test 12th graders as well, but none of the states mentioned above are participants.

Iowahawk, the usually comedic web poster, took a break yesterday from satire to mine some of this data--he compared Texas and Wisconsin, with Texas coming out mostly on top.

As it turns out, I was already compiling data from this same source. I decided to focus on the two main parts of the NEAP results: Math and Reading. The reports for 2009 are available here: Math and Reading.

I looked at three categories of student for each grade level: 1) All students, 2) Students eligible for subsidized lunches—which is a short-hand way of looking at children in poverty, and 3) English Language Learners. I also looked at two levels of educational achievement: 1) students meeting basic requirements and 2) students testing as proficient. I took the 50 states and sorted them from best to worst and gave them their numerical rank (I gave states with ties the same rank.) The charts below show the numerical rank of each state with respect to the other 49. (Some students, such as people with reading disabilities, are granted special accommodations for the exam, I used those scores for the data below.)

It is clear that Wisconsin does quite well, especially in math, but the results are really not much to crow about. Wisconsin’s reading scores are actually surprisingly low. The best ranking for Wisconsin in math in any of the categories came in at a rank of 8—that was for 8th grade English language learners testing at the basic level. In reading Wisconsin achieved a rank of two for educating ELL students in 8th grade to basic levels. It should be noted, though, that ELL students do very badly overall.

Here is what I found:

4th Grade Math



For all students, and for these six states—remember Virginia, the Carolinas, Texas, and Georgia are used for the comparison because they do not allow their teachers to collectively bargain—Virginia, North Carolina and Texas are all better at getting kids a basic understanding of 4th grade math than Wisconsin. Wisconsin is the best of the bunch at getting kids to a proficient level, but 13 other states in the union actually do better than the cheese state, putting the dairyland in the second quintile. The percentage of Wisconsin students ranking at basic and proficient levels are: 85% and 45% respectively.

For students in poverty, Wisconsin gets a ranking near the middle of the 50 states: 22 for students reaching basic levels, and a better 19 for students reaching proficient. In Wisconsin 73% of students in poverty can do math at a basic level and 24% are proficient.

For English language learners, Wisconsin is well above the median for the 50 states with rankings of 14 and 13, however, among the six states presented here, Wisconsin actually does poorly, only Georgia does worse. In Wisconsin 66% of ELL students in 4th grade are at the basic level and 15% are proficient. The ELL scores, though, do have a caveat: not all states report these results. On this exam nine states did not report: Alabama, Maine, Mississippi, Missouri, North Dakota, South Dakota, Vermont, West Virginia, and Wyoming. Whether those states would change the rankings can not be known from this data.

8th Grade Math



For 8th graders, Wisconsin does well overall, getting more kids to the basic and proficient levels than the other five states, and makes the top ten for overall basic skills. The percentage of students scoring at basic and proficient are a respectable 79% and 39%. Texas shows itself to be very good at educating children in poverty, getting a basic rank of 3 (they are tied for third place with South Dakota and Massachusetts, and North Dakota and Montana do better,) and a proficient rank of 9. (Wisconsin’s scores are 60% of student in poverty achieve the basic level, 20% proficient.) South Carolina is again the best at educating English language learners, tying with Virginia for the number 2 spot at the basic level and getting the second rank for proficient as well. Wisconsin does well (45% at basic, 9% at proficient), but lags behind Virginia, North Carolina, and South Carolina—Georgia did not report these scores, neither did: Alabama, Delaware, Iowa, Kentucky, Louisiana, Maine, Mississippi, Missouri, New Hampshire, North Dakota, South Dakota, Tennessee, Vermont, West Virginia, Wyoming (16 states did not report ELL scores.)

4th Grade Reading



Wisconsin does surprisingly poorly on this exam. For a state that is said to have a good educational system, scoring below or at the median for reading among all students is not the educational beacon it’s been made out to be. Only 67% of Wisconsin 4th graders read at a basic level, only 33% are proficient. For students in poverty, the numbers are even worse. For the first time in the data I’ve looked at, Wisconsin falls into the bottom quintile of states, wracking up a pathetic rank of being the 42nd best (worst!) state in which to be a student in poverty. Of these six states Wisconsin is the worst at educating students in poverty (with SC tying at the proficient level.) In Wisconsin, less than half—only 46% of 4th graders in poverty read at a basic level, and only 15% are proficient. My own state of California is in the dead-last position, with only 38% of students reaching the basic level, and only 10% reaching proficiency. Again, South Carolina seems to understand how to teach English language learners, or their cohort of ELL students is different from the ones in other states. They have the honor of being at the top of the league in ELL. Wisconsin is better than Georgia among these six states, but is the second last in both basic and proficient ELL students. (Eleven states did not report their ELL scores.) Among ELL 4th graders in Wisconsin only 31% are at basic and 8% are proficient.

8th Grade Reading



By comparison to the other five states in this dataset, Wisconsin does well at educating students in 8th grade reading. However, with the exception of ELL students, Wisconsin still isn’t doing very well. Wisconsin has a 50-state rank of 19 and 15 for educating all students to the basic and proficient levels respectively. In Wisconsin 78% of 8th grade students reach the basic level of reading, and 34% are proficient readers. Again Wisconsin is below the national median at educating students in poverty to the basic level of competence (61% of students in poverty achieve basic scores), but does slightly better than the median at getting kids to proficiency (18%). The ELL exam is again hampered by the fact that not all states report these results; in this case nearly half (23) failed to report. Among Wisconsin’s ELL students in 8th grade, only 49% can read at a basic level and only 7% are proficient.

Conclusions: Don’t send your kids to school in Georgia! Georgia did poorly across the board, though in a couple categories Wisconsin actually did worse. Wisconsin comes in at being a better-than-average state for education, but not if you are a student in poverty. Of the 24 categories presented here, Wisconsin was the best of the six states in nine of them, Virginia: eight, South Carolina: seven (all ELL), Texas: four, North Carolina: one, and Georgia: none.

Does Wisconsin’s collective bargaining state shine above the ones without it? Not according to this data. Unlike the skewed SAT results, this shows these states doing some things well and some things poorly. None are shown off in a really good light, and Wisconsin’s reading achievement is surprisingly bad when compared to its reputation as a great state for education.