Board Meeting Wrap-Up No. 1: Cherry-picking Bill Betzen

It's a neat trick to say everyone selects misleading data when you do it all the time.

cherry-picking-numbers

Trying something new this week, since I’m almost a week late with the wrap up (holiday weekend, D Magazine column due, gremlin attack) and since I’ve gotten a lot of feedback that boils down to this: I get it. You don’t like Foreman and Nutall. What else you got?

Fair point. Also incorrect. It’s not that they’re bad people. I think they really believe they’re helping kids. I also believe their actions exemplify the very thing the Dallas ISD board has done for 20 years: meddle in district matters in a way that ultimately harms kids, no matter the intention. So I want to explain why that is, and I’ll use last week’s board meeting as an example later this week. (Although I have some even better examples coming down the road.)

Overall, I want to do a couple separate blog posts that hit only a few key issues, to better give context as to what concerns me and what I think is substantive board discussion. This week, I have two main themes to highlight from last week’s board meeting: cherry picking data, and why DISD is different than City Hall. First, the cherry picking. To the (two) bullet points!

• We started this past week’s meeting with guest speaker Bill Betzen, who uses such occasions to basically run an add for his blog. I’ve been putting off pointing out Bill’s obvious bias against Mike Miles and the district for several reasons, including but not limited to:

1. I like Bill. I think he has his heart in the right place. And we agree on almost everything else — redistricting, City Hall oversight, the silly Trinity River toll road, problems in statewide education funding and oversight — of substance in the city. The reason we disagree so strongly on Dallas ISD is I think a fascinating study in the strange bedfellows of urban school reform, which I’m going to look at in-depth in a later post.

2. It calls attention to his misguided attempts to prove Miles’ reform efforts as a catastrophe for the district, which aligns with the drum beat pounded out every day by special interest groups and anti-Miles board members, but which in no way reflects the good, bad, and in between of reality.

3. It just seems mean.

That said, his speech typified why you should never, ever listen to Bill when it comes to Dallas ISD. The irony is that he says you should listen to him because otherwise you’re susceptible to people “cherry-picking” data. As he acknowledges, he has himself been accused of doing this. That’s because he does, which he proves right away.

Bill, basically reading the same words as you can find on his website, led with this statement regarding the TEA 2014 accountability ratings:

With nine more DISD schools rated “Improvement Required” (going from 34 to 43) in 2014 than in 2013, Dallas has the sad distinction of having more new failing schools than any district in Texas! On average the rest of Texas schools improved, with 5 percent fewer failing schools — that is 5 percent fewer schools rated as “Improvement Required.”

There’s a lot to unpack here, so stay with me:

First of all, yes, the number of DISD schools rated as IR went up, from 34 to 43. That is unquestionably a bad thing, as I’ve noted before. If you were at the board briefing three weeks ago, you saw a detailed discussion of this in which top administrators admitted this was bad, talked about the plan to get better, and noted specifically in what areas DISD was failing. (In general, reading scores for African American students, who make up 23 percent of the district, need to improve.) There are reasons for this which we went over already, but basically the hope is that the level of instruction will catch up to the increasing demands of the STAAR statewide test. You may, as trustee Joyce Foreman did, see that explanation as an excuse. Fine. Reasonable people can disagree on that. But it’s an excuse that a) didn’t stop ANYONE from saying the results were unacceptable, and b) was recently backed up by Texas Education Commissioner Michael Williams, who, as the Texas Tribune put it, “told state lawmakers [in August] that classroom instruction in the state had failed to meet the rigor demanded by the new tests.”

But I’m not here to defend bad results. (Even though the anti-Miles status-quo folks seem to continually put me in the position of seemingly doing so.) We’re here to see if Bill is cherry-picking. The answer is, yes and no. No in that he’s saying results are bad, and they are. Yes if you look at how he picks and chooses which numbers to highlight, and why.

So Bill says Dallas has more new failing schools than any district in the state. Of course. We’re far larger than any district except Houston, which decreased its number of IR schools from 58 to 44. (More on Houston in a moment.) No duh.

He then cites a peculiar statistic, one I hadn’t seen anywhere else: “The rest of Texas schools improved, with 5 percent fewer failing schools [statewide].”

To examine this, we can just look at these accountability summaries from TEA: Here is 2013 data, and here is 2014.

You’ll notice that the numbers are broken down by district, and by campus — sometimes with all schools tested, and sometimes with charters excluded. Bill chooses to compare campus-level numbers excluding charters. (We’ll see why in a second.) Here they are:

2013TEAminusCharter2014TEAminusCharter

On first glance, you’d be right to say, “Wait a minute, that reduction doesn’t look like 5 percent.” Because the reduction in number of schools (excluding charters) that were marked as IR was a smaller percentage than that. The reduction of 670 to 651 schools marked IR is 2.84 percent, not 5 percent.

Ah, but wait! He said that if you take out DISD, the rest of the state’s reduction is 5 percent. Okay, that means if you take out DISD’s 34 schools rated IR in 2013, you get 636. If you take out DISD’s 43 schools rated IR in 2014, you get 608. So now, a reduction from 636 to 608 is 5 percent. Got it?

Except that’s not right, either. That’s a reduction of 4.4 percent. Which doesn’t even round to 5 percent, even under 1970s math.

So, he’s just doing a little fudging, right? First, I’d argue he’s fudging by choosing the metric he chose. For one thing, he’s ignoring that the number of schools rated DROPPED by 17 in 2014, which means the percentage decrease is even lower than stated above, albeit very slightly. But ignore that. To me, the larger problem is that either you must take the entire sample of public schools — and charters are public schools, often with poverty levels MUCH closer to DISD’s than those of much smaller school districts — or you choose to compare districts with like size and poverty (apples to apples). Problem is, neither of those work as well as the (slightly fudged) numbers for Bill.

Let’s look at the metric that most of the statewide media use for comparison: all public schools in Texas. The IR results for 2013 and 2014 are pulled out below:

2013TEAplusCharter2014TEAplusCharter

You’ll see that now we go from 768 schools marked IR in 2013 to 750 in 2014. Again, the total number of schools rated changes, but again it wouldn’t change these numbers much if you accounted for that, so let’s use Bill’s method.

Overall, the decrease in number of schools rated IR in 2014 from 2013 is much lower than 5 percent: it’s 2.35 percent. But let’s use Bill’s method of taking out the DISD schools from our equation. We subtract 34 Dallas IR schools in 2013 from 768, giving us 734. We subtract 43 DISD IR schools from 750 in 2014, and we get 707. The decrease from 734 to 707 is 3.78 percent: lower than the 5 percent Bill states, so much lower no amount of rounding could get us above 4 percent.

Why not compare apples to apples: Dallas to Houston? Because even though Houston had a severe decrease this year, Dallas did MUCH better than Houston last year using his total schools metric (34 to 58) and better than them in terms of percentage of schools. And no way is he going to give Miles credit for good 2013 data — it doesn’t fit his twisted narrative of unending, complete decline in every quarter since Miles came aboard.

This is the EPITOME of cherry-picking. I can do the same thing to defend the district in any number of ways. For example: I could say that there are more college-ready kids in DISD now than before Miles arrived. And that is without question true. But does it ignore context? Oh my stars, does it ever. It ignores that two years ago, two out of three kids in the district took the ACT or SAT (each of which can provide a measure of college readiness), and last year more than three out of four took it. It ignores that there is a huge national debate about what is college readiness. It ignores that there are really three different levels, between DISD and the state, of college readiness. It ignores that we keep moving the target. It ignores that college graduation rates are often stronger than ACT/SAT scores predict, because of grit. It ignores that magnet students make up a disproportionate number of our college-ready kids — which I could easily argue DOES matter or DOES NOT matter; flip a coin. It ignores context, which is the whole reason we’re here. It also ignores, as Bill does, that this is the continuation of a problem already identified and being addressed, one that has been persistent in Dallas ISD since at least 1998.

This isn’t new. Schutze pointed out months ago that Bill likes to give dire warnings about the back-room evildoing going on in DISD, then moves on to another pet issue when those warnings prove incorrect. But I’ll give you a more recent, equally disappointing example.

A few weeks ago, before the last board briefing, Bill sent out an email to the media and trustees that said the following about expected teacher vacancies at the start of the school year (a recurring problem):

“Item 5 C on today’s briefing agenda is potentially the most critical item, but the report attached online to the agenda did not give any idea of the vacancies that may be anticipated. Hopefully there are none.”

Before the meeting, according to Bill, the number of teacher vacancies was potentially the most crucial item, on a day LOADED with important information (student outcome reports, overall district evaluation, etc.). Then the teacher vacancy report came, and it was outstanding news, showing fewer vacancies than at any time in recent memory. (The week before school started, the vacancy count stood at 25 and was headed toward zero by the end of week one.)

Was Bill’s response to say, “Huzzah! That’s great for kids!”? Of course not. That would give Miles (or his team members; same thing) credit, which he cannot bring himself to do. So on the Support Our Public Schools Facebook page, where the news story trumpeting this fact was placed, Bill commented, “Being able to hire record numbers of teachers faster is a strange thing to have to brag about.”

Oh. So now it’s NOT important. Okay, then.

Forgive me, maybe it’s not fair to cherry-pick one comment. But that’s the rules of the game, apparently.

• I forgot.

Newsletter

Keep me up to date on the latest happenings and all that D Magazine has to offer.

Comments