Jim Schutze has a good post on Unfair Park about how everything is Mike Miles’ fault, according to his critics. Even things like firing investigators who don’t work for him. But I want to highlight the first comment on that post from “MindingtheStore,” because it gives us a good place to start discussing how parents, teachers, some trustees, and other interested parties take data and frame it to suit their desired narrative.
MTS says several interesting things that are pretty easily refutable, but let’s just concentrate on the first graph (I fixed a few typos):
Jim, the one thing you cannot obfuscate in your “pro Miles” musings is that scores on STAAR have gone DOWN under Miles. IR schools have also increased greatly under “your leader.” His programs have not worked regardless of how many “recycled” ideas (personalized learning, choice schools, ACE schools, Imagine 2020) he comes up with. One reason they won’t work is because the teachers have NO faith in him. Another is that he has few veteran teachers left (many of the best ones have left for other districts where they will be treated like human beings) to mentor all the new folks.
Now, there are things here that are wrong. For example, STAAR scores have gone up in some areas (math, science, social studies). Another example: the district has never had anything more than a couple open-enrollment schools of choice, so the assertion that this is a recycled idea is also wrong. There are also a few things that are wrongish. The district used to offer incentive payments to individual teachers, evaluated on a system that didn’t differentiate quality, if those teachers moved to low-performing schools. The ACE plan, by contract, requires higher-performing teachers per an evaluation that does differentiate on quality and requires at least 30 of them to move to one campus along with a new leadership team. Not really the same thing. The district’s old learning center concepts seems similar to Imagine 2020, except Imagine 2020 is an entire feeder pattern approach intended to also harness wraparound resources in the community, whereas the learning centers stood on their own. Again, not really the same thing.
But this anonymous commenter got a few things right. STAAR scores are down in one area (reading). These results mirror what happened statewide.
And the number of IR schools did indeed increase. This is something that Miles critics point to weekly, and they are right that this is a concern. But when critics bring this up, they fail to mention that the definition of what it means to be “Improvement Required” has changed, specifically making it more likely that districts (like Dallas) with large numbers of kids who don’t speak English will have IR campuses. That’s a bit more complex as a comment, so I can see why they don’t keep mentioning that point. People always prefer simple over complex. And even if their points are wrong, they prefer points that support their worldview rather than oppose it.
People talk about the increase in turnover all the time, too, saying that veterans are leaving DISD in droves. They fail to note that the trend in turnover in Dallas ISD is consistent with increases in turnover at virtually every district in the state. The economy is getting better, and lots of teachers are taking other jobs. But there is still some truth in this argument: The turnover rate in Dallas has risen slightly faster than the state’s average. But it’s a lot sexier to say: ALL THE VETERANS ARE LEAVING rather than saying: TURNOVER IS SLIGHTLY ABOVE AVERAGE.
Beyond this comment, Miles opponents like to highlight the individual schools that are doing poorly. Some schools have definitely gotten worse. This is a very real problem. But the examples are rarely countered with success stories. For example, several Imagine 2020 schools are getting better – poor kids at troubled schools are doing better with extra teachers and tutors that Miles added. And let’s not forget about the logic of this line of reasoning. Do we think that no individual schools ever got worse in the history of the district? Do we think a superintendent should be fired if, say, 10 individual schools out of the 224 in DISD got worse, regardless what is happening in the rest of those 224 campuses?
Most important, there are things we don’t yet know, like pretty much everything else the anonymous-commenter-with-fact-problems mentions. Early results from schools like Mata suggest school choice is a welcome tool in the reform toolkit — it certainly seems to be reducing racial segregation — but it’s too early to judge the whole program. Personalized learning hasn’t even taken shape yet; nor has ACE.
Look, I talk to teachers all the time who hate Miles. I also talk to teachers who love their job and laugh at his critics. They are all great teachers because I assume every teacher I meet is great because try to find me a teacher who says he or she is less-than-great.
The problem with much of the analysis from Miles critics is it can barely even be called analysis. If we’re honest with ourselves, it’s really just too soon to tell the effectiveness of any of these reform efforts. Education is hard, and it takes a while for trends to show up. There are a few indicators we can look at, but the most effective, system-wide indicator that we have for Dallas ISD is actually quite positive.
I have yet to see a better comprehensive short-term success metric than the one(s) provided by Education Resource Group (ERG), the education analysts out of Houston. Because it gives you a snapshot every year of how the district is doing as a whole, relative to like districts, while factoring in poverty and its effect on outcomes. Oh, and it does so by looking at seven major factors, everything from STAAR scores to college readiness to college completion to graduation rates. It’s not perfect, but as a bottom line indicator of the system as a whole, it’s the best we’ve got.
I told you about these guys last year and how Dallas showed an incredible jump under Miles when looking at these metrics. ERG presented its most-recent numbers at the last trustee board briefing. And, once again, Dallas ISD showed to be maintaining its dramatic systemwide success when you factor in poverty.
Some quick background: What makes ERG’s data better than TEA’s grades or simple state test scores? Because the state is not very sophisticated in the way it reports or interprets education data. It focuses on raw test scores with no context. That’s fine if you’re Mom and Dad. It’s not very helpful, though, if you’re trying to evaluate the effectiveness of large urban school district, because demographics drive performance. So raw scores don’t indicate how good a district is doing at educating its population. Which is why we can look at, say, Richardson and determine that, when you consider the poverty of its student population, that district is doing a slightly better job than Highland Park at educating its kids – at making a difference. Only once you normalize for poverty can you compare poor school districts and rich school districts.
Once you understand this, you can see how Dallas, once a bottom-third performer in ERG’s “Performance Index,” has been climbing up to near the top quartile in the state – nearly on par with Houston, which has for years been considered far better than DISD at teaching kids. ERG showed how DISD has been steadily improving since 2007. But, in the past two years, DISD’s performance – given its 90 percent free-and-reduced lunch population – has been pretty spectacular, according to ERG. Here is the ERG Presentation to the trustees.]
ERG highlighted four districts in its presentation: Dallas, Houston, Irving, and Fort Worth. Dallas and Houston easily outperformed the other two districts the past few years. (Note to Irving ISD: You’ve got some work to do, friends.) When you add in financial performance to the mix (in other words, showing how effective a district is by combining its student outcomes with its balance sheet), DISD is right there with Houston as one of the best big districts in the state.
“Given the challenges that your students represent,” ERG’s Tim Tauer told trustee Eric Cowan, “you’re doing an incredibly good job.”
Which led Mike Miles to ask this loaded question of ERG: “Is it fair to say that our performance in the last couple of years has been significantly better than the years before that?”
Tauer’s response: “Clearly.”
Now, you didn’t read about this in the Dallas Morning News last year or this year, so it’s understandable that you wouldn’t know this information. (If ERG had bad news to report to DISD, you can bet it would have been on the front page.) And, no, this is not to say that DISD is wonderful. There are important questions to ask once you dive into the campus-level data (are a handful of schools driving the success and covering for other poor performers?). But if you’re going to try to evaluate how the district’s reform efforts are affecting student performance, cherry-picked quotes from Miles foes about programs just being implemented is a dumb way to do it. Start with the ERG data, and then ask questions from there. Unless you have an anti-Miles narrative that runs counter to the data. Then you can just pretend it doesn’t exist.
