Comments (0)

By Kyle, September 4, 2011 9:25 am

Bruce Springsteen – Devil’s Arcade

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Where’s the Devil?

According to Dante (in this amazing translation of Inferno I’m currently reading), he’s in the Ninth Circle. Yet, according to Doug McRae, he’s in the details.

Doug McRae shows just how confusing an misleading data can be sometimes. In his article, “Expansion of test for special ed students distorts latest API results”, he argues that the most recent API scores in California’s schools are partially due to the number of special ed students taking a modified version of the test. This article is definitely worth the read, and implicitly states what many LAUSD schools have found to be explicit: that schools will vastly increase the percentage of students taking this modified assessment in order to boost API.

While state assessment policymakers may have had the best in mind when creating these modified tests (i.e. doing what’s best for kids), the fact that this nuance has been completely glossed over in the media means that schools can be incentivized to use new tests to promote testing demographic changes as actual instructional improvements. Again, this illustrates the problem of using absolute data points to make policy arguments. As a remedy, I would again point to the spirit of using a “value-added” approach to measuring school gains, which would more accurately measure the impact that schools have on whichever students they serve.

Comments (0)

By Kyle, September 3, 2011 11:27 am

Mayer Hawthorne – Mr. Blue Sky

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

• My Letter to Monica Garcia

Nothing irks me quite like a person who rants about his dislike for something, but fails to suggest a viable replacement for it. I would be amiss then, in wrapping up this series, to finish ranting about the terribly misleading phrase “high performing schools” without likewise, suggesting a replacement phrase that better captures what it is we’re trying to measure with the term “performance”.

And so (you heard it here first), I would like to see the education community replace the word “performance” with the word “impact” (as specifically measured by a value-added model).

So rather than talking about “high performing schools” which results in name-calling, misapplied data, and juking the stats to make performance stand out, we should talk about “high impact schools”.

What would the effects of a vocabulary shift be?

1. It reaffirms the moral obligation we have to education all children in the nation. By using the phrase “high impact school”, we are holding schools accountable to do exactly what it is we’re in the business of doing, educating children, specifically whatever children walk through our doors. Different schools serve different communities, each have different strengths, challenges, and opportunities. Whether schools are educating high-income children in the suburbs or low-income children in the city, our mission is nonetheless the same: transform students’ lives. If we can do this, we’ll have an impact.
2. Schools will be able to focus resources on transforming educational outcomes for their students based on their students’ needs. Instead of responding to perverse incentives to artificially boost test scores, schools will actually be freed to focus on improving instruction.
3. Charter schools and district schools may actually be able to collaborate and learn from each other. Perhaps this is still a pipe dream, but part of the reason that charters and district schools in LA seem to hate each other (or, at least, latch onto whatever data conveniently boosts their cause), is because of this unfortunate rhetoric of performance. By moving to the term “high impact schools”, we might perhaps be able to have a civil discussion. Charges of “teaching different student populations” will be moot because the idea of “impact” is measurable not only in absolute terms, but respective to whichever group of students one happens to be educating.

Do I see this vocabulary shift happening? Unfortunately, not anytime soon. The media seems to still be focused on “performance” although the way that this district is going, we’re on a collision course between some of our initiatives and this idea of value-added.

However, I’m hoping that with a little encouragement, some LAUSD Board Members might start to act on this.

• My Letter to Monica Garcia

Comments (0)

By Kyle, August 27, 2011 11:46 am

Les Miserables – Master of the House

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

To summarize so far, I’ve argued that the phrase “high performing school” is both misunderstood, and even when agreed upon, often refers to a metric that doesn’t actually measure school performance.

One possible attitude at this point would be to say, “Well, so what. After all, isn’t it just semantics that we’re arguing here?” Today, I’ll make the case that misunderstanding this phrase not only is not pure semantics, but actually can incentivize actions that harm students.

By talking about and measuring “performance” in this current method (i.e. subtracting the percent of proficient students from year to year) in this manner, all schools need to do to “show” they are high-performing is ensure that next year’s class does better than this year’s class. While one would certainly hope that this would be achieved by improvement of the instructional program, the reality is that there are several other (and easier, thus providing a dangerous incentive) ways to do this. Specifically, schools can change the population of next year’s class. Schools might attract higher achieving students by adding a magnet component, or a school for advanced studies (which are likely to attract higher achieving students). Schools might also change their population by “counseling out” students that are not achieving. If this occurs, all of a sudden a cosmetic shift is mistakenly viewed as a performance shift.

Simply put, in the context of education, improvement and performance needs to be about better instruction, not better recruiting. And until we begin to get at what we really mean when we way “performance” (rather than merely skirt around the issue), schools will be incentivized to find the much easier way to (intentionally or unintentionally) “massage the data”, rather that working on effectively improving instruction, which is what every child in Los Angeles truly deserves.

Next post, I’ll wrap up with what I believe could be an honest replacement term for the phrase “high performing” that might just actually get at what we mean, and what we hope for.

Comments (1)

By Kyle, August 24, 2011 6:31 pm

Alex Russo writes a very interesting article here about how one of the charter operators is responding to the L.A. Times article I recently posted about.

If I was a CMO, I’d be upset too, but the problem is not that media studies like these upset education organizations, but rather that most generally vocalize opposition when the media portrays them in a negative light, regardless of the validity of the data. LAUSD should be writing emails complaining of this sort of measurement of growth as well.

For my “edu-wonky”  friends out there, I’d very much recommend subscribing to Russo’s blog.

Comments (1)

By Kyle, August 21, 2011 9:21 am

Fleetwood Mac – Second Hand News

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

As this blog series continues, I’ve come to realize two things:

#1. There’s no chance of this series fitting into the 4 posts I had originally intended.

#2. It is fully appropriate to interrupt the conversation whenever “breaking news” occurs.

Today is such a day for an interruption.

Last week, the L.A. Times published a story with the headline “L.A. Unified bests reform groups in most cases, data show”. A few interesting things happened immediately following this publication. First, union activists and other similarly minded folks jumped on this opportunity as evidence supporting the contention that current reform efforts (charters, pilots, etc…) are misguided. Second, one of our local radio stations, KPCC, republished the story, subtly (and perhaps unintentionally) altering the headline to read “Tests show LAUSD outperforms charters” (emphasis mine).

Both of these responses have only served to frustrate me. Let’s tackle them in reverse order.

While KPCC’s re-headlining of the story certainly makes the contents potentially more compelling, the sad fact is that the data has very little to do with performance. Rather, the data that is cited simply shows that some of this year’s students mastered more content than some of last year’s students, and that that difference is higher for some district schools than for some Partnership schools and some charter schools. This is something I have hopefully articulated well here, here, and here. Unfortunately, when news outlets misinterpret data (and then meta-misinterpret the stories themselves), they create more massive misunderstanding of what is good data, what schools are doing (and doing well), and thus are more likely to jump of quick fixes and fads. If we’re going to get the story right, the mainstream media has to dedicate itself to getting that story (and the data) right.

Second, there is often a great outcry among many public school teachers when the media portrays charter schools in a positive light, particularly in the complaint that the media source is failing to use data correctly. For example, take the January 2010 L.A. Times story titled “Charters generally perform better than traditional schools, not as well as magnets.” Immediately following publication, many district school advocates quickly let the L.A. Times know that they needed to be far more critical of the data they were citing to arrive at that headline. This critique was thoughtful, and necessary. Yet, much to my frustration, the same group of folks will generally blindly accept a headline that now, just perhaps, favors their cause. What’s frustrating is not who’s right and who’s wrong, but our tendency as advocates for our own particular cause to simply latch on, perhaps uncritically, to any “data” that supports our cause.

Ultimately, if we do have strong beliefs (and I certainly do), that is all the more cause for us to critically examine data before we taut it as evidence that our way is the right way. For me, I am committed to public schools, I am committed to district schools, and I do have suspicions about how district schools and charter schools actually educate the children of our LA. And it is because of that commitment that I am unwilling to latch on to these most recent L.A. Times / KPCC media stories covering the “performance” of different models of schools – first, because the term “perform” is misleading, and second, because the data cited (literally subtracting % proficient/advanced from year to year) is not the type of data that could be used to actually support this claim.

Give me better data (like a truer measure of growth) and then I’ll get excited about your newspaper article.

Comments (0)

By Kyle, August 16, 2011 3:18 pm

Tal Bachman – She’s So High

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Thus far, I’ve contended that the phrase "high performing school" must be extricated from the current educational debate because there is inconsistency regarding whether it refers to student achievement or student growth.

Continuing this series, today, I’ll argue that our traditional metric of student growth is still a wildly inaccurate way of measuring performance.

Unfortunately, the way that most pundits who talk about student growth as the measure of "high performance" usually winds up something like this:

In 2009, 60% of Central High School’s (CHS) 9th grade Algebra students passed the state test. In 2010, 75% of the CHS 9th grade Algebra students passed the state test. Therefore, CHS improved their student growth significantly, and thus, is a "high performing school".

The issue lies not in the fact that their year-to-year outcome data is better (it is). The issue lies in the fact that this means of measuring student growth does not account for the reality that the 2010 9th grade students are a completely different group than the 2009 9th grade students. Simply put, a metric like this one may be measuring a school’s growth in performance, but it may also be measuring a school’s changing population.

It’s a move in the right direction to talk about measuring student growth when evaluating schools, but unfortunately, the methodology mentioned above may not actually do that. A more nuanced metric is needed.

That being said, what is needed is a metric that actually measures a school’s ability to do what it sets out to do, namely to “educate youth”. Fortunately, we are actually beginning to have a conversation about a metric that may actually capture a school’s ability to do just that, “educate youth”. And for we who, as public educators, are committed to educating any and all youth that walk through our doors.

As some have no doubt, guessed already, the metric I am alluding to is “value-added” modeling (VAM). Unfortunately, the concept of VAM had only been talked about with respect to individual teachers.

Now let’s be clear. I, like many in education circles, have some issues and hesitancies with the VAM methodology, its application to teacher evaluation, and the side-effects it may cause in the now super-high-stakes testing world. I definitely think more thought, conversation, and study is needed here as we move forward. Simply put, the method is far from perfected at this point.

Yet, one piece of this VAM debate that goes un-talked about is how it could actually serve as one means to talk about what is honestly going on in schools. By framing the conversation with an attempt to gauge the “value a school adds” (which is what we’re in the business of doing), we can actually start to measure the very thing we’ve been talking about for the last decade – “school performance”.

Unfortunately, the notion of a “high performing school” being based on a school’s actual impact on student learning is far from being the standard case. The sad fact is that we have a metric (albeit imperfect), but we don’t actually mean it when we refer to “high performing schools”.

Hopefully now, it’s clear why the term “high ‘performing school” is detrimental to the conversation. Simply put,  the vast majority of the time you hear someone talking about “high performing schools”, there is no sound thought about either what they’re measuring, or how to measure it. Until this is remedied, the phrase must simply be banned.

In my next post, I’ll explore exactly what the consequences of this mis-terminology are.

Comments (2)

By Kyle, August 9, 2011 3:42 pm

The Standells – Dirty Water

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

In my last post, I wrote that the phrase “high performing school” actually muddies the education policy conversation that is happening across the country. It’s not helpful for districts, it’s not helpful for schools, it’s not helpful for teachers, and most importantly, it’s not helpful for kids.

Today, I’ll explain why.

Simply put, the reason that the phrase “high performing school” should be banned from conversation (at least at this stage) is this: No one agrees or even understands what the word “performing” exactly entails.  Putting it more directly, the word “performing” alludes to some form of measurement, yet there is no consensus on what it is we’re actually measuring, especially given our various philosophies of what we expect the outcomes to be. Very simply, the term “high performing” can allude to two very distinct measurements of performance.

1. The first basic interpretation of “high performance” is simply to look at student outcomes, and that’s all. If, or so the logic goes, students are performing highly at the end of a grade level, then the school must be a “high performing school”. Luckily, many have seen the flaw in this measurement methodology, as it fails to take into account the concept of student growth. After all, schools are not just about promoting student achievement, but promoting student growth as well.

2. The second interpretation is to attempt to look at student growth from one year to the next in a school. Now since most schools manage a fixed number of grade levels, the logic here goes something like this: If 60% of a school’s seniors master the content one year, while the following year, 70% of the seniors master the content, the school must have grown, and might even be “high performing”.

Because there are two varying notions of what “high performing” actually alludes to, the conversational waters are muddied. This difference is almost never stated in conversation and that a speaker and hearer mean the same thing by “high performing” is almost always assumed. The reality is that these differing perspectives are, in fact, different. Furthermore, they can be especially different depending on the role of the person referring to these “high performing schools”. Ask an inner-city teacher and a parent of a small child about to enroll in school, and you’re bound to get two very different perspectives about what the definition of “high performing” is.

And what’s worse (if it could get worse), is that I believe (and so do a growing number of others) that neither metric actually measures what it is that a school is supposed to do, and thus, both fail to accurately classify schools as “high performing”. Stay tuned for more in my next post…

Comments (0)

By Kyle, August 1, 2011 6:45 pm

AC/DC – It’s a Long Way to the Top

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

The phrase “high performing school” is now en vogue. Frequently thrown around by district leaders, charter organizations, parent advocacy groups, politicians, and even newspapers, the phrase (often equivalently stated “high achieving school”) has become a battle cry for those who do not merely seek out education reform, but often a certain type of education reform. (For my grammarian readership, the hyphen seems to be optional).

And while the ideas and ideals of those who tout the merits, vision, and governance models of “high performing schools” are certainly worth considering, it’s essential not to overlook the power of language, and specifically, the way this single phrase – “high performing school” – actually muddies the water of collaborative educational progress.

And so I begin a series – a series on these three words, “high performing school” (or “high achieving school” if you like). I will explore why it’s so confusing (there are 2 reasons), what the effects of this confusion are, and what phrase we should replace it with.

Stay tuned for more…