Van Halen – Right Now

Now we’ve reached the third post in the “Observer Effect” series, having learned both what the Observer Effect is¹, and the nitty-gritty of why and how it can powerfully co-opt even systems designed with the best of intentions². So before I get around to actually making recommendations about how school systems need to account for the reality of the Observer Effect, it’s worthwhile taking a look at how the Observer Effect interacts with our current system of teacher evaluations and observations.

Recent white papers³, op-eds⁴, and lawsuits⁵ have all shown that, even among people who disagree incredibly, there is a consensus that the current teacher evaluation system is broken. The reality that almost everybody receives a satisfactory evaluation is just the tip of the iceberg (under the surface we find that because of this, no one is helping us get any better).

In conversations will a multitude of colleagues, I’ve found that my bi-yearly evaluation is fairly typical. Namely, I submit a form addressing the ways I will seek to improve my teaching this year, and then, on a predetermined date, an administrator will show up in my classroom with a checklist, and proceed to “observe” my instruction. Finally, a few weeks later, I will take a look at their observation notes, all of which rate me as satisfactory or higher, sign the thing, and then I’m done.

That’s literally it.

Now many have pointed out the obvious flaws in this system, so I won’t repeat them, but what I’d like to do is take a quick look at how the Observer Effect (i.e. how the process of being observed actually impacts the results of my evaluation) interacts within this current system.

Using the framework from the last post, we notice the following process of the Observer Effect:

1. Principal wants her teachers to effectively teach students.
2. Principal announces observation procedure (submitting a form called a “planning sheet”, a 30 minute observation, discussing and signing the form).
3. Me, hearing about this type of observation, considers my options
4. Options are some combination of (A): Thoroughly prepare a planning sheet, in which I’ve thoughtfully considered how I want to improve this year. Ask my principal to observe my most challenging class. Ask for specific ways of improvement in the post-conference. Or (B) Fill out a “jargony” planning sheet, with words like “scaffolding”, “multiple-modalities”, and “student outcomes”. Have my principal come during my easiest class. Simply nod and smile at the post-conference, and sign my name, breathing a sigh of relief knowing that I won’t have to repeat the process for another two year.
5. Observation/Evaluation occurs.
6. I as a teacher, demonstrate a certain level of mastery of the teaching profession.

A couple of notes of the reality of the Observer Effect.

First, my performance evaluation (the outcome) is largely a function of which of the options in Step 4 I take. The fact that I know that I’m being observed, and I know how I’m being observed actually affects the results of my observation.

Second, and more importantly, the specific option I choose is largely a function of the observation content (“How rigorous will my observing principal be?”), its form (“Will this evaluation actually help me get better?”), and its result (“How much weight will this observation play in my future career?”).

The Observer Effect guiding my decisions leads me to pick Option B (question 4 above), simply because my principals are typically too overburdened with running the school to be extremely rigorous, my evaluation won’t help me get any better as a teacher, and has virtually no impact on my future career. Time after time for me, it’s Option B.

So in summary, the point of this post of connecting the Observer Effect to the current evaluation system shows that not only is the current system flawed, but that the Observer Effect (i.e. teachers knowing how the system is flawed) actually exacerbates the situation.

If we’re going to improve evaluation (which everyone and their mother says we should be doing), we’ve got to design a system which does not ignore the reality of the Observer Effect in observation and evaluation.

More on exactly how to do this in the upcoming posts!

Stay tuned.

1. http://blogs.hunsbergermath.com/1141/2012/02/03/the-observer-effect-a-prologue-part-1-of-6/
2. http://blogs.hunsbergermath.com/1141/2012/02/06/the-observer-effect-an-abstraction-part-2-of-6/
3. http://widgeteffect.org/
4. http://www.huffingtonpost.com/teach-plus/the-war-over-teacher-eval_b_1162891.html
5. http://latimesblogs.latimes.com/lanow/2011/11/suit-filed-to-make-la-teacher-evaluations-include-student-data.html

Bach – Ricercar A 6

On Friday, I began this series¹ by introducing a phenomenon known as the Observer Effect.

To recap, the Observer Effect describes the fact that the mere act of observing or measuring something actually affects what we are observing. We also saw that the Observer Effect is not inherently a bad thing, especially when we desire a certain outcome to be measured, and the act of measuring it actually helps achieve the measurement we want. The classic example of this in the classroom setting was announcing an upcoming test, in the hopes that the test will not only measure student achievement, but that by merely announcing it, student achievement will be improved because many students will then study.

Today, we’ll take a step back and look at the general form of the Observer Effect, including the mechanisms by which it occurs. [Disclaimer: This post, as the title suggests, is fairly abstract, though not at all difficult to understand (i.e. it may satisfy your inner geek). If you are denying your don’t have an “inner geek”, then I strongly suggest skipping this post, and re-joining in with Part 3.]

To restate the Observer Effect in a different way, it makes sense to begin with the classic question “If a tree falls in the forest, and there is no one there to hear it, does it make a sound?” The answer is, of course, “yes”, however, the sound that an observer hears it make is different than the sound it makes when no observer is present².

So our question for today is, what is the general form of the Observer Effect and how does it change that which is observed, whether physical changes (like the tree in the forest), or behavioral changes (in a classroom setting)?

To ease this meta-view, lets first look at the specific, then generalize. The following chart explains (using the “giving a test” example) how the observer effect actually works.

• If we care about the way that Johnnie considers his options upon knowing that he will be observed, we MUST, we MUST, we MUST, take great care in designing HOW observation occurs, as its effects trickle down.

2. Sound is literally air molecules vibrating in patterns that strike our ear nerves and transmit sounds to our brain. Physically standing next to the falling tree changes the path that these air molecules vibrate in towards the ear, thus changing the sound, even if unnoticeably slightly. Being in a forest to hear a tree fall is similar to seeing how waves in a lake hit the short differently depending on whether or not they are traveling around a moored boat.

Willie Nelson – The Scientist

Yogi Berra¹ once said “You can observe a lot by watching”. And aptly so, as one of the main things that an education teaches us² is not only the power of careful observation, but the methods by which we learn to be careful observers.

As we are told, the better observing and measuring we can do, the better results we’re likely to achieve.

Or so the story goes.

What I will be exploring in this blog series, beginning today, and continuing over the next six posts is a concept called The Observer Effect: specifically, what it is, how it currently applies to classrooms and education policy, as well as how it might affect proposed changes to education policy. Be sure to subscribe to the blog to follow along. So without further ado, here we go.

So what on earth is “the observer effect”? Well, as best as I can explain it, while the complex definition of “the observer effect” involves a combination of particle physics, quantum mechanics, and general technical gobbledygook, the simple definition is this: The very act of observing something affects that which is being observed. Or put even more simply, you can’t observe something without affecting it in some way³.

Because my plan for this series is to mainly hit the theme of how the Observer Effect fits within the context of teacher evaluation, it’s almost natural to want to start off with the question of “How do we minimize the Observer Effect?”

In reality, however, this is the wrong question to be beginning with. After all, if you’re a teacher like I am, you depend on the Observer Effect as part of your classroom instruction. For example, let’s say I am giving a test next Friday, the purpose of which is to evaluate and observe the levels of learning that my students have achieved. As a teacher, I will, along with most of my colleagues, announce the upcoming test in advance to my students. Now here’s the kicker: The whole reason I announce the test in advance is because I not only believe in the reality of the Observer Effect, but I hope in it’s reality to do good.

Now any teacher worth his or her salt, will not only announce an upcoming test, but will no doubt, also encourage their students to study for it. A student “studying” for an upcoming test is exactly the result of the Observer Effect. The student’s behavior (i.e. “studying”) has, in fact, been affected by a mere observation (the test). Not only to teachers know that students’ behavior will (or, at least, should) be affected by the observation, I’m guessing most teachers, including myself, actually hope for this. Yes, of course I want to have my students study for an upcoming test. What’s even more fascinating (although not necessarily a great teaching practice) is when teachers observe (i.e. assign upcoming assessments) in order to change student behavior more than assess current levels of progress. So before we go decrying the Observer Effect, it’s worth noting that it’s not necessarily something to be completely minimized.

A better question to start with than “how do we minimize the effect of observing” would be “to what extent is the Observer Effect likely to result in changed behaviors that we actually want (or, at least, tolerate)?” or “how can or should observations be structured so that the naturally occurring change in behavior of the observed is something desirable?”

Simply put, as a teacher, I’m going to keep announcing my tests in advance to my students because I believe in the value of studying for an upcoming test. When the changed behavior is a desired one, the observation (i.e. the test) is, in fact, aligned to what I am willing to tolerate (or in this case, even want).

In conclusion, as we move forward in this series, it’s important to think not of whether the Observer Effect is “good” or “bad”, but acknowledging that the Observer Effect exists, and knowing how to use it to the advantage of kids.

Stay tuned for more on this intriguing topic…or at least for songs as cool as the aforementioned Willie Nelson’s cover of Coldplay.

1. http://www.yogiberra.com/yogi-isms.html
2. http://teachers.greenville.k12.sc.us/sites/cthornto/Shared%20Documents/observandinfer.pdf
3. http://www.aqr.org.uk/glossary/index.shtml?observereffect

Holst – Jupiter (The Planets)

It’s been two weeks since I last posted on my blog, and in my absence here, I’ve been pretty active on Twitter.

Unfortunately, 140 characters sometimes isn’t enough to explore a thought very thoroughly. In that vein, I am offering a preview of a new blog series I’ll be beginning later this week.

So what’s the series? This series will be called “The Observer Effect”, and while I don’t want to give too much of it away, a simple search on Wikipedia will lead to the revelation that The Observer Effect refers to a principal in quantum physics. I’ll then proceed to connect it with, yep, you guessed it, education policy.

What?! “Quantum physics”?! You’re kind of out of your league here. Yes, I know, but I love seeing connections between education policy and seemingly disparate worlds. (Plus, it’s not necessary to really understand the physics at all. I certainly don’t).

So why write a series? Why not just a post? Two reasons. First, if it were a single post, it would be FAR too long. The best arguments, I’ve found, are the ones that are developed over pages, not over paragraphs. Second, and probably more profoundly, writing actually helps me think, and figure out what I believe about certain issues. So while I have a general in-place framework for what this series will consist of, I’m still teasing out some of the details.

Fantastic. I look forward to reading your new series. And, by the way, isn’t it kind of weird talking to yourself like this? It certainly is. Now go away.

Be sure to subscribe to keep abreast of all the latest happenings in this blog. (If you’re not into education policy, I apologize in advance….sorry, Jenna). In the mean-time, feel free to check out my other Blog Series’ from days of yore.

• All I Want for Christmas: My 2011 Education Wishlist
• A New Vocabulary: Why the Phrase “High Performing School” Must Simply Go
• Charter Schools vs. District Schools
• A Response to the LA Times “Grading the Teachers”

The Entrance Band – M.L.K.

A shout out to my friend and colleague, Tyler, whose alma mater, Chapman University, hosted Rev. Martin Luther King in 1961. While the text of Dr. King’s address, found here, speaks specifically to the progress of race relations, both Tyler and I couldn’t help but notice that his characterization of the current status of race relations runs parallel to the current philosophical bifurcation that is plaguing the education reform debates today (particularly as it relates to the hot-button issues of evaluation, and “value-added”).

In short, Dr. King’s call for a realist, progressive, and (most importantly) actionable attitude and commitment speaks volumes. In this analogy, it seems Dr. King can also teach us something about the dangers of extremism and polarization. Simply put, for Dr. King, the biggest problem of opposite and extreme viewpoints is not that they are wrong, but that they lead toward inaction.

Below, I’ll quote a bit of a lengthy portion from the address, but it is well worth it. Dr. King’s words of wisdom continue to speak not only across generations, but in all walks of life.

There are three basic attitudes that can be taken toward the whole question in the area of race relations. The first attitude that can be taken is that of extreme optimism. The extreme optimist in the area of race relations could contend that we have made tremendous strides in the struggle for racial justice. He would point proudly to the progress that has been made in the area of civil rights over the last few decades. From this he would conclude that the problem is just about solved now and that we can sit down comfortably by the wayside and wait on the coming of the inevitable.

The second position that can be taken is that of extreme pessimism. The extreme pessimist in the area of race relations would contend that we have made only minor strides. He would argue that the deep rumblings of discontent from the South, the presence of federal troops in Little Rock, Arkansas, the resurgence of the Ku Klux Klan, and the birth of White Citizens Councils are all indicative of the fact that we are going backwards instead of forwards and that we are creating many more problems than we are solving. And from this the extreme pessimist would conclude that there could be no real progress in the area of race relations.

Now it is interesting to notice that the extreme optimist and the extreme pessimist agree on at least one point. They both feel that we must sit down and do nothing in the area of race relations. The extreme optimist says do nothing because integration is inevitable. The extreme pessimist says do nothing because integration is impossible.

But there is a third position that can be taken, namely the realistic position. The realist in the area of race relations seeks to combine the truths of two opposites, while avoiding the extremes of both. So he would agree with the optimist that we have come a long, long way. But he would seek to balance this by agreeing with the pessimist that we have a long, long way to go.

John & Yoko – Happy X-mas (War is Over)

Dear Santa,

Tomorrow is Christmas! And with it, I’m wrapping up with this last item on my 2011 Educational Wishlist. If you can’t recall (or have misplaced) all 10 so far, be sure to check the list below.

So Santa, what’d I’d really, really, really like this Christmas is a paradigm shift. Unfortunately, there do seem to be two “sides” in this debate, without much room for a more thoughtful and nuanced middle ground.

So Santa, my last item on my Christmas list is for me. Santa, give me the gift of patience. Give me the gift of not assuming malicious intentions of any education writers, bloggers, policymakers. Give me the gift of being able to challenge ideas that are wrong without resorting to vilification.

So Santa, just to recap, here’s my list one more time. Everything I want in 2011!

1. Rhetorical Nuance
2. Local Examples of Schools that Successfully Engage Families
3. District/Charter Collaboration
4. Real Coaching
5. More Classroom Visitations
6. A Revised Public School Choice
7. Better Data
8. Wrap-around Family Services
9. Increased Voter Turnout
10. End Social Promotion

The Who – Christmas

We’re almost there, Santa!

The recent personal event of having another nephew being born yesterday highlights, once again, the critical role that parents and families play in the education of their children.

While school districts¹, reform advocates², parent groups³, and unions⁴ all seem to agree on the importance of “parent engagement”, the sad truth is that every single initiative I’ve encountered from these groups has been a top-down strategy that, in the end, is either too specific to apply to an individual school like mine, or too vague to be any use at all.

For example, in LAUSD, we now have a whole division dedicated to family engagement. Led by former director of the non-profit Families in Schools⁵, the LAUSD Families⁶ office has begun its work in earnest. But what exactly are they doing? Well, they’ve written a document called Parents Bill of Rights and Responsibilities⁷(a document which no one at my school seems to know exists), and developed a poll (as of now about 700 parents have responded to it, which is in the range of one- to two-tenths of a percent of my estimate of the number of parents in LAUSD). Clicking on the “School Resources” link gives very few actionable steps to take at my school, and the “Parent Resources” link is even more dismal – “Hey parents! Click here and download a giant PDF of the LAUSD Parent handbook”.

Look, Santa. Here’s what I want. And I know you can do it because it’s easy. EASY. While having a district-wide strategy is important, tell the district that they mainly need to be focused on becoming a "Parent Engagement Resource Center”, or PERC, since we seem to acronymify everything nowadays (and “verbify” everything too). What does a PERC do? Really simple. It’s main jobs are to:

• Scour the district and find schools that actually are doing parent engagement well. I KNOW there are schools out there.
• Develop a system to catalogue not only what these schools are doing, but who at the schools are doing them (more on this later).
• Include both numerical data AND, just as (if not more) importantly, anecdotal data as well.
• Allow, encourage, and foster communication between schools who want to learn parent engagement strategies with those who seem to be leading the way.

But how do they actually do this? Well, assuming that the district know which schools seem to be doing parent engagement well, here are some ideas:

• LAUSD could compile an online database with names and emails of leaders at each school with exemplary parental engagement.
• LAUSD could provide stipends for leaders at these exemplary schools to provide workshops, classes, or PDs.
• LAUSD could provide sub-time for teachers, and principals to visit schools where parent engagement is done well.

Simply put, there ARE great things going on in the district right now, but no one seems to know about them. If the district can act as a facilitator rather than a manager in this regard, schools might actually be able to see not only “what’s working” but be inspired to tweak it to meet the needs of their own school community.

Santa, make it happen!

1. LA Daily News: LAUSD aims to engage parents in schools
2. Intersections South LA: “Don’t Hold Us Back” Movement Rallies at LAUSD Meeting
3. Transparent: A grassroots organization focused on making LAUSD better for our kids!
4. Long Beach Press-Telegram: UTLA workshop will help parents teach kids over summer
5. Families in Schools website
6. LAUSD Families website
7. Parents Bill of Rights and Responsibilities

Aimee Mann – Calling on Mary

Dear Santa,

First, I hope you appreciate the format change of this blog post. Rather than embedding links, I’ve referenced all the links at the bottom of this post. Hopefully easier reading, and more knowing what you’d like to explore. Thanks and happy reading!

Waiting For Superman director Davis Guggenheim characterized charter schools as incubators. “We can take those ideas and pull them into mainstream schools,” he explains in a 2010 NPR interview [1].

To a certain extent, we’re seeing some of that right now. LAUSD and our union, UTLA, recently came to an agreement (you can view both LAUSD’s take [2] and UTLA’s take [3] on this agreement). That this agreement is characterized by some as giving charter-like autonomies [4] to campuses speaks to the fact that some do actually believe in the charters-as-idea-incubators descriptor. It is to this end that the original charter schools were created and it is to this end that I am actually in support of charter schools.

Yet, while the original intent was likely noble, the surrounding policies have created incentives that, more often than not, result in competition rather than collaboration. Again, while initially noble in nature, policies like California State Prop 39 [5] and LAUSD’s Public School Choice [6] devolved into systems that incentivized both District schools and Charter schools to not collaborate, but rather act in their own self-interests when it came to things like enrollment, data, and rhetoric.

Fast forward to 2011: We’re in a state of being in which, as a firmly committed LAUSD teacher, my mere mention of anything but vilification of charter schools will cause some to question just how firm my commitment to the district actually is.

So Santa, for this Christmas, what I’d really like is forums and opportunities to actually work with charters to explore actual strategies that might help kids. Everything from how to best serve all kids [7], to school culture [8]. Everything should be on the table as we work together to teach our kids!

Links:

1. Davis Guggenheim’s NPR Interview
2. LAUSD’s take on Tentative Agreement
3. UTLA’s take on Tentative Agreement
4. Deasy’s Twitter account regarding charter-like autonomies
5. Daily News: Charters See Win in Court Ruling (Prop 39)
6. Youtube: Deasy on Public School Choice
7. Miami Herald: South Florida charter schools admit few special needs children
8. Hechinger Report: A look inside a successful charter school culture

Smashing Pumpkins – Christmastime

In all honesty, Santa, I had written this list for you several weeks ago, not realizing that you had already started granting my wishes. It seems as if the next item on my preconceived wish list is already in the works.

Santa, what I was merely hoping for was a process that was less political, more transparent, and based on, you guessed it, better data.

However, Santa, it looks as if you’ve gone far past my wish for tweaking the Public School Choice process and gone ahead and killed it. While I am open to the idea that the original intent behind PSC was likely a noble and perhaps even forward thinking one, the process quickly deteriorated away from instructional improvements and into political posturing. While a tweak may have worked, Santa, saying goodbye to PSC may not be a bad thing insofar as it gives our schools time to improve instruction and our district time to improve the ways of measuring and supporting instruction.

The Beach Boys – I’ll Be Home For Christmas

Dear Santa: I apologize in advance for the wonkiness of the following item on my 2011 Christmas list.

So I’ve ranted plenty on this blog about the travesty that has been the manner in which education management organizations have analyzed data in recent years.

The good news is that at least our district is starting to take data a bit more seriously now. For example, up until last April, the only way to gauge a school’s performance was the results of the California Standardized Test results, which as everyone realized, but no one admitted, was largely a function of the incoming performance of the students at the schools.

In a vast, but far from perfect, improvement of data systems, LAUSD has begun having the value-added (LAUSD calls this Academic Growth Over Time, or AGT) conversation, which, despite its potential shortcomings, actually is the right conversation to be having. In a revolutionary affirming that there are non-school factors that influence student outcomes, the value-added conversation attempts to isolate school factors by controlling for these non-school factors (see below).

However, a more detailed reading of the LAUSD FAQ sheet reveals its own internal shortcomings, particularly in the way it fails to differentiate in the continuum of socio-economic status, and rather relies on the arbitrarily defined and broadly categorized partition of “Free or reduced lunch status”, as well as the potentially ignored data on peer-effects. The district does admit that more precise measures aren’t included because data for it cannot be gathered.

So if this is the status of things, then what better data could I possibly wish for?

Well Santa, if you’re listening, I’d like some fresh ideas on how to include better data in these measures, which might include new and innovative ways of measuring them. Unfortunately, the rhetoric of the district implies that these data metrics are not only solid, but are as good as we’ll ever get. This has resulted in posturing that wants to place an arbitrarily high weight on this portion of an individual teacher’s evaluation. My issue with allowing the AGT to initially have a 30% weight is that it assumes that this metric has no room for improvement. I highly doubt this. We are in right at the start of this discussion, and with time, the data metrics will be better, the applications will be more appropriate, and as a result, better data will be had. Until that day, however, the usage of AGT should start small, and then, with improvement, be given a greater weight.

So Santa, with this rant, I’m hoping for a bit of modesty among ed-policymakers. Please give them a bit of humility as they apply data metrics to our schools and our teachers, realizing that current data, while a good start, may be far from perfect. Santa, help them to make policy decisions that actually allow room for the improvement of the data metric.

So Santa, for Christmas this year, I’d like better data!