Bach – Ricercar A 6
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
On Friday, I began this series1 by introducing a phenomenon known as the Observer Effect.
To recap, the Observer Effect describes the fact that the mere act of observing or measuring something actually affects what we are observing. We also saw that the Observer Effect is not inherently a bad thing, especially when we desire a certain outcome to be measured, and the act of measuring it actually helps achieve the measurement we want. The classic example of this in the classroom setting was announcing an upcoming test, in the hopes that the test will not only measure student achievement, but that by merely announcing it, student achievement will be improved because many students will then study.
Today, we’ll take a step back and look at the general form of the Observer Effect, including the mechanisms by which it occurs. [Disclaimer: This post, as the title suggests, is fairly abstract, though not at all difficult to understand (i.e. it may satisfy your inner geek). If you are denying your don’t have an “inner geek”, then I strongly suggest skipping this post, and re-joining in with Part 3.]
To restate the Observer Effect in a different way, it makes sense to begin with the classic question “If a tree falls in the forest, and there is no one there to hear it, does it make a sound?” The answer is, of course, “yes”, however, the sound that an observer hears it make is different than the sound it makes when no observer is present2.
So our question for today is, what is the general form of the Observer Effect and how does it change that which is observed, whether physical changes (like the tree in the forest), or behavioral changes (in a classroom setting)?
To ease this meta-view, lets first look at the specific, then generalize. The following chart explains (using the “giving a test” example) how the observer effect actually works.
|
Not Observing
|
Observing
|
-
Johnnie has, to a certain degree, mastered California Algebra Standard 4.0, simplifying and solving equations
|
-
Teacher wants students to master standard 4.0
-
Teacher announces upcoming assessment
-
Johnnie, hearing his teacher announce upcoming assessment, considers his options.
-
Options may include: studying, not studying, or cheating (to name a few)
-
Teacher observers (i.e. gives the test)
-
Johnnie demonstrates a certain level of mastery of standard 4.0
|
|
Notes: The major issue with this is that no one knows to what extent Johnnie has mastered this material. His teacher doesn’t know, his parents don’t know, prospective colleges don’t know, and to a certain extent, he himself doesn’t know. This untenable situation is one of the reasons we have assessments in the first place.
|
Notes: The primary takeaway here is that (a) Johnnie’s demonstrated level of mastery is a function of the option he takes, and (b) the options he considers are a function of the teacher’s method of observation. A further (and perhaps more interesting) corollary is that the specific option Johnnie takes is a function of both the observation itself and its process. Namely, the way Johnnie prepares for the test is a function of its content (how difficult he thinks it will be) and its form (computer? pen and paper? multiple choice? essay? will it be proctored? How will it affect the overall grade?)
|
So what’s the point of this? The point of this is to show that simply the fact of observing something causes the observed results to be different than they are without observation. Furthermore, these differences are not random or arbitrary, but functions of decisions within the observer’s control. In this case, the teacher got to decide what to put on the test, how to give test, how much time, how to weight it, and more. All of these decisions in observations affect the way in which Johnnie changes his behavior towards mastery. Put differently, the observer influences changes in observation. Now in this case, all of us would probably say this is a good thing, as simply having a math test will likely cause Johnnie to study, learn, and retain more information. In short, the reason teachers put care into to content and method of assessments is because (a) we accept that Johnnie’s behavior will change knowing there’s an upcoming test, and (b) we want to encourage his changed behavior to be good behavior (i.e. studying). Teachers don’t ignore the Observer Effect, but we use it to our advantage.
For my geeky math readers out there, one way to think about this is in terms of functions. If you’re not into this, skip this box. Seriously.
|
Let’s set up some variables and functions:
-
dj: Johnnie’s demonstrated knowledge in a subject.
-
Function a: Johnnie’s specific action taken in awareness of the upcoming assessment.
-
The set of functions{a1, a2, …an}: The set of all possible actions Johnnie might take in response to being aware of the upcoming assessment.
-
t: The teacher-designed observation, in this case, the test itself.
-
ds: The demonstrated knowledge that the teacher wants Johnnie (or all students) to demonstrate in a subject.
Having defined our variables, it turns out that they are functionally related in the following ways.
-
dj= a(t)
-
a = h(a1, a2, …an)
-
{a1, a2, …an} = g(t)
-
t = f(ds)
In other words: The desired student achievement affects the way the teacher designs the observation which affects how the student perceives his options for studying which affects which options he takes which affects his actual demonstrated achievement.
The absolutely fascinating part of this is when we pause to consider the undefined functions in the equations above, specifically f, g, and h. It turns out that f is pretty boring, as it simply refers to how a teacher designs an assessment based on what he wants his students to demonstrate mastery of. The function g is more interesting in that it refers to how the test itself influences Johnnie to consider his options. Also fascinating is the function h, by which Johnnie chooses one particular option from those he considers. Simply put, the variable t actually influences these outcomes, which is why test design and observation/ assessment/ measurement in general must be taken up with such great care.
|
-
If we care about the way that Johnnie considers his options upon knowing that he will be observed, we MUST, we MUST, we MUST, take great care in designing HOW observation occurs, as its effects trickle down.
This finding can be more generally applied to all areas of observation and measurement, but in the upcoming days, I will be specifically talking more about the ways that the Observer Effect influences current teacher evaluation (a hot topic in policy right now) and the corresponding need to take great care in designing a workable observation system for the future.
2. Sound is literally air molecules vibrating in patterns that strike our ear nerves and transmit sounds to our brain. Physically standing next to the falling tree changes the path that these air molecules vibrate in towards the ear, thus changing the sound, even if unnoticeably slightly. Being in a forest to hear a tree fall is similar to seeing how waves in a lake hit the short differently depending on whether or not they are traveling around a moored boat.
Comments (0) 
As fellow ed-blogger, Joanne Jacobs, recently notes, schools should be teaching the arts for the arts’ sake.
Regardless of the analogical faux pas, Weingarten is absolutely right. Implementation IS absolutely key. In all of the current rhetoric surrounding teacher evaluation, value-added, and all of the pitfalls and opportunities therein, the biggest obstacle is not whether we can come up with good ideas, but rather can we actually make those good ideas occur.
Themocracy