06 August 2016

The TOK Presentation as Compliance, not Performance

photo credit: William Murphy

Or how TOK Presentation moderation is ruining classroom learning autonomy

Click here to read the reply to this post by the TOK Subject Manager

This year marked the second year of using the new holistic IB rubrics for marking both the Theory of Knowledge essay and presentation. It was also the second year of the TOK Presentation being subjected to moderation procedures by external IB examiners. The moderation procedures follow more or less the same procedures IB uses for all other IA moderation, using dynamic sampling and tolerance bands.

Except for one glaring difference: the work being moderated for the TOK Presentation is not the product itself, as it is with all other externally moderated IAs. Rather the work moderated is a "Presentation Planning Document (TK/PPD)," which asks the students to, in 500 words or less, describe, state, explain, outline, and show how their presentation "succeeds in showing that TOK concepts can have practical application."

Last year, May 2015, the first year of the new moderation procedures, 5 of our TK/PPD's were selected and the scores were either unchanged or moderated up 1 point. So I thought I was being sufficiently cautious and consistent with my marking, and that my students had taken the task of filling out their TK/PPD seriously and competently. Overall, I was pleased with the presentations and the new holistic rubric, which I considered an improvement over the old analytic rubric.

This year, May 2016, the moderation results flabbergasted me. Again, 5 were selected. However, this year, my scores were moderated down 1-4 points. 6s became 2s, 7s became 4s, 8s became 6s, and 9s became 8s. On the whole, I thought my students did better, overall, in creating, presenting, and explaining how TOK concepts have real-world application than did last year's class. Their reward for working hard and for me thinking I taught it better? Lower scores.

This is especially frustrating because I take the time to write 1 page teacher comments that I put on the TK/PPD, where I explicitly spell out the logic behind my marks, according to my understanding of the rubric, and the performance as a whole. IB states:

“Marks awarded by teachers for the presentation will be subject to moderation procedures through sampling of the associated TK/PPD forms that have been uploaded. The objective of this process is to judge whether the contents of the TK/PPD form justify the marks given by the teacher for the presentation.”
Shouldn't my justification of my judgements count for something? I am the one who saw the presentation.

(This isn't a one off occurrence either, the ramblings of a disgruntled and vindictive teacher; anyone with OCC access can view the TOK blog thread on the assessments year after year to view the mounting and growing vocalization of TOK teachers around the world over assessment subjectivity).

This experience has made something that was opaque before quite obvious. The TOK Presentation as an assessment is now not about performance, it is about compliance. And this is not a good development.

A performance assessment is part of a larger complex of "competency assessments" that tries to give students the autonomy to show their mastery of certain knowledge and skills is a real and relevant way. Performances are powerful learning agents, especially when students are given the opportunity to iterate and receive feedback before they are required to do it for real.

The TOK Presentation used to be that one rare IB Assessment that almost fully put the learning into the hands of the students themselves. This is true learning autonomy, for both the teacher and the student. The IB should be embracing more of these types of assessments. Why? Because they are at the heart of deep learning. Instead, they've ruined what is probably the last one.

The Presentation will now need to become an exercise in compliance to unknown moderation standards by filling out a document that is not the authentic knowledge artifact, which in turn places the actual true learning experience—the performance—as a near perfunctory gesture. Which is unfortunate.

This act of moderation itself belies sound assessment policy. Marks for whole classes of students are being judged not by the actual, authentic product itself, but by a moderated sample of planning document. Where in the real world would we allow this? Imagine...great job out in the field today Mike Trout, but you know, you didn't fill in your daily performance chart correctly, so unfortunately, we're going to mark down your accomplishments for today. On top of that, the 4 teammates we selected did a poor job too on their charts too, so everyone gets marked down. Which means you lose the game today.

It. Defies. Logic. But then, when have centralized, officially moderated assessments ever corresponded to the logic of learning? They've always been about giving adults a false sense of assurance of their rankings of students by using numbers as a proxy for the real thing.

So here is my conclusion.

Nowhere on the rubric does it indicate the criteria to fill out a TK/PPD. The rubric is explicitly about making TOK real. However, I feel I will now be forced to somehow mark the TK/PPD as the main indicator of the final presentation grade in order to approximate compliance to an unknown standard: an anonymous moderator's interpretation of what proper TOK Presentation planning looks like. And I will be forced to spend limited class time on form filling, instead of critical thinking.

We just completed our Class of 2017 presentations in May. At the bottom of my one-page comments regarding the actual performance, I will be putting the disclaimer that while this presentation deserved an 8, based on the obligation of my part to try to judge a planning document in accordance to unknown criteria, I will give instead give the final mark of 6.

In our TOK classroom the past two years, we worked hard together to scaffold the approach, to practice, to receive feedback, and to create performances we could and should be proud of. This real learning has been taken away and treated as a formality. We now get a form to fill out, and the empty promise of official, moderated, compliance.


NB1: The Official IB TOK Subject Manager wrote what appears to be a more defensive and paternalistic than substantive reply to the growing OCC discontent that addresses none of the actual concerns about moderation and only furthers the confusion...

NB2: The Official IB TOK Subject Manager wrote a reply to this post, which I posted here with further comments by me.

16 comments:

  1. As an IB ToK workshop leader, and I share your concerns. I am finding it increasingly difficult to justify - even to explain it, to be honest - what is happening and, whilst workshop leaders are not part of the IB 'proper' (we are just classroom teachers who happen to have been around the park a bit), we are nonetheless seen as such by workshop participants. It is becoming a thankless task....

    ReplyDelete
    Replies
    1. They're losing quite a few of us...

      Delete
  2. I think you hit the main nail firmly on the head: it must be the only assessment in the world where a student is given a mark based NOT upon the actual examinable work that the student has produced. That is the most iniquitous thing about it. What is being 'moderated' is your and your students' ability to fill in a form and where, as you state, with no criteria for doing so. We all have our suspicions regarding why such a change was introduced...this actually makes things worse than the thing it was meant to prevent...

    ReplyDelete
    Replies
    1. "this actually makes things worse than the thing it was meant to prevent"--Exactly!

      I just spent a block today having all of my seniors rework their TK/PPDs from last year´s presentations...

      Delete
    2. If you're interested, the TOK Subject Manager replied, and I just got around to posting it with further comments.

      http://blog.joekoss.me/2016/08/tok-presentation-as-compliance-reply-by.html

      Delete
  3. Thank you for writing this. I shared it at a TOK teacher facebook group. Hope you don't mind.

    ReplyDelete
    Replies
    1. No, please share. Hopefully together we can enact some changes.

      Delete
    2. Hi Michelle, the TOK Subject Manager replied to me via email. It took awhile but I posted her reply and my rejoinders as well today. Feel free to share again on your TOK teacher facebook group. This group seemed to get a lot of discussion and sharing going!

      http://blog.joekoss.me/2016/08/tok-presentation-as-compliance-reply-by.html

      Delete
  4. I feel like in my two years as the TOK coordinator at our school, I never really felt confident about what my students would receive. I had 7 or my 19 presentation reviewed in May 2015 and the grades were consistent. I felt that the papers on the other hand varied greatly and some of the results I got were unpredictable to say the least.

    I asked for examiners comments to better understand what had happened and each time was told we had asked too late. It was a very frustrating experience to deal with. I simply wanted to know the disconnect between my directions and the examiners view of my students work.

    I now teacher my old TOK syllabus at a non-IB School to fulfill a ministry of education philosophy requirement. The freedom to explore who students are and how they know without the external pressure of the IB and what I found to be inconsistent, often confusing results is amazing. Truly relaxing.

    ReplyDelete
    Replies
    1. My essay were all over the place this year; previous years I was pretty close except for every year I'd have a A turn to a C or D, and a C or D turn to an A. Not this year—I had my first 1 on what I deemed a pretty good essay.

      Sounds like a great gig.

      Delete
  5. Joe. Thanks so much for writing this. You echo my sentiments. This has to stop!

    ReplyDelete
  6. Thanks Joe, you have expressed my thoughts precisely. I am also one who has expressed my frustrations about this situation on the OCC. I wish I had some confidence that this situation will improve, however at the moment I have my doubts. I would really like the TOK subject manager to read your post and provide us with a response.

    ReplyDelete
    Replies
    1. Thanks Kevin. I sent a PDF copy to IB Complaints and the TOK Subject Manager sent me a quick reply but nothing in-depth. I am holding out hope for more too.

      Delete
    2. Hi Kevin, If you're interested, the TOK Subject Manager replied, and I just got around to posting it with further comments.

      http://blog.joekoss.me/2016/08/tok-presentation-as-compliance-reply-by.html

      Delete
  7. This is a disgusted by this situation. My students presentation grades were more or less cut in half. The IA feedback stated that 'the teachers comments were very helpful in awarding the final grades'. I asked for a remoderation, but have just received notification that the grades stand.

    ReplyDelete
  8. My man! great take. It's behavioral economics at its worst..."so, wait. my planning document is the most important thing, not the presentation?" Yep. "but my 10 minute presentation is longer than 500 words and more nuanced." Yep. So what you're saying is is more important than the presentation itself?" I'm not saying that, but unfortunately the IB is. Imagine the next steps! Teachers around the world running focus lessons on "how to ace your Tok/PPD Document" ridiculous.

    ReplyDelete

Your comment will be moderated before publishing.