|photo credit: William Murphy|
Or how TOK Presentation moderation is ruining classroom learning autonomyClick here to read the reply to this post by the TOK Subject Manager
This year marked the second year of using the new holistic IB rubrics for marking both the Theory of Knowledge essay and presentation. It was also the second year of the TOK Presentation being subjected to moderation procedures by external IB examiners. The moderation procedures follow more or less the same procedures IB uses for all other IA moderation, using dynamic sampling and tolerance bands.
Except for one glaring difference: the work being moderated for the TOK Presentation is not the product itself, as it is with all other externally moderated IAs. Rather the work moderated is a "Presentation Planning Document (TK/PPD)," which asks the students to, in 500 words or less, describe, state, explain, outline, and show how their presentation "succeeds in showing that TOK concepts can have practical application."
Last year, May 2015, the first year of the new moderation procedures, 5 of our TK/PPD's were selected and the scores were either unchanged or moderated up 1 point. So I thought I was being sufficiently cautious and consistent with my marking, and that my students had taken the task of filling out their TK/PPD seriously and competently. Overall, I was pleased with the presentations and the new holistic rubric, which I considered an improvement over the old analytic rubric.
This year, May 2016, the moderation results flabbergasted me. Again, 5 were selected. However, this year, my scores were moderated down 1-4 points. 6s became 2s, 7s became 4s, 8s became 6s, and 9s became 8s. On the whole, I thought my students did better, overall, in creating, presenting, and explaining how TOK concepts have real-world application than did last year's class. Their reward for working hard and for me thinking I taught it better? Lower scores.
This is especially frustrating because I take the time to write 1 page teacher comments that I put on the TK/PPD, where I explicitly spell out the logic behind my marks, according to my understanding of the rubric, and the performance as a whole. IB states:
Shouldn't my justification of my judgements count for something? I am the one who saw the presentation.
“Marks awarded by teachers for the presentation will be subject to moderation procedures through sampling of the associated TK/PPD forms that have been uploaded. The objective of this process is to judge whether the contents of the TK/PPD form justify the marks given by the teacher for the presentation.”
(This isn't a one off occurrence either, the ramblings of a disgruntled and vindictive teacher; anyone with OCC access can view the TOK blog thread on the assessments year after year to view the mounting and growing vocalization of TOK teachers around the world over assessment subjectivity).
This experience has made something that was opaque before quite obvious. The TOK Presentation as an assessment is now not about performance, it is about compliance. And this is not a good development.
A performance assessment is part of a larger complex of "competency assessments" that tries to give students the autonomy to show their mastery of certain knowledge and skills is a real and relevant way. Performances are powerful learning agents, especially when students are given the opportunity to iterate and receive feedback before they are required to do it for real.
The TOK Presentation used to be that one rare IB Assessment that almost fully put the learning into the hands of the students themselves. This is true learning autonomy, for both the teacher and the student. The IB should be embracing more of these types of assessments. Why? Because they are at the heart of deep learning. Instead, they've ruined what is probably the last one.
The Presentation will now need to become an exercise in compliance to unknown moderation standards by filling out a document that is not the authentic knowledge artifact, which in turn places the actual true learning experience—the performance—as a near perfunctory gesture. Which is unfortunate.
This act of moderation itself belies sound assessment policy. Marks for whole classes of students are being judged not by the actual, authentic product itself, but by a moderated sample of planning document. Where in the real world would we allow this? Imagine...great job out in the field today Mike Trout, but you know, you didn't fill in your daily performance chart correctly, so unfortunately, we're going to mark down your accomplishments for today. On top of that, the 4 teammates we selected did a poor job too on their charts too, so everyone gets marked down. Which means you lose the game today.
It. Defies. Logic. But then, when have centralized, officially moderated assessments ever corresponded to the logic of learning? They've always been about giving adults a false sense of assurance of their rankings of students by using numbers as a proxy for the real thing.
So here is my conclusion.
Nowhere on the rubric does it indicate the criteria to fill out a TK/PPD. The rubric is explicitly about making TOK real. However, I feel I will now be forced to somehow mark the TK/PPD as the main indicator of the final presentation grade in order to approximate compliance to an unknown standard: an anonymous moderator's interpretation of what proper TOK Presentation planning looks like. And I will be forced to spend limited class time on form filling, instead of critical thinking.
We just completed our Class of 2017 presentations in May. At the bottom of my one-page comments regarding the actual performance, I will be putting the disclaimer that while this presentation deserved an 8, based on the obligation of my part to try to judge a planning document in accordance to unknown criteria, I will give instead give the final mark of 6.
In our TOK classroom the past two years, we worked hard together to scaffold the approach, to practice, to receive feedback, and to create performances we could and should be proud of. This real learning has been taken away and treated as a formality. We now get a form to fill out, and the empty promise of official, moderated, compliance.
NB1: The Official IB TOK Subject Manager wrote what appears to be a more defensive and paternalistic than substantive reply to the growing OCC discontent that addresses none of the actual concerns about moderation and only furthers the confusion...
NB2: The Official IB TOK Subject Manager wrote a reply to this post, which I posted here with further comments by me.