Coded marking…

#15MinForum – 4/11/16

Our latest 15 Minute Forum was led by Lucy McDonald on the topic of coded marking. This is an idea that Lucy first started exploring as a pgce student and which she continued to develop last year when she joined us as an NQT. It is certainly a nicely refined approach to marking that sits nicely with the message we are trying to push around being as efficient as possible with written feedback (see some thoughts here on the topic of trying to ‘Mark less but mark better’).

At the heart of the approach that Lucy adopts when it comes to providing written feedback on assessments, tests, key pieces of book work etc, is the principle that any sort of feedback should require more effort on the part of the student than the teacher. This is an idea that a number of staff are exploring in a variety of ways as part of the work of our Learning Communities this year, and sits nicely in our drive to make sure students are being made to think hard as part of their learning.

Lucy’s full presentation can be seen below, but the steps are as follows:

  1. Rather than providing each student with individual written comments on their work, create a numbered list of feedback points (this could be done in advance or built up while looking at student’s work). Then, as you go through each students work, simply write down (in the margin or at the end of the work) whichever numbers relate to the appropriate feedback points for this student. Depending on the nature of the work being assessed, a student may be given several numbers, or just a couple.
  2. Students are then provided with the full list of feedback points (on the projector or via showbie) and are then given time to write out the feedback points that the teacher has indicated to be most relevant to them. By the end of this stage, every student has a number of detailed feedback points for their work, but written out by them rather than by the teacher. As well as being a significant time-saver for the teacher, there is also the added benefit that in getting the students to write the comments out themselves, they are forced to engage with that comment.
  3. The final stage – and for me the big development on similar models that I’ve seen elsewhere – is that each of the numbered feedback points also correspends to a DIRT (Directed Improvement and Reflection Time) task. Again, students write these out… and then do them! This means that not only have the students been forced to write, read and consider the feedback points, but they are also then given an improvement task that relates directly to whatever their feedback was.

 

It might not work for every task on which a teacher is wanting to provide feedback after the task has been done (as opposed to feedback in the classroom to shape the work as it is being done), but there is a lot of scope here for increasing the return on our investment of time.

Other examples of coded marking would include providing literacy codes or letters which highlight where something is missing or underdeveloped. As long as the students know what the codes mean and how the system works, it would seem that any sort of coding could be very valuable! I think there is also scope to link this sort of approach to one where feedback is given, coded or otherwise, but the student then has the job of locating the section of the work to which the feedback refers. For example, on simple recall questions, why not tell a student they have got X number correct and Y number wrong, and then let them try and identify which ones were right and which were wrong…

For more suggestions of ways to reduce the time spent by teacher’s providing lengthy written feedback, this article from @joe_kirby has some thought-provoking suggestions!

 

See Lucy’s full presentation here…

This slideshow requires JavaScript.

Advertisements

‘mark less, but mark better’

 The EEF’s review into marking, published last month, has come at an opportune time as we continue to embed and refine our approach to written formative feedback.

Is it groundbreaking? No. Is it worth a read anyway? Yes.

3242197170_3950b7b362_bPhoto Credit: *janine* via Compfight cc

We all know the score with regards to teacher workload – this report from the TUC in February of this year provides some interesting contextualising figures:

The most unpaid overtime is done by teachers and education professionals (with more than half of them working an average of 11.9 hours unpaid every week)

… and I’d wager that there are more than a few teachers who occasionally – or even routinely – do more than this average! While such figures will inevitably continue to colour the perception of many  outside the profession, making a challenging climate for recruitment even more so, our focus at the moment is on doing what we can to support those staff that are already in our school to help them find manageable balance.

According to the Government Response to the Workload Challenge, published in February last year, 53% of those who participated in the survey identified marking as one of the areas that represents opportunity to reduce workload (only ‘recording, inputting, monitoring and analysing data’ featured more often in responses, at 56%). At around the same time as this report was published, we started the process of rethinking our assessment policy…

 

Borrowing from a phrase that I’d heard Christine Harrison use at a conference where she spoke about her work on AfL, one of the guiding principles for a central policy that we knew needed to work across the whole school in a range of contexts was the idea that we wanted consistency of principle rather than needing uniformity of practice. To this, we added the mantra (in relation to written assessment) that it should be done at the right time, for the right reasons (that is ‘to support the progress of students‘ rather than ‘to prove to an observer/ inspector/ line manager that I do it’!), and off we set…

Much of the final document focussed on the written feedback (i.e. ‘marking’) side of things; the classroom-based side of assessment and feedback (i.e. the ‘short cycle’ formative assessment I referred to in this post) is picked up elsewhere through our focus on classroom practice in the learning and teaching programme. It sets out minimum expectations and core principles (whilst avoiding being unnecessarily directive or prescriptive)  in terms of frequency of formative feedback and the importance of students  being given time to reflect and respond to feedback (DIRT) etc. It also prompted us to make a few potentially risky decisions (for good reasons!), for example removing half-termly data drops, opting instead for a ‘live’ system. This allows subject areas and class teachers to add interim assessment data as and when summative assessments are completed, thus allowing schemes of learning to be planned and scheduled in a way that makes sense for the learning and development of ideas rather than scheduling them just so that the assessment data from that unit can be included in an arbitrary data drop each half term.

thumbs up

Photo Credit: MartinShapiro via Compfight cc

Is it all working perfectly? Not yet. However, we are convinced that the principles are the right ones and we are taking every opportunity to remind staff that we want them marking at the right times and for the right reasons: we care about staff wellbeing and we care about the learning experience of our students… if we want our staff to work sensible hours, then we recognise that there may be occasions where compromises have to be made: we want staff to make decisions about judicious, high quality use of the red pen in a way that maximises impact at key times, and ensure that time is prioritised for the planning of great learning experiences across lessons and units.

In this post on sharing effective practice in relation to marking and using this to refine practice across the school (emphasising the long-term developmental focus rather than short-term monitoring of compliance), Stephen Tierney (@leadinglearner) shares some interesting ideas on moving a policy from words on a page to tangible changes in practice. There are certainly some things for us to attend to in this respect over the remainder of the academic year – creating opportunities for staff to see the detail of what is happening around the school and what seems to be working, not only from the point of view of workload, but also from the point of view of what effective written feedback actually looks like: if we’re looking at written feedback from the point of view of economy and efficiency, we have to look carefully at what has the most impact. We have work to do around ensuring it has impact for the students: impact that they are able to reflect on in a meaningful way and can articulate – for their own benefit, but also to others – the specific links between the feedback they are being provided with and the progress they are making.

Clearly, this has to be done alongside ongoing reflection on what the research is telling us, both that which is being gleaned as part of the work of one of our Professional Learning Project groups and also the larger-scale and more robust findings of the EEF’s long-awaited review into marking, published last month. Although one of the key messages from the document is that the evidence is actually fairly scant in relation to the impact of marking, don’t let that put you off reading it – there are still suggestions that emerge from the research that does exist, and many of these are to do with the fine details of how we mark, rather than recommendations that would impact on the broad-strokes with which we have set out the principles of our assessment policy.

 

“Does our marking approach require our pupils to work to remember or reach the correct answer?” p12.

This is the question that I think struck me most in the whole report. It isn’t necessarily the most significant point, but given our recent reflection on the way we are using summative assessments formatively and the conclusions we’ve reached about the need for students to think hard for themselves, this seems like a potentially useful rule of thumb in terms of considering whether our marking is focussed on surface-level corrections or development of deeper understanding.

The contention presented by the EEF review seems to be that mistakes (something a student can do but has not on this occasion) should be marked as incorrect but left for students to correct themselves, while errors (resulting from misunderstanding or having not yet mastered something) should perhaps be dealt with by providing hints or questions to lead the students to developing a more complete and accurate understanding of the topic at hand.

Although the distinction between errors and mistakes isn’t made in the same way in this transcript from a talk given by Dylan Wiliam, he does place a similar emphasis on the idea that it should be a standard expectation that students are expected to do the thinking:

We suggested that instead of telling students that they got 15 out of 20, the [maths] teacher could, instead, tell them that five of their answers were wrong, and that they should find them and fix them.  The important feature of this feedback, like comment-only marking, is that it engages students, leaving them with something to do. This technique was subsequently adopted by English teachers when they provided feedback on students’ final drafts of writing assignments. Rather than correcting spelling, punctuation and grammar, the teachers put a letter in the margin for each error in that line using a G if for an error in grammar, an S for a spelling mistake, a P for a punctuation, and so on. For the stronger students, the teacher would simply put a dot rather than S/P/G in the margin for each error,  and for the weaker student, the teacher might indicate where in the line the error was.  The idea is that the feedback gives something to the learner to do so that the immediate reaction of the learner is that they have to think.

Elsewhere, in this fairly weighty review of research on formative feedback from Professor Valerie Shute, though not specifically about written formative feedback, there are some interesting comments about the idea of ‘directive feedback’ (providing corrective information) as opposed to ‘facilitative feedback’ (providing guidance and cues):

“Conventional wisdom suggests that facilitative feedback…would enhance learning more than directive feedback…yet this is not necessarily the case. In fact, some research has shown that directive feedback may actually be more helpful than facilitative—particularly for learners who are just learning a topic or content area (e.g., Knoblauch & Brannon, 1981; Moreno, 2004). Because scaffolding relates to the explicit support of learners during the learning process, scaffolded feedback in an educational setting may include models, cues, prompts, hints, partial solutions, as well as direct instruction (Hartman, 2002). Scaffolding is gradually removed as students gain their cognitive footing, thus directive feedback may be most helpful during the early stages of learning. Facilitative feedback may be more helpful later on, and the question is when. According to Vygotsky (1987), external scaffolds can be removed when the learner develops more sophisticated cognitive systems, where the system of knowledge itself becomes part of the scaffold for new learning.”

So, like everything else to do with learning and teaching, it isn’t straightforward (and this is before we’ve even get into John Hattie and Helen Timperley’s work on the power of feedback and the importance of considering the ‘level’ at which feedback is directed: ‘task’, ‘process’, ‘self-regulation’ or just ‘self’…)

It is unlikely to be easy to prescribe for teachers in simple black and white exactly what they should do and when… and nor should we need to, if we have faith in their professional judgement and intuition – based on a detailed understanding of each individual – about what sort of feedback is most appropriate at any point in time for any single student. Perhaps the emphasis should be on making sure that our staff are proficient in working with a range of strategies and that they have an appreciation for the rationale behind which strategies can work and when, then trust them to make the right decisions.

 26642624651_a00b392734
Photo Credit: nicolegalpern1 via Compfight cc

 

The EEF review makes various other suggestions based on the evidence that is already available, some of it reassuring in terms of the direction we are heading, some of it giving pause for thought and highlighting areas that we would do well to put more thought to… but then if anyone in education reads a report like this and concludes that they’ve got it all nailed already, I suspect we could say that they either haven’t read the report properly, they haven’t really understood it, or they don’t really have a true appreciation of what is going on in their own setting…

Is anything in the report groundbreaking? No, but it offers some tangible suggestions around which we could do some developmental work with staff to ensure we get the most impact for the most reasonable amount of input. I’m looking forward to reading whatever comes next…

 


 

As we continue to look outside our own school to learn from others, I’m also rather intrigued by the idea of ‘marking the Michaela way’, which seems like a minimal (to say the least) whole-school approach to marking which could have a lot going for it… A thought provoking read!

A few more interesting reads…

Making summative assessments formative

#15MinForum – 10/5/16

This week’s 15 Minute Forum was led by Nikki Cloudsdale (@NikkiCloudsdale), our Maths Subject Leader, with some suggestions for taking summative assessment tasks (tests, exam papers etc) and turning them into formative learning strategies (perfect timing, having just finished internal exams for year’s 7 and 9, and with exam groups in years 10-13 ploughing through past papers!)…

Teachers Marking
Photo Credit: superdave3451 via Compfight cc

In a webinar that I attended a couple of months ago (if one ‘attends’ a webinar?!), I listened to Terry Morgan and Beth Carr from the Learning Sciences Dylan Wiliam Center talk about ‘The Case for Short Cycle Formative Assessment‘. Much of the webinar, like the others in the series, revolved around unpacking the key dimensions of formative assessment (and the compelling evidence base that underpins it):

formative assessment overview

Credit ‘The Case for Short Cycle Formative Assessment

In this particular webinar, this was specifically set in the context of short-cycle formative assessment (i.e. the minute-by-minute processes) as opposed to medium- or long-cycles, which look at assessment between teaching units or across units and terms:short cycleCredit ‘The Case for Short Cycle Formative Assessment

Listening to Nikki talking through some of the strategies that the maths department are playing with at the moment, I was struck by the potential for some of the strategies to help tie the different ends of the cycles together, taking summative assessments completed to assess progress across entire units or even entire key stages, and then using them as the basis for learning activities rooted back in the short-cycle formative assessment processes.

Nikki started by outlining the journey that the Maths department has been on in the last 18 months with regards to assessment within the department, highlighting some of the features of their current approach, including tracking sheets that students keep in their exercise books on which they record progress in various tests alongside the key feedback from their teacher (in the form of WWW/EBI) for each key objective; formative assessment tasks and feedback as they move through a unit of work; then a final summative assessment which is graded. But, as Nikki herself put it, the grade is just one part of their feedback – the most important part comes next…

At the heart of the strategies listed below, is the idea that reviewing a summative assessment should only rarely (if ever?!) be about simply showing the students a mark scheme and talking through it. Rather, it should be about engaging the students in meaningful reflection – with them in the driver’s seat – on the hows, whys and wheres of their mistakes and misunderstandings in order to help them move their learning forwards.

Here are a few of the ideas:

  • Peer assessment of a selection of responses to a question: project photos of selected responses or share them on nearpod, and charge students with marking it (without looking at the mark scheme!) and then perfecting it. Round it off by asking students to suggest a suitable improvement target for this individual, forcing students to think about the skill, exam technique, or approach (metacognition!)
  • Champions of a question: identify areas of strength within the test and make every pupil a champion of some aspect of the assessment. Provide opportunity for students to move around to find the solution from the champion, or use a jigsaw seating approach to help structure the sharing and collaborating (activate students as resoures for each other!)
  • Develop a model solution/answer: working in small groups or pairs, task students with developing model solutions/answers for themselves (taking ownership of their own learning!) before then rearranging groups for peer-teaching (activating them as resources!)
  • And, of course, at the end, having used the assessment to identify priority areas (some sort of gap analysis), point students in the direction of specific resources that they can use to support them with these specific priority areas… and provide them with tme to actually do it!

I wonder if we might really get them thinking carefully if, when marking the work, we were to give the question a score, but not use any ticks to show which aspects of the answer gained the credit. The students are then tasked with devising their own mark scheme to work out for themselves specifically where they did and didn’t gain the marks…

In all of these strategies, we could argue that the teacher should be acting as a conductor rather than a controller, freeing them up to coach and probe students in the construction of their own understanding, rather than simply telling them what the mark scheme wanted them to write…

Doing so will surely encourage the students to be full involved in reflecting on the important part of any test: what it is that they need to attend to in order to move forward… (but then we all know that, according to John Hattie at least, the point of testing is not really about telling the students anything at all, but rather it is to find out what you as the teacher did well – who did you teach well and who not so well, what did you teach well and what not so well, and so on“………….)