Rules for Writing Quality Criteria

Good quality criteria should:

1. Enable an inference to be made about developmental learning – there should be no counts of things right and wrong or pseudo-counts (e.g. some, many, etc.)

• Counts and pseudo-counts do not give an indication of quality or sophistication and their use can discourage students from testing their own limits. For example, a rubric that counts the number of words used incorrectly may encourage an ambitious student to play it safe by using only one-syllable words rather than trying to use more complex language.

2. Avoid language that is ambiguous or contains comparative terms (e.g. appropriate, suitable, adequate) to define quality of performance

• Subjective terms can lead to inconsistent marking and disagreements during moderation about, for example, what constitutes ‘appropriate’ or ‘suitable’. Even when teachers have a consistent view as to what these terms mean, it does not provide students with guidance about what is required.

3. Discriminate between performances of increasing quality without procedural steps in a sequence of operations

• If three steps are necessary in order to complete parts of a task, rubrics sometimes turn these procedural steps into three criteria (completes step one, completes step two, etc.). This implies that the student who reaches the end of the sequence of operations has shown greater quality than the student who only completes step one. However, each of the steps can be performed with differing degrees of quality, so each should have its own quality criteria. This allows a judgement to be made about the difference between a student who completes step one only, but does so to an amazingly high level, and a student who just scrapes through all three steps.

Good quality criteria should also:

4. Describe performances such that each successive description implies a progressively higher level of performance quality

• Developmental taxonomies can be very useful in helping to identify progressively higher levels of quality. See Chapter 2 for examples.
• This ensures that the quality of the student’s performance is recognised in the rubrics used to judge quality. If teachers find themselves realising when marking that the rubrics being used do not allow them to recognise the quality in the student work, then there is something wrong with the rubrics and they should be reviewed.

5. Contain one central idea that can be recognised

• If more than one idea is contained in a criterion, it becomes difficult to judge which should be selected when a student has achieved one part but not the other.

6. Be directly observable (do, say, make, write) and avoid negatives

• When criteria are directly observable, teachers are not required to make inferences in order to assess the work. Within a set of quality criteria there is no need to specify what the student cannot do, as this is implied by the higher criteria.

7. Reflect typical behaviours that cover a diverse range of quality, including a stretch for the most proficient

• When assessment rubrics are interpreted in a criterion-referenced manner – rather than being converted to a percentage achieved, as is common practice – students are not penalised for not reaching the highest criteria. This allows teachers to set aspirational targets for the highest achievers, so all students are encouraged to stretch their capabilities. Often teachers are surprised when students achieve criteria that were considered aspirational, providing an important opportunity for teacher learning.

8. Self-weight based on their capacity to separate by performance quality, i.e. no weightings are to be used

• Many traditional assessments give greater weight to some rubrics than others. Weightings are often used because teachers recognise that some rubrics are harder than others and they wish this to be recognised when the rubrics are converted into a score or percentage. With a criterion-referenced interpretation, this reason for weighting some rubrics over others disappears, as the rubrics are mapped to levels on a developmental progression, not a percentage. This allows the assessment data to be used directly to plan teaching.
• In other situations, teachers may weight rubrics to give students an indication of the time/effort they should spend on each component. This is a legitimate use of weightings and can easily be achieved through an accompanying instruction.

9. Have four or fewer criteria for any indicator (to support consistency of judgements)

• Experience in many instances has shown that when more than four criteria are used teachers struggle to distinguish consistently the different levels of quality, because the differences between them are too difficult to recognise. Distinctions are more easily made when the jumps in quality are larger. This results in more consistent judgements.

10. Be transparent so persons assessed can verify their assessment – no jargon.

• It is ideal if all parties understand the criteria fully. This allows students to have greater control over their own learning and facilitates open communication between students, parents and teachers. Transparent criteria allow students to self-assess, which helps them make decisions regarding the way they utilise their time and energies.

Rules 1 to 3 (in red) are the core rules. Rubrics that do not follow these rules require learners to guess what the assessor is looking for. This is like asking learners to jump but providing no answer to the question “How high?” Rules 4 to 10 are additional rules that increase the usefulness of rubrics. Intriguingly, Rules 1 to 3 are the easiest to use when evaluating existing rubrics, but are the hardest to apply when learning to write rubrics. You will find that rubric writing is a challenge that rewards persistent effort.