10 assessment myths?

I believe that the following 10 opinions on assessment are mistaken. Do you agree with me?

  1. For the sake of fairness, all students should be assessed identically.
  2. The more summative assessment there is, the greater the impact on learning.
  3. If a higher proportion of students achieve the top grade, standards must be slipping.
  4. Once a marking team has agreed upon essay criteria and standards, consensus will be achieved on grades for individual essays.
  5. There are no valid ways to assess transferable skills such as teamwork or communication.
  6. Formative assessment benefits all students in equal measure.
  7. It is not feasible to measure abstract qualities such as personal integrity or multicultural awareness.
  8. It is best to show students their grades before providing qualitative feedback.
  9. If a student has received quality feedback on a formative assessment task, they will be completely prepared to perform better on a subsequent, related summative assessment.
  10. Setting a summative assessment is the most effective way to motivate learners.

 

A mini-guide to lesson observations (part 2)

In part 1, I made the recommendation that identifying the purposes of lesson observations is vital and clarified several common purposes. I also highlighted the importance of establishing and following quality processes to enable worthwhile outcomes from observations.

Now, in part 2, I look in detail at the skills required by observers and observees at different stages of lesson observations.

Before the observation

Communicate /Listen actively

One of the main purposes of a pre-observation meeting is to establish the context. Therefore, the observee should endeavour to provide the observer with as a full a picture as possible of the learning environment. If the observer is already familiar with the school/department, then discussion can focus on the nature of the learners, their relative progress, any factors that hinder or help learning, etc. The observer can listen and ask for clarification if the observee is not clear enough or makes assumptions about the observer’s prior knowledge of the situation.

Design suitable observation instruments

For appraisal, there is likely to be a standard observation form, but for professional development and action research, the observer and observee can work together to narrow down a focus and design an observation instrument accordingly.

Lessons are complex events and there are many possible aspects that can be observed. Here are some examples:

  • Planning and preparation
  • Learning outcomes – transparent and achieved?
  • Effectiveness of learning materials
  • Learning-teaching strategies
  • Classroom dynamics
  • Behaviour management
  • Motivation of students
  • Questioning skills
  • Interaction patterns

For example, if the observee was concerned about student participation and the balance between teacher talking time and student talking time, then a simple instrument to quantify participation could be designed. The observer could be provided with the class seating plan, and, when observing, tick the teacher and students’ names on each occasion that they made a contribution to a class discussion.

Online there are many sources of observation instruments, which can be used as they are, or adapted to circumstances. There also examples in Brown, Jones and Rawnsley (referenced in Part 1 of this guide) and Scrivener (2005), amongst other published guides to teaching.

During the observation

The observee will be busy teaching (and reflecting in action), of course, so the skills I list here are for observers.

Remember what to record

The observer should keep in mind the agreed foci. It is easy to become distracted and start commenting on other aspects of the lesson. I think each observer has preconceived notions of “good teaching” and it is tempting to focus on what the observer considers important rather than what was agreed before the observation. For example, it is a pet hate of mine, as an observer, when teachers omit to check comprehension of concepts before proceeding to practice tasks. However, if the agreed focus is something else, then I need to ignore that perceived failing.

Write two kinds of notes

When the observer makes notes about what takes place during the lesson, then the notes can be of two varieties. Firstly, notes that are a “…non-judgmental description of classroom events…” (Gebhard, 1999). I would recommend using past simple tense when writing such narrative notes, e.g. ‘The teacher gave clear verbal instructions for the learning task together with a quick demonstration.’ If the observation is for research or professional development, arguably narrative notes are sufficient. However, if appraisal is required, then alongside narrative notes the observer can write evaluative comments, e.g. ‘Most learners understood the learning task from the teacher’s explanation and demonstration, but two groups started off doing the task in the wrong way, implying that the teacher had needed to check instructions.’

Be as unobtrusive as possible

Observers commonly sit outside the line of sight of most of the students. This is done to reduce distraction and the effect of the observer being present (known as the Hawthorne Effect). The observer can select a seat to view the outcomes of the observee’s actions and student activity can be seen and heard well. Probably, the best position will be indicated by the focus of the observation.

Sometimes, the observer may need to stand up and walk around to see the work that students are doing, or move closer to be able to monitor a particular group. It is a good idea to make sure that students are prepared for this eventuality before the lesson. A simple technique to reduce the inhibitions students may feel is for the observer to look at one group while really listening to another group.

I would also recommend that observers and observees agree beforehand not to make eye contact during the lesson, so as not to influence the teacher’s confidence. Observers should also be mindful of their facial expressions in case the lecturer/teacher does look their way.

After the observation

At the post-observation meeting, both observers and observees will need to exercise numerous skills to get the maximum benefit from the experience, especially in formal appraisal situations:

  • Building rapport and developing trust
  • Active listening
  • Neutral questioning to elicit observees’ uninfluenced views on their lessons
  • Giving and receiving constructive criticism calmly and professionally
  • Counselling and conflict management

Specifically, observers should:

  • be accepting of alternative teaching strategies as long as learning objectives are met successfully
  • focus on actions and results of actions (rather than character traits)
  • be as specific with praise as they are with criticism, and support opinions with evidence
  • use concrete vocabulary when describing what was observed
  • comment only on behaviours that can be changed
  • ask questions that provoke reflection, but not leading or loaded questions
  • suggest concrete action points for future development

Meanwhile, observees need to:

  • give an honest and balanced self-evaluation
  • support the self-evaluation with evidence
  • be open to alternative strategies suggested by the observer
  • use the observer as a resource to plan future development in a practical, concrete way

References

Borich, G. D. (1994). Observation skills for effective teaching. New York : Merrill.

Brown, S., Jones, G. & Rawnsley, S. (Eds) Observing Teaching SEDA Paper 79

(Birmingham, Staff and Educational Development Association): 19–22.

Cowan, J. (1998). On becoming an innovative university teacher. Buckingham, UK : SRHE and Open University Press.

Gebhard, J.G. (1999). ‘Seeing Teaching Differently Through Observa­tion’, in Gebhard, J.G. & Oprandy, R. (eds), 1999, Language Teaching Awareness, Cambridge University Press, Cambridge

Higher Education Academy Resource Pages on Peer Observa­tion of Teaching: http://www.heacademy.ac.uk/resources/detail/subjects/escalate/1043_Peer_Observation_of_Teaching

Montgomery, D. (2002). Helping teachers develop through classroom observation. London : David Fulton.

Schön, D. A. (1983). The reflective practitioner : How professionals think in action. New York : Basic Book.

Scrivener, J. (2005). Learning teaching: A guidebook for English language teachers. Macmillan ELT.

A mini-guide to lesson observations (part 1)

The purpose of this short guide is to help teachers/lecturers and their managers to:

  • be precise about objectives in observing or being observed,
  • follow lesson observation processes accordingly,
  • raise awareness of the skills sets of accomplished observers and observees, and thereby
  • set about participating in lesson observations with increased confidence.

(My apologies if this guide sounds rather dogmatic. I have, however, based it on relevant literature and my experience as a teacher trainer.)

Objectives of lesson observations

  1. Performance appraisal
  2. External inspection
  3. Development of teaching skills
  4. Action research

1. Performance appraisal / 2. External inspection

It is common for teaching staff to be evaluated once or twice a year on their teaching performance through observation. The observer may be a department head or school vice-principal. Some would question the reliability of this form of appraisal, particularly in the case of inspections. See this article by Nick Morrison on Forbes for the arguments against and further references: http://www.forbes.com/sites/nickmorrison/2014/03/19/lesson-observations-are-no-way-to-grade-teachers/

However, for many teaching professionals it is still a reality to be faced. So, it is important to ensure that it is done as carefully as possible. There are many ways to conduct a lesson observation, but I suggest a three-step process based on reflective practice:

Reflection for action (Cowan, 1998)

    • Before the observation

Reflection in action (Schön, 1983)

    • During the observation

Reflection on action (Schön, 1983)

    • After the observation

Before the observation

Ideally, there should be a face-to-face meeting beforehand to clarify several matters. As Brown (1993) said, “It is not enough simply to devise a universal checklist and send line managers out to do it.”

    • The teacher tells the observer about the learners and the lesson objectives
    • The observer takes note of any special considerations
    • The teacher & observer agree on the observation process and focus
    • The teacher & observer make sure they understand the appraisal criteria & standards in the same way. For example, what does “maintains good pace of learning” mean?

During the observation

    • The observer tries not to distract the learners or the teacher
    • The observer pays attention to all criteria that were selected
    • The teacher focuses on the learners and learning and teaches as normally as possible
    • The observer summarises impressions by the end of the lesson

After the observation

    • The teacher completes a self-evaluation before forgetting the details
    • The observer considers the teacher’s self-evaluation and adjusts appraisal or feedback accordingly
    • The observer and teacher exchange views on how well assessment criteria were met, or on the focus of the lesson observation
    • The observer and teacher decide upon future priorities and concrete action points

Professional development

By contrast, this type of lesson observation is characterized by the following terms: voluntary, forward-looking, formative and constructive. Such an observation can be beneficial for lecturers/teachers at any stage of development.

Frequently, the observer is a peer, or possibly an external consultant without vested interests. The role of the observer is to act as a trusted colleague to assist their partner in reflecting on their teaching. The observer does not need to be more knowledgeable about learning-teaching approaches.

I would argue that written impressions by observers should not be recorded and stored in teachers’ records. Otherwise, there is a danger of them being used to inform decisions about contract renewals or promotions.

“The process of observation should be developed between those staff involved.” I suggest that the three-step process above could still be helpful, minus the appraisal element.

The focus of observation is also negotiable. For example, lecturers teaching the same subject could witness alternative ways of presenting the same content, or developing the same skills in students. Alternatively, a lecturer could experiment with a revised learning task and get a second opinion on its design and effec­tiveness. In order to ensure that the observer’s attention remains on the selected focus, an observation instrument can be devised.

Action research

Action research observations are similar in nature to professional development observations in that they are non-evaluative of the lecturer/teacher. They are distinguished by, for instance, their greater formality, use of more precisely designed observation instruments and pre-conceived ways to process collected data. Findings are written up for publication or presentation and may inform revisions in learning/teaching practices or curricula design.

In part two of this mini-guide, I will provide more detail about the skills of participating in lesson observations.

References

Borich, G. D. (1994). Observation skills for effective teaching. New York : Merrill.

Brown, S., Jones, G. & Rawnsley, S. (Eds) Observing Teaching SEDA Paper 79 (Birmingham, Staff and Educational Development Association): 19–22.

Cowan, J. (1998). On becoming an innovative university teacher. Buckingham, UK : SRHE and Open University Press.

Higher Education Academy Resource Pages on Peer Observa­tion of Teaching: http://www.heacademy.ac.uk/resources/detail/subjects/escalate/1043_Peer_Observation_of_Teaching

Montgomery, D. (2002). Helping teachers develop through classroom observation. London : David Fulton.

Schön, D. A. (1983). The reflective practitioner : How professionals think in action. New York : Basic Book.

Managers’ feedback and its impact on development & motivation

Recently, I read an article by Anmol Jain highlighting the role of continuous feedback in motivating junior colleagues. I appreciated the way it was written because he provided a concrete example of feedback from a manager to a team member:

‘The deliverable you produced is very nice and very close to my expectations, but if you can just make these few minor changes it would really stand out’.

This example exhibits features of worthwhile feedback. The manager indicated that the deliverable was almost aligned with expectations. As long as the team member knows and understands the expectations, this is useful information. The manager goes on to detail changes that would enhance the deliverable, thus refocusing the team member’s actions and efforts. The manager could also have commented on the efficiency of processes by which the deliverable was produced, if those had been observed. The feedback aids motivation because it is constructive, acknowledging what has been achieved already and enabling further progression.

To build upon Anmol’s article, I thought that I would provide a few more details on feedback summarised from educational research. If the arenas of education and business are sufficiently relatable, my hope is that this might prove useful information for managers.

There are several ways that managers can respond to completed work tasks and they are not all beneficial.

Try dividing the following types of feedback according to their effects on performance and achievement. Which do you believe have a positive impact, and which a negative impact?

  1. giving material rewards for a high quality deliverable
  2. directly correcting errors in the deliverable
  3. providing prompts to enhance the quality of the deliverable
  4. confirming what is good about the deliverable
  5. inviting peer evaluation of the deliverable
  6. penalizing flaws in the deliverable
  7. verbally praising a good quality deliverable
  8. clarifying how the deliverable meets or doesn’t meet expectations

Well, perhaps surprisingly, the following negatively impact on achievement and enthusiasm:

1. Rewards

2. Correction

6. Penalties

7. Praise

Why so? Because they are controlling strategies. Generally, people dislike being coerced, and, in the case of correction, by taking control a manager is effectively doing the task rather than letting the junior staff member develop so that s/he can do it better next time. Praise by itself is not useful because it does not describe what is good and therefore does not inform future development.

My recommendation to managers, therefore, is to utilise feedback types 3, 4, 5 and 8 above.

References

žBrookhart, S.M. (2008). How to give effective feedback to your students. ASCD.

Butler, D.L. & Winne, P.H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245-281.

Hattie, J. & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77, 81 – 112.

Key research findings about formative assessment

Formative assessment has a significant impact on learning, as much as 1 or 2 grades on GCSE* results.

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.

(This seminal article was a meta-survey of over 250 publications linking assessment and learning.)

Students can achieve learning goals when they (a) understand the goals, (b) feel like those goals are personal goals, and (c) can evaluate their own progress during courses.

Sadler, D.R. (1989). Formative assessment and the design of instructional systems. Instructional Science 18, 119-144.

Formative assessment promotes effective learning and raises the quality of teaching.

Black. P, Harrison. C, Lee. C, Marshall. B, William, D. (2003). Assessment for Learning: Putting it into practice. Oxford University Press.

Learning gains from formative assessment are disproportionately greater for less-able students.

Assessment Reform Group (2002). Assessment for learning – 10 principles: Research-based principles to guide classroom practice. ARG/Nuffield Foundation.

Formative assessment encourages students to take an active role in the management of their own learning.

Juwah, C. et al. (2004) Enhancing Student Learning Through Effective Feedback. The Higher Education Academy.

Purely formative assessment, with no summative standard measurement, is ineffective.

Smith, E. & Gorard, S. (2005). ‘They don’t give us our marks’: The role of formative feedback in student progress. Assessment in Education, 12(1), 21-38.

*GCSE = General Certificate of Secondary Education in England, Wales & Northern Ireland

The formative and summative assessment disambiguator

Assessment for learning

Assessment of  learning

Also known as…

Formative assessment

Summative assessment

Purposes

To  diagnose student difficulties in reaching learning objectives; To identify student strengths that can be built upon; To inform future learning and teaching. To measure and summarise student achievement of learning objectives in the form of grades; To inform selection procedures for, e.g. promotion, entrance to higher education, employment.

Formality

Formal or informal (ad hoc)

Formal

Timing

Prior to / During school term

During / At end of school term

Consequences

Low-stakes

High-stakes

Perspective

Forward-looking

Backward-looking

Feedback

Specific, qualitative feedback in the form of evaluative comments on performance or   highlighted descriptors on a criteria marking scheme.  Students are provided with constructive and concrete advice on how to improve, time and opportunities for further development and practice. Scores   may be given to show students the result they would have achieved in a ‘real’ test, but the scores do not contribute to grades. Students should absorb teachers’ comments fully before seeing scores. Feedback is optional. If the students are going to do a similar test/exam in future,   then feedback could serve a formative purpose. If the test/exam content is the last time they will be tested on this, then of course no feedback, just grades. If the summative assessment is criterion-referenced, then students can refer to the   criteria marking scheme to understand what the grade implies about their   capabilities, but there is nothing they can do to change their grade.

Feedback providers

Self / Peers / Teacher

Teacher / Examiner

Understanding learning goals

Enhances student understanding of learning objectives, criteria, standards, etc. Merely checks student ability to meet learning objectives

Motivation

Positive

Negative