Previous Next

Part 3: Pedagogical and Evaluation Tools

Readings for Reflection

Ethics and performance evaluation

Few authors address the ethics of performance evaluation but it is important to keep in mind that ethical rules do apply.

With respect to ethics, here is what Villeneuve (1998) proposes:

  1. Only provide feedback if it is necessary.
  2. Evaluate performance based on pertinent, sufficient, and representative information.
  3. Be honest when evaluating the obtained facts. Avoid speculating on probabilities.
  4. Keep verbally expressed comments consistent with written comments.
  5. Recognize that the evaluation reflects your personal opinion and the way you perceive things.
  6. Only share information relating to someone’s performance evaluation with individuals that have a right to know.
  7. Only accept an assessment made by someone else if it is constructive, related to the objectives, and honest.
  8. Never talk about a student’s performance to others (i.e. clinical coordinator) before feedback has been given to the student.
The “how to” of formative evaluation

Considerations

 Implementation Examples

Supervisor’s preparation
  1. Summarize what you wish to say.
  1. Specify the reasons for your feedback to avoid an impulsive or negative reaction.
  1. Establish whether the feedback is truly necessary to reach the objectives.
  1. Identify the action worth noting and improving, and compare it to the performance criteria.
  1. Separate the student’s behaviour from personal objectives or pride.
  1. Set realistic expectations that relate to the student’s level of study (1st, 2nd year, etc.).
  1. What do I wish to say and why?
  1. What is my state of mind? How do I feel?
  1. Will my feedback help the student achieve the objective? “He must master … to complete the technique.”
  1. What is the exact knowledge base, skill or attitude that I wish to focus on or see improved?
  1. Do I feel like the student’s behaviour makes me look bad?
  1. Will my comment lead to a solution?

Principles of evaluation
  1. Is based on facts and multiple sources of information.

Feedback should be based on certainties, concrete examples (gesture, actual or observed facts, words, body position, etc.).

Collect your information from varied and trusted sources (direct observation, verified comments from colleagues or patients, documents written by the student).

  1. Is focused on behaviour, not on the person.

Avoid passing judgement; focus your comment on the behaviour that requires improvement.

  1. Is a legitimate comment made respectfully and ethically.

Relate your feedback to the placement objectives, the standards of practice and the institution’s expectations.

  1. Is based on improvement and requires a follow up.

State a precise comment that leads to a solution.

  1. Is a two way process.

Encourage feedback instead of imposing it. (Ex.: Invite the student to ask questions on his performance, his progress).

Invite the student to express his perception of the situation, to suggest solutions and a plan to improve.

Ask open-ended and general questions that encourage the student to think.

Share information with the student instead of providing him with a list of advice.

Decide on common action plan.

  1. Is confidential.

Choose a discrete place where colleagues and patients will not overhear and where you will not be interrupted.

Discuss face to face. The telephone or e-mail is not the preferred means of providing feedback.
  1. “I saw you…” or “I read your chart and…” “I received comments from my colleagues and patients on your use of professional jargon…”
  1. Say “I noticed you interrupted the patient, on three occasions today to give your opinion while he was trying to explain something to you.” instead of “You are too controlling with the patients.”
  1. “You must be able to work with this type of patient before the end of your placement. If you feel you need more practical experience, would you like me to find you opportunities?”
  1. “Here are a few things you can do to improve the way you greet your patients…”
  1. “What do you suggest to help you improve?” or “Would you like me to help you find some opportunities for you to practice, if this is what you’re missing?”
    “I suggest that you reflect on what you’ve learned and on the areas you feel you need to improve. We can discuss it at the end of each day.” Discuss verbally and document discussion in a log.
    “What will happen if you do this intervention or procedure?”
    “I’d like to discuss with you the meeting that you had yesterday with Mr. X. Here’s what I noticed…What do you think of this result? How does this relate to your placement objectives?”
  1. “Come with me so we can find a quiet place to speak”
    “I understand that you have many questions about this patient, but we are in a high traffic area (public, where many can overhear us). Come, we’ll find a more appropriate place to continue this discussion.

Considerations

 Implementation Examples

Feedback process: How to proceed…
  1. Meet the student as soon as possible after the event.

Choose an appropriate time when the student is receptive and when you are both available (intellectually and emotionally).

  1. Find a calm, quiet place.
  1. Focus on what recently happened – avoid references to the past or speculations on the future (to reduce anxiety and confusion)
  1. Give a specific comment.

Start by mentioning the topic of your feedback.

Talk in terms of “I”.

Choose the proper words, understandable (avoid jargon), use specific words to describe the action. Avoid using generalities (always, never).

  1. Limit the amount of information.

Limit your feedback to the essentials (to reduce confusion).

  1. Express yourself by different means.

When a student doubts or denies the need to improve a point that you brought up, find another way to reach him.
  1. If the situation does not allow it: “I wish to go over the experience you just had, however, we are short on time. Why don’t you sleep on it and we can discuss this first thing tomorrow.”

“The question you asked me is complicated. I’d prefer to look at the case with you tomorrow, please bring the chart.”

  1. “Is it a good time to discuss Mrs. X?”
  1. “To intervene properly with this types of patient, you must review the process…” instead of “If you don’t master this intervention, you will have difficulty with the others.”
  1. “I want to discuss your attendance.”
    “I’d like to speak with you about your progress notes.”

“I observed your approach…”
The right words: “You had to try twice to complete the technique. Are your clothes restricting your movements?” instead of “Your clothes are too tight and you cannot move properly.”

“Your progress notes could be more concise” instead of “You always write too much.”

  1. “I’d like to discuss the approach that you used.” Instead of “… approach X, your time management, your clothing, your hair, your knowledge…”
  1. “Come, I’ll place my hands over yours to guide you.”

Considerations

 Implementation Examples

Feedback process: How to proceed …

(cont’d)
  1. Ask open-ended questions.

Let the student express himself and share his ideas; afterwards, share yours.

  1. Avoid a flattering or accusatory tone, including qualities (such as good, nice, likeable, best) and avoid placing blame or picking sides (you must, he should…).
  1. Avoid winner-loser situations.

Keep the student’s perspective in mind.
  1. “How did you feel when the patient began to cry? Do you have any suggestions?”
  1. “The clients appreciate your humour and your attention.”
    Say “I noticed that you have already mastered the development of a professional rapport with you patients.” instead of: “You are nice, you work very well and the team really likes you.”
  1. When opinions differ: “As you can see, there are many ways of doing this. Here, we use method A for these reasons…. What do you think of presenting the method you propose to the staff?” or “I understand your point of view. Instead of what I’m suggesting, would this way suit you better?”

Considerations

 Implementation Examples

Feedback process: How to say it…
  1. Confirm the student’s strong points.

Highlight what has been achieved.

  1. Constructively identify the behaviours that require improvement.
  1. Identify the possible obstacles to achieving the objectives.
  1. Encourage the student to adopt other behaviours or to take alternate actions.

Make a link between the purpose of the feedback and the solution.

  1. Help the student to motivate himself for change.

Identify the advantages of a change in behaviour.

  1. Make the change easy

Help the student apply the suggested change.

Work on a plan to improve.

Intervene in a concrete manner (Ex.: Demonstrate).

  1. Show commitment and interest

Discuss openly.
  1. “I appreciate your diligence; You arrive on time, your work is ready…”
  1. “I would like to review the way you organize your day.”
  1. “I see that you are trying to overcome your fear.” or “It is difficult when…”
  1. “After observing you, I propose that you try…”

“What do you think about beginning the preparations earlier so that your patients are ready on time for their appointments in rehabilitation? or You could try…”

  1. “I noticed that you have been leaving after hours every night this week. Let us look at ways to help you leave on time.”
  1. “Review the steps to follow and I will try to find opportunities for you to practice.” or “Before we go in the room, I’ll review the steps with you.” or “I ask that you read this for tomorrow. It is a complete review of the procedure and it will be help you react in a logical sequence when you are faced with situation like this again.”
  1. “Do you have any ideas on how you could practice this technique more often?” “

“How can I help you?” or “Let me demonstrate the technique and identify the points that should be improved.”

Considerations

 Implementation Examples

Feedback process: How to say it…

(cont’d)
  1. Clarify the student’s perceptions with regards to his performance.

Allow a time for thought.

Invite the student to inquire on his performance.

React with calm and control.

  1. Follow up.

Note changes and point them out to the student.

Advise the student of gaps in performance and confront him if the gaps persist after discussion.

In a case where there is no change in the situation, bring up the point again, verify that the student understands and applies the necessary corrective measures.
  1. “What would the results be if you proceeded differently?” or “What do you think? …Do you have a solution to propose? or “I’d like you to reflect on your attitude this afternoon; I recommend that you read over your notes on professional conduct and the code of ethics. We’ll discuss this again tomorrow morning.”
    “I see that you do not grasp the importance of my concern.”
  1. “Last week, we spoke about…, since then, on two occasions, I’ve noticed that…”

Specific cases

Note: These specific cases can elicit strong emotions or reactions. It is important to be aware of them so that you can react calmly and with control.

  1. When dealing with a student who masters the theory and expresses himself well, accept it and focus on what must be improved; do not allow yourself to be intimidated.
  1. In the presence of a shy student with perhaps a low self-esteem, be precise and suggest an action plan.
  1. “On one hand, I see that you understand the material and on the other hand, I see that your intervention did not lead to the expected result. What do you think about this?”
  1. “I don’t see that you understand what needs to be done here. I suggest that you write down the steps to follow for this procedure and we’ll discuss it tomorrow morning. However, for your own safety and for your patient’s safety, please abstain from doing the procedure until I say otherwise.”

Considerations

 Implementation Examples

Specific cases

(cont’d)
  1. When dealing with a student from a different culture, keep in mind that you may have different styles of communication. However, avoid trying to explain everything as a cultural difference.

Verify whether, in the student’s culture, it is improper to admit a weakness. Observe the body language (signs of frustration, disagreement, etc.).

Read up on the influence of culture on sickness and health care relationships.

  1. When dealing with an angry student,

Listen without interrupting. Be aware of your body language (i.e. arms crossed indicates closure).

While keeping your calm, acknowledge the emotion and summarize the student’s remarks.

  1. When dealing with an excuse maker (may indicate insecurity), ask him not to interrupt you while you point out the behaviour that needs improvement.
  1. When dealing with a negative student remember that negativism is a means to control a situation. It is often an indicator of a low self-esteem.

Clearly voice your thoughts and feelings.

Formulate your feedback delicately and leave out formalities when you begin the discussion.
  1. “I am unable to determine whether you have any difficulties with the intervention that we discussed. Can you tell me in your words, what you need to do to provide this care?”
  1. “I see that you are frustrated. If I understand what you are telling me, you don’t like it when I follow up on you. It gives you the impression that you are being watched all the time. Is this correct?”
  1. “I want to talk to you about something and I do not want you to reply immediately. I want you to think about it and we can discuss again a little later.”
  1. “It is normal to make a mistake or to forget. I expect you to learn from your mistakes but I don’t expect you to be perfect.”
    Another point of view: “Learning implies that we can identify our mistakes.”
Providing feedback: how to say it…

  1. Confirm the student’s strong points
  2. Constructively identify the behaviours that require improvement
  3. Identify the possible obstacles to achieving the objectives
  4. Encourage the student to adopt other behaviours or to take alternate action
  5. Help the student to motivate himself for change
  6. Facilitate the change
  7. Show commitment and interest
  8. Clarify the student’s perceptions with regards to his performance
  9. Follow up

Adapted from Dolan & Schuler, 1995

The Most Common Errors

Even with the best intentions, performance evaluation is a subjective process. Judgement errors are possible when one person evaluates another, and this can skew the evaluation process. Turgeon (1997) states that some biases are caused by the supervisor’s attitude about for example, the profession or the placement process and even by his personality. These biases may be caused by the supervisor’s inability to put aside his feelings while evaluating a student.

Here is a summary of the most common errors done by supervisors during a performance evaluation. (Adapted from Dolan & Schuler, 1995, pg. 344-345; Turgeon, 1997, pg. 178).

  1. The halo effect is without a doubt, the most common error made, especially when the evaluation involves many aspects of professional practice. In this case, the supervisor has a tendency to pass an overall judgement on the student’s performance based on one or two events that he judges to be more significant. He then carries them over throughout the evaluation process.
  2. The error of leniency is the second most frequent error and it is often done intentionally. It is also referred to as an error of extremes and results from too much clemency by a supervisor. This error is made when the performance is overstated in order to avoid any potential conflict or to appear to be a “nice” supervisor.
  3. The error of harshness consists of systematically giving a poor evaluation, regardless of the observed performance (the opposite of the error of leniency). This is also referred to as an error of extremes since the supervisor is too strict. This error generally occurs with inexperienced supervisors, individuals that have low self-esteem, recently appointed clinical supervisors who want to impress their superiors by their sternness or with individuals who use performance evaluation to “get even with others”.
  4. The central tendency error occurs when supervisors wish to avoid risks and therefore average out their evaluations, even when the performance level varies. This type of error can occur when a supervisor does not directly or regularly follow the student and ends up placing his performance ratings in the average.
  5. The effect of the first or last impression occurs when the supervisor does not remember what happened during the course of the placement (he never took notes). The supervisor then judges the student based on information gathered at the beginning of the placement and categorizes the student as good or bad. As a result, the supervisor seeks out information to support his position and ignores information that contradicts his initial impression. Alternately, the supervisor may use the most recent information to base his evaluation on. It is a double edge sword since it can cause the supervisor to be too harsh or too lenient.
  6. The contrasting effect results when a supervisor compares the student’s performance to his own standards. He judges the student’s performance based on the gap between how the student did and how he, himself, would have done in the same situation. It is also a contrasting effect when the supervisor compares his current student with a previous student. For example, comparing the student with a weaker student may give the impression that an average student is excellent and vice-versa.
  7. The carryover effect happens when there is an unfair influence of past evaluations (good or bad) on the current evaluation. A new supervisor who has heard a previous supervisor speak of a student may make this type of error.
  8. The error of similarities refers to the tendency of some evaluators to exaggerate the performance of an individual with whom they get along well.
  9. Stereotypes are negative or positive attitudes of an individual or group of individuals (discrimination based on gender, race, religion, city, university, etc.). The supervisor’s preconceived opinion or stereotypes will skew the evaluation process.
How to Reduce the Consequences of Errors

1. Improve one’s knowledge on the evaluation process and available tools.

When the supervisor understands the evaluation process and the tools available (evaluation grid, journal, observation grid, role play, etc.), he is more sensitive to the risk of making a judgement error.

2. Clearly define the components to evaluate (indicators, competencies).

It is important to identify which parameters are to be measured. A judgement error occurs when the supervisor fails to measure the performance against the set objectives and expectations. According to Bernard and Goodyear (1998), a poorly developed and poorly implemented evaluation process can negatively impact the student’s performance instead of improving it. It is therefore necessary to take particular care in providing detailed feedback at every step of the evaluation process.

3. Establish periods of direct observation of the student’s performance.

Direct observation is a primary source of information. It helps to validate or confirm the progress and observations obtained from other sources and it provides a chance to witness important events. The supervisor who does not include direct observations to his sources of information will not be able to provide a fair and complete assessment of the student’s performance.

4. Keep a journal of your observations

Memories are flawed and this can lead to inaccuracies when trying to remember facts. With regards to the student’s performance, a journal or a notepad serves as an excellent tool for recording facts, behaviours, incidents, dates and time of significant events.

5. Avoid giving an “average” grade (the midpoint or middle).

An average grade is subject to interpretation. Instead, it would be better to use different methods of communication, such as:

  • Meets, partially meets or does not meet the objectives (areas of strength and areas that need improvement)

  • Exceeds expectations

  • Does not demonstrate expected competencies on a regular basis without supervision or assistance.

Next page:

Video Clip: The Value of Pedagogical and Evaluation Tools