During medical school and residency, standardized patients, simulation cases, observed structured clinical exams (OSCE), direct observation serve as evaluation tool to assess learner competencies and skills. The inherent problem is the observer effect. This means that people change their behavior because they know that they are being observed.
The authors of this study introduce a novel approach of using unannounced standardized patients who present for "fake" medical reasons. Although this could potentially take the resident away from treating other patients, this exercise seemingly best reflects the resident's true actions and interactions with patients.
The study aimed to assess resident communication and professionalism skills. Fifteen PGY-2 EM residents were only told that they would be seeing standardized patients sometime in their next 4-6 weeks in the Urgent Care.
Because of scheduling difficulties, only 17 of the 27 planned standardized patients presented themselves to the resident. The standardized patients, who were paid actors, were trained to portrait two cases. The first was an angry patient who had a film misread and called back. The second was a dissatisfied, "frequent-flier" patient with chronic pain. The actors also received training on how to evaluate the residents on the following items:
Communication
- Information gathering
- Relationship development
- Education and counseling
- Accountability
- Manage difficult situation
- Giving bad news
- Treatment plan and management
- Patient centeredness
- Despite the small sample, the data showed a range of scores for communication and professionalism.
What did the results show?
The residents detected only 44% of the unannounced standardized patients. Furthermore, most residents did not feel uncomfortable or suspicious of patients. I personally might feel a little suspicious, but then I'm always a little paranoid.
The mean resident score for communication items with a rating of "well done" was 60%.- The mean resident score for professionalism items with a rating of "well done" was 53%.
- Areas of weakness: Patient education and counseling (43%), information gathering (68%), and relationship development (62%).
On a bigger picture level, this methodology is directly in line with the ACGME's Phase 3 push for more assessment tools which focus on patient outcomes and external measures. Unannounced standardized patients provides educators another tool in their assessment "toolbox".
Reference:
Zabar S, Ark T, Gillespie C, Hsieh A, et al. Can Unannounced Standardized Patients Assess Professionalism and Communication Skills in the Emergency Department? Acad Emerg Med. 2009 Aug 10; 16:1-4.