What different ways can we assess learners?
This fascinating study assesses a new tool - Script Concordance Test (SCT).
References
Humbert AJ, Besinger B, Miech EJ. Assessing clinical reasoning skills in scenarios of uncertainty: convergent validity for a Script Concordance Test in an emergency medicine clerkship and residency. Acad Emerg Med. 2011;18(6):627-34. .
Assessing clinical reasoning skills in scenarios of uncertainty: Convergent validity for a Script Concordance Test in an Emergency Medicine clerkship and residency
What are Scripts?
Scripts are organized networks of knowledge. Integrating them improves decision making. Using scripts, experts see associations while novices struggle with causality. In ambiguous cases, experts process multiple scripts with influx of new information.
What is the format of a Script Concordance Test?
The learners are presented with a short clinical vignette with a series of proposed diagnoses and/or plans. The learners are then presented one new piece of information and asked what effect this information has on the proposed diagnoses and/or plans. They score their decisions on a Likert scale, ranging from -2 to +2.
What did this paper study?
An observational study comparing the scores of 4th year med students (n=314) , residents (n=40) and faculty (n=12) on a SCT with scenarios in Emergency Medicine. The student score was compared to USMLE Step 2 score, and resident score with their ABEM in-training exam score.
What were the results?
The SCT scores were able to differentiate students from residents and residents from faculty.
- Students vs residents: 60% +/- 6.2 vs 70% +/- 5.4
- Residents vs faculty: 70% +/- 5.4 vs 79% +/- 2.9
There was a significant correlation between resident score and ABEM exam score and a modest correlation between student score and USMLE Step 2 score.
What were the limitations?
It is a single centre study. The internal reliability of the assessment tool was suboptimal.
What were the conclusions?
The SCT may be useful in assessing clinical reasoning in uncertain scenarios.
What do I think?
I enjoy the examples given in the paper. While it is different and likely will take some getting used to, it could be a useful assessment tool.
References
Humbert AJ, Besinger B, Miech EJ. Assessing clinical reasoning skills in scenarios of uncertainty: convergent validity for a Script Concordance Test in an emergency medicine clerkship and residency. Acad Emerg Med. 2011;18(6):627-34. .
