Article review: Pitfalls in writing test questions

Which is the best answer?

• A. Yes
• B. No
• C. Maybe
• D. 2 of the 3 above
• E. None of the above

Wait, what?! 

What a terribly written test question! 

Have you encountered similarly poor questions on exams? It turns out that writing multiple-choice test questions is actually pretty difficult. There are some basic rules to follow and pitfalls to avoid.

In an article, the authors (hey, I know most of them! Wait, why wasn't I invited?!) talks about the lack of a National Board Medical Exam in Emergency Medicine. Such "shelf exams" exist in other specialties but in EM. Frankly, it has to do with how expensive it is for medical schools and clerkships to purchase these tests. Within EM, 59% of clerkships are using an end-of-clerkship exam, most of which are designed by the local institution.

The authors also provide an excellent review on the art of writing multiple-choice test questions.

So what are the basics in writing a good multiple-choice test question?

There are 2 parts to each test item:

1. The stem: The question itself
2. The answer choices: Keyed response (correct answer) vs Foils/Distractors (wrong answers)

The Stem

• The test question should be clear and answerable without looking at the possible choices.
• The test question should have only one undisputable answer.
• Avoid being too wordy. State the question concisely.
• Avoid "negative" questions (eg. Which of the following is NOT a cause for...)

The Answer Choices

• There are smart test-takers out there. For instance, choices which have the word "always" or "never" are usually foils and thus incorrect answers. Grammatically incorrect choices are usually wrong. When choosing between 2 answers, the really long one is often the right choice, because the test-writer wants clarify and ensure that the answer is correct. Be aware of these when writing the keyed response and foils.
• When listing the choices, put the responses in logical order (alphabetical or numerical).

The authors also discuss the importance of determine test reliability and content validity. Are the students performing poorly because you just suck at writing test questions? Perhaps a better solution than having all the clerkships working in isolated silos is to have a single validated exam.

Thus, the authors conclude the need for a standardized, national EM final exam, now that a formal EM curriculum has been created by CDEM this past year.

Reference
Senecal E, Askew K, Gorney B, Beeson M, Manthey D. Anatomy of a Clerkship Test. Acad Emerg Med, 2010, 17: S31-37. DOI: 10.1111/j.1553-2712.2010.00880.x
.