Test Items Should . . .

Basic Test Item Formats

Multiple Choice Questions

Levels of Learning That Can Be Tested:

Provided by John Boker, PhD.

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

In General, Test Items Should…….

• Assess achievement of instructional objectives
• Measure important aspects of the subject (concepts and conceptual relations)
• Accurately reflect the emphasis placed on important aspects of instruction
• Measure an appropriate level of student knowledge
• Vary in levels of difficulty

Basic test item formats:

Provided by John Boker, PhD.

• Multiple-choice
• Matching
• True-False
• Completion/Short-answer
• Essay
• Demonstration

                                                                                      top of page

Multiple Choice Questions

Case, S. and Swanson, D. (1998). Constructing Written Test Questions for the Basic and Clinical Sciences. (2^nd ed.). Philadelphia:NBME.

Testing Students’ Problem Solving Ability

This is a summary guide on writing multiple choice questions that assess students’ abilities to apply basic science facts, concepts, relationships to clinical problems, etc. The guide was developed from the new edition of NBME’s test writing manual Since the Boards are increasingly using questions with clinical scenarios and since many Course Directors are introducing case-based activities into their courses to promote students’ ability to use basic science information in clinical problem-solving, use of this guide by all faculty who write test questions is encouraged.

• Tests should mirror the explicit objectives of a course. If your teaching is aimed at having students just memorize the facts, definitions, and relationships of a discipline, then test items should evaluate their ability to recall the correct fact, definition, or relationship. If your objective is to promote students’ abilities to correctly reason using these facts, definitions and relationships, then test items must evaluate their ability to apply facts and principles in solving a problem. (Note: The NBME I and II are increasingly using questions that test students’ abilities to correctly apply basic science facts, concepts, and principles to answer questions about clinical problems. If we want students to continue to do well on the Boards, it may be wise to give students opportunities to practice applying facts and to use these kinds of questions on locally prepared tests.)

• Application Question Structure

All multiple choice questions consist of a stem and a set of answer options. What differentiates the "recall" question from the "application" question is the contents of the stem. Application questions consist of two parts: (1) data and (2) the task which express the problem the student is think about. The data may take the form of a lab experiment vignette or patient data unfamiliar to the student. (If the student is familiar with the problem, the task will likely be one of recall, not reasoning.) The task asks them to use the data to come to a correct answer that is contained in the answer set. Thus, a general application question template is:

Lab Experiment or Patient Data + Application Task = Correct answer from a list of plausible answers

Template Examples

Data: A (patient description) has a (type of injury). Task: Which of the following anatomical structures is most likely to be affected?

Data: A (patient description) has (history findings) and is taking (medications). Task: Which of the following medications is most likely to be the cause of his (history or PE or lab findings)?

Data: A (patient description) has (abnormal findings). Task: Which additional finding would suggest a diagnosis of (disease 1) rather than (disease 2)?

Data: A (patient description) has (symptoms and signs). These observations suggest that the disease is a result of the (absence or presence) of which of the following (enzymes, mechanisms)?

Data: A (patient description) follows a (specific dietary regime). Which of the following conditions is most likely to occur?

Data: A (patient description) has (signs, symptoms, a specific disease) and is being treated with (drug or drug class). The drug acts by inhibiting which of the following (functions, processes)?

Data: A (patient description) has (abnormal findings). Which of the following (positive laboratory results) would be expected?

Data: (Time period) after a (event such as a trip or meal with certain foods), a (patient or group description) became ill with (symptoms and signs). Which of the following (organisms, agents) is most likely to be found on analysis of (food)?

Data: Following (procedure), a (patient description) develops (signs and symptoms). Lab findings show (findings). Which of the following is most likely to be the cause?

Data: A (patient description) dies of (disease). Which of the following is the most likely to be the finding on autopsy?

Data: A patient has (signs and symptoms). Which of the following is the most likely explanation for the (findings)?

Data: A (patient description) has (signs and symptoms). Exposure to which of the (toxic agents) is the most likely cause?

Data: Which of the following is the most likely mechanism of the therapeutic effect of this (drug class) in patients with (disease)?

To decrease the likelihood that students can guess the correct answer, make sure that your answer options are:

1. Homogeneous in content (e.g., all are diagnoses, all are next steps in patient care).
2. Incorrect or inferior to the correct answer.
3. Plausible and attractive to the less well informed.
4. Similar to the correct answer in construction and length.
5. Grammatically consistent and logically compatible with the stem.

                                                                                         top of page