Multiple choice

For many instructors, multiple choice questions are a popular option. They are easy to grade, and can often be easily aggregated for data analysis. This assumes that the questions are valid and reliable, which cannot be assumed if they are generated by individual instructors and used solely within their own classrooms. It simply isn’t something an instructor can guarantee, but one can work on writing questions that are of a high enough quality and that target an important component of the learning process.

A good practice before writing any multiple choice questions is to identify the concepts that you wish your students to demonstrate on the quiz or exam. Try to limit each question to a single concept. To give an example from a mathematics course, many teachers will include questions about solving equations of one variable that include fractions. When they evaluate the question, can they be convinced that the student had difficulty with solving the equation itself, or was it a lack of familiarity with the relationships expressed by fractions? If two concepts cannot be isolated, efforts should be made to vary the concepts that are being combined so that the overall picture can suggest meaningful information about how the student has mastered the material.

Once you know which concepts you want to test, you should focus on writing several questions for each concept. The more questions a student can answer on the same topic, the more confident you can be that their incorrect answers are the result of a lack of understanding and not a random mistake. There is no perfect amount of questions, but replication should provide more validity to your results. When constructing exams, it is best practice for the number of questions about particular concept to be in proportion to the importance of that concept (often based on the amount of class time devoted to it) in the course. In other words, if you spend a week on topic A but only two days on topic B, you should ask more questions about topic A than topic B.

A multiple choice question consists of a stem (the question itself) and the leaves (choices). One leaf will be the correct answer, and the others are known as distractors. It is important to make sure that each distractor is connected to a common mistake (i.e. a reason or misconception) made by students. Many educators aren’t sure what kinds of answers students will come up with, and so the alternate answers to their multiple choice items are simply randomly generated. This practice allows students to game the system without demonstrating understanding. If you don’t feel confident that you have fully understood how students are confused by a particular concept, a multiple choice item is probably not the best way for the concept to be addressed. Instead, use short answer questions to help you start to analyze common misconceptions among your students.

A poor example of a multiple choice item regarding conditional execution.

A typical computer science question asked in multiple choice format.

When composing multiple choice questions, here are a few good rules of thumb to follow to make sure that your questions are well-constructed:

    • Ask questions that get at one main idea or concept, and provide only one correct or “best” answer.
    • If the stem is not a complete sentence, put as much of the question as you can in the stem. Do not repeat words in the leaves that could be in the stem.
    • Use negatives sparingly, and if you do use a negative in the stem highlight it (underline or bold) to make sure students notice it. Do not use double negatives, i.e. a negative in the stem and in one or more of the answers.
    • When you can, keep alternate responses about the same length, and be sure that all responses are grammatically consistent with the stem.
    • Use consistent formatting when writing questions, i.e. indent answers the same amount, start each alternative on a new line, etc.

The ultimate takeaway of this section should be that multiple choice should not be used lightly, in spite of the advantages gained from grading ease. Assessment needs to be a thoughtful process, and without the ends to the means the amount of information gained will be limited.