Developing multiple choice questions that test higher order thinking
Multiple-choice questions are the most commonly used objective assessment question type which means that it is a question type with a pre-determined single correct answer. While the usefulness of multiple choice questions in teaching and learning is undisputed, there is a common misconception that multiple-choice questions can only test the lower levels of Bloom’s Taxonomy such as remembering and understanding. It may be true that multiple choice tests should not be the sole assessment method in any module and that multiple-choice questions are not sufficient to test certain skills (such as writing and communication skills for instance), however, it is certainly not true that multiple-choice questions cannot be used to test higher-order thinking. It is important that you first ensure that you set up a good question before you focus on testing higher-order thinking.
In this section, four strategies for creating multiple-choice questions that test higher order thinking skills (at least at an application level of Bloom’s Taxonomy) is provided. Each strategy is explained with an example. These strategies are not discipline-specific.
Strategy 1: Two-tier multiple-choice question
In a two-tier multiple-choice question, the first tier pertains to a knowledge or comprehension statement and the second tier facilitates testing of higher-order thinking. It is in effect asking a lower-order question and then asking students to provide a reason (which requires higher-order thinking). Two-tier multiple-choice questions can also be seen as scaffolded questions.
Example:
Suppose you are given two clay balls of equal size and shape. The two clay balls also weigh the same. One ball is flattened into a pancake shaped piece. Which of these statements is correct?
- The pancake shaped piece weighs more than the ball
- The two pieces still weigh the same
- The ball weighs more than the pancake shaped piece
Give a reason for your answer:
a) the flattened piece covers a larger area.
b) the ball pushes down more on one spot.
c) when something is flattened it loses weight.
d) clay has not been added or taken away.
e) when something is flattened it gains weight.
f) a and c
g) a and e
Strategy 2: Use real-world scenarios/ case studies
One of the best ways to promote and assess higher-order thinking is to use scenario-based questions. This changes a question from requiring simple recall to, at the very least, requiring the application of knowledge. This strategy particularly, can be used in most disciplines. You can also ask more than one multiple-choice question based on a single scenario/ case study.
Example:
“Because of rapidly rising national defence expenditures, it is anticipated that Country A will experience a price inflation unless measures are taken to restrict the growth of aggregate private demand. Specifically, the government is considering either (1) increasing personal income tax rates or (2) introducing a very tight monetary policy.” If the government of Country A wishes to minimize the adverse effect of its anti-inflationary policies on economic growth, which one of the following policies should it use?
a. The tight money policy because it restricts consumption expenditures more than investment.
b. The tight money policy, since the tax increase would restrict consumption expenditures.
c. The personal income tax increase since it restricts consumption expenditures more than investment.
d. Either the tight money policy or the personal income tax rate increase since both depress investment equally.
Strategy 3: Analysis of visuals
Visuals as part of the question (stem) of a multiple-choice question, such as diagrams or graphs require higher-order thinking to solve. Interpreting a graph, for instance, in most instances require analysis as the student needs to understand the ‘whole’ as well as how individual parts function within the whole. Although graphs and diagrams may not form part of all disciplines, other visuals can also be used as part of a question to test higher-order thinking. For example, cartoons that depict a situation or certain behaviour can be used.
Example:
Without other data, which conclusion can you make from reviewing Figure 17?
a. The average American uses more drugs than citizens of any country except the United Kingdom
b. The average Mexican or Chilean consumes fewer drugs than citizens from other countries.
c. Americans are more likely than residents of other countries to use new drugs.
d. Japanese have regulations that make it very difficult to obtain new drugs
Strategy 4: Multi-logical thinking
A good strategy for testing higher-order thinking, is to develop questions that require multi-logical thinking. These are questions with more than one premise, from which a student must make the correct conclusion. It thus requires a student to take several facts and aspects in account before making a decision. In the example given, the student is required to both diagnose the symptoms and decide on what the best way would be to break the news to a parent.
Example:
Tim’s second grade teacher is concerned because of the following observations about Tim’s behaviour in class:
- Withdraws from peers on the playground and during group work
- Often confuses syllables in words (ex: says mazagine instead of magazine)
- Often confuses b and d, p and q, etc. when writing or recognizing letters
- Loses his place when reading
The teacher has arranged a meeting with Tim’s mother to discuss these concerns. Which of the following statements is best for the teacher to say to Tim’s mother?
A. Tim needs extra practice reading and writing problematic letters and words at home at least 30 minutes per day.
B. Please discuss the importance of schoolwork to Tim so that he will increase his efforts in classwork.
C. These are possible symptoms of dyslexia so I would like to refer him to a specialist for diagnosis.
D. Please adjust Tim’s diet because he is most likely showing symptoms of ADHD due to food allergies.
