Objective vs Subjective question types - Blended Learning Resources

Objective vs Subjective question types
Before you start using online assessment, it is important to consider what you want to achieve with the assessment. This will help you choose the tools and methods that will help you to achieve your assessment goals. A common misconception about online assessment is that your test will be automatically graded. Reducing marking time is also the most common incentive for starting to use online assessment. The fact is that only some question types are marked automatically by an online assessment platform such as an LMS. This is where objective vs subjective questioning plays an important role. The difference between these two question types are:

Objective question types require students to select a response from a list of alternatives provided to them or to supply a word or short phrase. Only one correct answer is identified. Subjective question types allows a student to organise an original answer, using their own words. These answers are usually longer (such as a paragraph or a couple of sentences).

Therefore, if the purpose for using online assessment is to reduce marking time and to have an online assessment program mark for you, you need to make use of objective question types. Subjective question types are not marked automatically by any platform. You will always need to mark subjective question types yourself, usually the marking can be alleviated by creating a rubric.

Objective question types
The most well-known and commonly used objective question type, is a multiple choice question which is a question with a number of options of which a student must select the correct or most correct option. There are, however, typically many other objective question types available to use on an online assessment platform such as Respondus or an LMS, which allow you to use a wide spectrum of different types of questioning in your test. In addition, online assessment provides several options for multiple choice questions that are not necessarily practical or feasible in a written format. All of the objective question types shown in this section will be automatically marked.

Multiple choice questions (MCQ)
A simple example of a multiple choice question is the following:

You can do more with a MCQ online, however. For instance, consider using an image as part of the question, or a short video or audio clip (see examples here). Multiple choice questions can also be developed to test higher-order thinking.

Multiple response questions
A multiple response question is similar to a multiple choice question but instead of having to choose one option from a list, a student is required to choose more than one of the options.

A simple example of a multiple response question is the following:

As with a MCQ, you can incorporate images, videos or audio clips as part of your multiple response questions.

Matching questions
Matching questions are similar to match the column questions sometimes also used in written assessments.

A simple example of a matching question is the following:

Match the capital cities in the column on the left, to the countries in the column on the right:

A matching question can be more difficult if there are some extra options in one column that do not have a match in the other column, such as in the following example:

Match the capital cities in the column on the left, to the countries in the column on the right:

In addition, matching questions do not only have to be questions with single words in each column. Instead, a single word or phrase can be used in one column to describe an explanation or description in another column. Another option is using images in one column and short descriptions in the other column. Matching questions are often used to test not only knowledge recall but also application of knowledge. Here are some examples of different types of matching questions. It is important to note that the assessment program (both Blackboard and Questionmark) automatically shuffles one of the two columns so that each students who completes an assessment sees a different order.

Ranking question
A ranking question requires a student to rank steps of a process or order of events in the correct sequence. An example of a simple ranking question is:

Select the correct order of the following significant events in world history:

Instead of one or two words, a short phrase or paragraph can also be used in the left column of a ranking question (to describe a step in a process for instance) in order to make the question more complex. It is important to note that the assessment program (both Blackboard and Questionmark) automatically shuffles the column on the left so that each students who completes an assessment sees a different order.

Select a blank question
A select a blank question is a longer question, usually in a paragraph format, with multiple answers. This question type is not commonly available on all online assessment platforms, but it is available on Questionmark, for example.

An example of a simple select a blank question is:

Human Resources Management consists of various processes within an organisation. The process aimed at attracting applicants that match job criteria is known as [Recruitment/ Selection/ Hiring/ Remuneration Management/ Training and Development/ Performance Management/ Employee relations]. The aim of [Recruitment/ Selection/ Performance Management/ Remuneration Management/ Employee Relations] is to evaluate performance in order to reward good performance and address poor performance. [Recruitment/ Selection/ Hiring/ Remuneration Management/ Training and Development/ Performance Management/ Employee relations] is the process that involves determining salaries, incentives, and fringe benefits.

With this question students are provided with a list of possible answers for each ‘blank space’ in the question. This is an alternative to the better-known fill in the blank question. The advantage of this question, however, is that grading the correct answer is not influenced by incorrect spelling or synonyms of the correct answer. With a fill in the blank question students can easily make a spelling or typing error and will then not receive a mark as the system only recognises key words as identified by the lecturer as correct answers. Students also sometimes use synonyms for the identified key words, which is also not recognised by the system. The select a blank question eliminates these potential issues when a list of potential options is provided for each blank space in the question.

Hotspot question
A hotspot question allows a student to click somewhere on an image to indicate an answer. The system automatically gives a mark if the student clicks within a predetermined area but receives a zero if he/she clicks outside of this area.

An example of a hotspot question is:

Refer to the image of a heart below. Click on the aorta:

Students who clicks anywhere in the green marked area will receive a mark and students who click outside of the area will receive zero. Students do not see the green marked area as part of the question, the image in the example above is to illustrate how the question is set up on the system (Blackboard/ Questionmark). As the example illustrates, this question type works well in modules where students need to visually be able to distinguish between certain areas such as in Physiology or Biology. However, it can also be used creatively in other types of modules as in these examples. It is important to note that this question type should be used with caution if you have students in your class who are visually impaired as the software on their computers will typically not read an image.

Drag and drop question
Drag and drop questions allow a student to drag and drop a number of labels to appropriate spaces on an image. A student can either get a single mark for placing all the labels at the correct spaces on the image or can get a mark for each label placed correctly.

A simple example of a drag and drop question is:

Drag and drop the labels at the bottom of the image to the corresponding countries on the image. Place the label anywhere on its corresponding country:

[h5p id=”2″]

Students do not see the colour-coded marked areas as part of the question, the image and labels in the example above is to illustrate how the question is set up on the system. Drag and drop questions also work well for an image of text such as matching a step in a process with a description. It is important to note that the labels at the bottom of a drag and drop questions should be short (preferably one or two words) as it must fit in next to each other at the bottom of the image. This question type should be used with caution if you have students in your class who are visually impaired as the software on their computers typically cannot read an image. This question type is not commonly available on all online assessment platforms but it is available on Questionmark.

Developing good multiple choice questions
It is important that a multiple-choice question is constructed well, before focusing on constructing questions that test higher order thinking. A multiple-choice question consists of a question (sometimes also referred to as the stem) and a number of options, of which only one is correct. Both the question and the options need to be constructed well to ensure the quality of a multiple-choice question. Since multiple-choice questions are often re-used, it is worth setting up good quality questions from the start. These questions can often be adapted to build a large question bank over time. The following guidelines are useful to consider in setting up multiple-choice questions.

Guideline	Example of a poor question	Suggestion for improving the question
Avoid unnecessary and irrelavant material in the question to avoid confusion and misdirecting the student.	Paul Muldoon, an Irish post-modern poet who uses experimental and playful language, uses which poetic genre in “Why Brownlee Left”? a. Sonnet b. Elegy c. Narrative poem d. Dramatic monologue e. Haiku	Paul Muldoon uses which poetic genre in “Why Brownlee Left”? a. Sonnet b. Elegy c. Narrative poem d. Dramatic monologue e. Haiku
Use clear and straightforward language in the question. Avoid making the question a reading comprehension rather than an assessment of the subject matter.	As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation? a. A decrease in the labour force participation rate of women b. A dispersing effect on population concentration c. A downward trend in the youth dependency ratio d. A broader base in the population pyramid e. An increased infant mortality rate	A major decline in fertility in a developing nation is likely to produce which of the following? a. A decrease in the labour forces participation rate of women b. A dispersing effect on population concentration c. A downward trend in the youth dependency ratio d. A broader base in the population pyramid e. An increased infant mortality rate
Use negatives sparingly. If negatives must be used (as you may sometimes want to test knowledge about exceptions to a rule), capitalize, underscore, embolden or highlight it.	Weak question: Which of the following is not a symptom of osteoporosis? a. Decreased bone density b. Frequent bone fractures c. Raised body temperature d. Lower back pain	Which of the following is NOT a symptom of osteoporosis? a. Decreased bone density b. Raised body temperature c. Hair loss d. Painful joints
Put as much of the question wording in the stem as possible, rather than duplicating material in the options, which is difficult to read and makes the question look unwieldly.	Theorists of pluralism have asserted which of the following? a. The maintenance of democracy requires a large middle class. b. The maintenance of democracy requires autonomous centres of countervailing power. c. The maintenance of democracy requires the existence of a multiplicity of religious groups. d. The maintenance of democracy requires a predominantly urban population. e. The maintenance of democracy requires the separation of governmental powers	Theorists of pluralism have asserted that the maintenance of democracy requires which of the following? a. A large middle class b. Autonomous centres of countervailing power c. The existence of a multiplicity of religious groups d. A predominantly urban population e. The separation of governmental powers
Avoid giving grammatical clues to the correct answer. As the example of the poor question shows, the question ends with the word ‘an’. Since only one of the options start with a vowel, it is the only plausible answer.	A fertile area in the desert in which the water table reaches the ground surface is called an a. Mirage b. Oasis c. Water hole d. Polder	What is a fertile area in the desert in which the water table reaches the ground surface called? a. Mirage b. Oasis c. Water hole d. Polder

Other guidelines to consider:

Avoid ‘all of the above’ and ‘none of the above’ options – It is often an easy option to eliminate. For instance, if students can eliminate one of the other options then they can also immediately eliminate ‘all of the above’. If you do want to use these options in some of your questions, ensure that it is sometimes also the correct option. Do not only use it to ensure that you have a certain number of options (e.g. 4 options or 5 options).
Statements that are correct but that do not answer the question can be strong distractors.
Avoid ‘always’ and ‘never’ options – these are easy options to rule out as extreme statements like this are often incorrect. Few things are universally true or false.
Do not create distractors that are so similar to the correct choice that it confuses students who really do know the answer. Distractors should be clearly different from the correct answer and not only in how it is phrased or what is emphasised.
Ensure that your options are mutually exclusive (that they do not overlap) – An example of overlapping options is:

How long does an annual plant generally live?

It dies after a year
It lives for many years
It needs to be replanted each year
It lives for 3 months

In this example, options a and c are not mutually exclusive. If a plant dies after a year it implies that it needs to be replanted every year. If this is the correct answer (as in this example) it technically means that both answers are correct. If both a and c were incorrect, it would have meant that a student would be able to eliminate both of these options and would only have two options left to consider.

Keep your options as short as possible – often, the more difficult a multiple-choice question becomes (see next section on creating questions that test higher order thinking) the longer the stem becomes. It is best practice to keep the options as short as possible to avoid the question becoming unwieldly.

You can use the below checklist (which includes the guidelines discussed in this section) before setting up multiple-choice questions. This checklist can also be used to assure the quality of your exiting question bank:

1.	There is no irrelevant material in the question (stem)
2.	The question is asked in clear straightforward language
3.	The question is stated positively (or if not the negative part of the question is highlighted/ emboldened/ underlined so as to sufficiently stand out)
4.	As much of the question wording as possible is in the stem and not repeated in the options
5.	There are no grammatical clues in the question
6.	‘All of the above’ or ‘none of the above’ options are avoided as far as possible (if not they are feasible options)
7.	‘Always’ and ‘never’ options are avoided
8.	Distractors are not so similar to the correct option that it would confuse a student who knows the correct answer
9.	All options are mutually exclusive (none of the options overlap with any of the other options)
10.	Options are not unnecessarily long

Bloom’s Taxonomy
Developing good multiple choice questions
Some types of online tests can be automatically graded, reducing your marking time and allowing you to conduct more regular assessment in your course. In addition, it allows you to use a variety of assessment methods. Depending on how you choose to incorporate online assessments in your overall assessment strategy, online assessment can lead to more prepared students and it can help your students to understand the module content better. Online assessment can be used particularly effectively in formative assessment.

Bloom’s taxonomy is a helpful resource to determine and classify the difficulty-level of your questions.

Description of different levels of Bloom’s

Bloom’s level	Description
Remember	Remembering previously learned material (ONLY remembering is enough Lowest level of learnin Listing learned informatio Remembering terms, methods, facts, concepts, specific items of information
Understand	Ability to grasp the meaning of materia Communicating an ide Explaining idea Summarizing materia Understanding facts and principles
Apply	Applying concepts and principles to new situation Applying laws and theories to practical situation Demonstrating correct usage of a method or procedur Applying rules, methods, concepts, principles, laws, theorie Requires higher level of understanding than comprehension
Analyze	Breaking material down into component part Understanding the organizational structur Analysis of relationships between part Recognition of organizational principles involve Understanding both the content and structural for Analyzing the elements
Evaluate	Ability to judge the value of materia Use of definite criteria for judgment Value judgments based on clearly defined criteria
Create	Putting parts together in a new whol Formulating new patterns or structure Abstract relationship Communicating an idea in a unique wa Proposing a new set of operation Creating new or original thing Take things and pattern them in a new way

Examples of multiple choice questions on different levels of Bloom’s

Example question	Bloom’s level	Explanation
Which of the following is one of the major approaches to psychology? a) psychoanalysis b) structuralism c) psychiatry d) New Age Movement	Remember	Students would only have to remember the major approaches to psychology and that psychoanalysis is one of them.
Which one of the following describes what takes place in the so-called PREPARATION stage of the creative process, as applied to the solution of a particular problem? a) The problem is identified and defined. b) All available information about the problem is collected. c) An attempt is made to see if the proposed solution to the problem is acceptable. d) The person goes through some experience leading to a general idea of how the problem can be solved. e) The person sets the problem aside, and gets involved with some other unrelated activity.	Understand	This question is on an understand level if the descriptions in options a to e are slightly different than the descriptions students received in their class notes or text books. If the descriptions are a summary/ using different wording students would need to understand what happens in the preparation stage to select the correct answer. If the descriptions are exactly the same as in their class slides or textbooks, however, this would be a remember question.
Sally is newly appointed in a managerial role. She has noticed that two of her subordinates, Lisa and Joan, do not get along. Lisa and Joan need to work as part of a larger team, and the conflict between them causes stress for the rest of the team. Because of the conflict between Lisa and Joan they are often abrupt and unclear when they communicate in writing with each other which often leads to misunderstandings. Sally sends both Joan and Lisa for a training course in how to professionally communicate via email in order to solve the problem. Which of the following problem solving steps did Sally skip in her attempt to find a solution to the problem? a) Evaluate and learn b) Define the problem c) Select the best solution d) Implement	Apply	The problem solving steps need to be applied to the scenario provided.
The Swift Fox is a small mammal (about the size of a cat) that once lived throughout the prairie grasslands of central Canada. However, due to a number of factors including hunting, it had completely disappeared from Canada by the late 1970s. In an attempt to restore this species, foxes from American populations have been released in Canada. However, the number of Swift Foxes in Canada still remains low. Which factor is most likely NOT contributing to this? a) Much of the fox’s original grassland habitat has been replaced with agriculture, which has reduced the quality and quantity of habitat for the foxes and the availability of suitable prey. b) The foxes released in early reintroductions did not survive well, likely due to a lack of knowledge of sources of food and den locations, or because the American foxes were not adapted to the longer Canadian winters. c) Coyotes, which are increasing in numbers, are feeding on the foxes, or competing with the foxes for available resources. d) Swift foxes are still being trapped for their pelts (fur), as they have been since the early 1800s. e) Swift foxes may have been the unintended victim of trapping and poisoning campaigns directed at other mammals, such as coyotes, wolves, and ground squirrels.	Analyze	The student is required to analyse each of the provided options in relation to the scenario and decide on the factor that is least likely to contribute to the problem. An argument can also be made that this question is on an ‘evaluate’ level.

Planning your assessment using Bloom’s Taxonomy

It can be helpful to create a test blueprint before setting up questions for a test to plan the proportions of different levels of questions that you want to include. For instance, if you want the biggest portion of your test to consist of application level questions, you can allocate a larger percentage of the total marks to this level. It is also useful to make use of a test blueprint if you are part of a teaching team where more than one person sets up a test. A blueprint can then be used to divide the work between the different lecturers. An example of a test blueprint is shown below:

Example of test blueprint:

	Unit 1	Unit 2	Unit 3	Unit 4	Total
Remember	7 (step 3)	2 (step 3)	1 (step 3)	1 (step 3)	11 (step 2)
Understand	4 (step 3)	6 (step 3)	8 (step 3)	2 (step 3)	20 (step 2)
Apply	4 (step 3)	10 (step 3)	12 (step 3)	24 (step 3)	50 (step 2)
Analyze	0 (step 3)	3 (step 3)	3 (step 3)	8 (step 3)	14 (step 2)
Evaluate	0 (step 3)	0 (step 3)	0 (step 3)	5 (step 3)	5 (Step 2)
Create	0 (step 3)	0 (step 3)	0 (step 3)	0 (step 3)	0 (step 2)
TOTAL	15 (step 4)	21 (step 4)	24 (step 4)	40 (step 4)	100 (step 1)

Unit 1

Unit 2

Unit 3

Unit 4

Total

Remember

(step 3)

(step 2)

Understand

(step 3)

(step 2)

Apply

(step 3)

(step 2)

Analyze

(step 3)

(step 2)

Evaluate

(step 3)

(Step 2)

Create

(step 3)

(step 2)

TOTAL

(step 4)

100

(step 1)

Setting up a test blueprint:

Initial setup: Create a column for each of the units of work (or chapters) that will be covered in the test with a ‘total’ column at the end. Create a row for each of Bloom’s levels with a ‘total’ row at the bottom.

Step 1: Fill in the test total in the cell at the bottom right

Step 2: Fill in the total marks you want to allocate to different levels of Bloom’s in the column on the right

Step 3: Fill in the total marks that you will be able to get from each of the units/ chapters to get to the total in the column of the right for each of the Bloom’s levels

Step 4: Calculate the totals of each of the columns to confirm that you get to the total score of the test.

References
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. Raths, J. & Wittrock, M. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.
University of Toronto. (2001). Guidelines for composing multiple-choice test questions. Retrieved from http://bio150.chass.utoronto.ca/tips/mc.htm

Except where otherwise noted, content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

About | Contact