| 1. Berlin: | |
| 2. Rabat: | |
| 3. Luanda: | |
| 4. Copenhagen: | |
| 5. Kabul: |
Except where otherwise noted, content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Methods and implementation ideas
Click here to add your own text
Objective vs Subjective question types
Before you start using online assessment, it is important to consider what you want to achieve with the assessment. This will help you choose the tools and methods that will help you to achieve your assessment goals. A common misconception about online assessment is that your test will be automatically graded. Reducing marking time is also the most common incentive for starting to use online assessment. The fact is that only some question types are marked automatically by an online assessment platform such as an LMS. This is where objective vs subjective questioning plays an important role. The difference between these two question types are:
Objective question types require students to select a response from a list of alternatives provided to them or to supply a word or short phrase. Only one correct answer is identified. Subjective question types allows a student to organise an original answer, using their own words. These answers are usually longer (such as a paragraph or a couple of sentences).
Therefore, if the purpose for using online assessment is to reduce marking time and to have an online assessment program mark for you, you need to make use of objective question types. Subjective question types are not marked automatically by any platform. You will always need to mark subjective question types yourself, usually the marking can be alleviated by creating a rubric.
Objective question types
The most well-known and commonly used objective question type, is a multiple choice question which is a question with a number of options of which a student must select the correct or most correct option. There are, however, typically many other objective question types available to use on an online assessment platform such as Respondus or an LMS, which allow you to use a wide spectrum of different types of questioning in your test. In addition, online assessment provides several options for multiple choice questions that are not necessarily practical or feasible in a written format. All of the objective question types shown in this section will be automatically marked.
Multiple choice questions (MCQ)
A simple example of a multiple choice question is the following:
You can do more with a MCQ online, however. For instance, consider using an image as part of the question, or a short video or audio clip (see examples here). Multiple choice questions can also be developed to test higher-order thinking.
Multiple response questions
A multiple response question is similar to a multiple choice question but instead of having to choose one option from a list, a student is required to choose more than one of the options.
A simple example of a multiple response question is the following:
As with a MCQ, you can incorporate images, videos or audio clips as part of your multiple response questions.
Matching questions
Matching questions are similar to match the column questions sometimes also used in written assessments. A simple example of a matching question is the following:
| Berlin | Germany |
| Rabat | Afghanistan |
| Luanda | Morocco |
| Copenhagen | Denmark |
| Kabul | Angola |
A matching question can be more difficult if there are some extra options in one column that do not have a match in the other column, such as in the following example:
| Berlin | Germany |
| Kabul | Afghanistan |
| Luanda | Angola |
| Copenhagen | Denmark |
| Rabat | Morocco |
| Fez | |
| Astana | |
| Monrovia | |
| Islamabad |
In addition, matching questions do not only have to be questions with single words in each column. Instead, a single word or phrase can be used in one column to describe an explanation or description in another column. Another option is using images in one column and short descriptions in the other column. Matching questions are often used to test not only knowledge recall but also application of knowledge. Here are some examples of different types of matching questions. It is important to note that the assessment program (both Blackboard and Questionmark) automatically shuffles one of the two columns so that each students who completes an assessment sees a different order.
Ranking question
A ranking question requires a student to rank steps of a process or order of events in the correct sequence. An example of a simple ranking question is:
| Titanic sinks | 1st |
| Pluto discovered | 2st |
| Helicopter invented | 3st |
| Moscow Olympics | 4st |
| Rwandan genocide | 5st |
Instead of one or two words, a short phrase or paragraph can also be used in the left column of a ranking question (to describe a step in a process for instance) in order to make the question more complex. It is important to note that the assessment program (both Blackboard and Questionmark) automatically shuffles the column on the left so that each students who completes an assessment sees a different order.
Select a blank question
A select a blank question is a longer question, usually in a paragraph format, with multiple answers. This question type is not commonly available on all online assessment platforms, but it is available on Questionmark, for example. An example of a simple select a blank question is:
Human Resources Management consists of various processes within an organisation. The process aimed at attracting applicants that match job criteria is known as [Recruitment/ Selection/ Hiring/ Remuneration Management/ Training and Development/ Performance Management/ Employee relations]. The aim of [Recruitment/ Selection/ Performance Management/ Remuneration Management/ Employee Relations] is to evaluate performance in order to reward good performance and address poor performance. [Recruitment/ Selection/ Hiring/ Remuneration Management/ Training and Development/ Performance Management/ Employee relations] is the process that involves determining salaries, incentives, and fringe benefits.
With this question students are provided with a list of possible answers for each ‘blank space’ in the question. This is an alternative to the better-known fill in the blank question. The advantage of this question, however, is that grading the correct answer is not influenced by incorrect spelling or synonyms of the correct answer. With a fill in the blank question students can easily make a spelling or typing error and will then not receive a mark as the system only recognises key words as identified by the lecturer as correct answers. Students also sometimes use synonyms for the identified key words, which is also not recognised by the system. The select a blank question eliminates these potential issues when a list of potential options is provided for each blank space in the question.
Hotspot question
A hotspot question allows a student to click somewhere on an image to indicate an answer. The system automatically gives a mark if the student clicks within a predetermined area but receives a zero if he/she clicks outside of this area.
An example of a hotspot question is:
1
Students who clicks anywhere in the green marked area will receive a mark and students who click outside of the area will receive zero. Students do not see the green marked area as part of the question, the image in the example above is to illustrate how the question is set up on the system (Blackboard/ Questionmark). As the example illustrates, this question type works well in modules where students need to visually be able to distinguish between certain areas such as in Physiology or Biology. However, it can also be used creatively in other types of modules as in these examples. It is important to note that this question type should be used with caution if you have students in your class who are visually impaired as the software on their computers will typically not read an image.
Drag and drop question
Drag and drop questions allow a student to drag and drop a number of labels to appropriate spaces on an image. A student can either get a single mark for placing all the labels at the correct spaces on the image or can get a mark for each label placed correctly.
A simple example of a drag and drop question is:
Drag and drop the labels at the bottom of the image to the corresponding countries on the image. Place the label anywhere on its corresponding country:
[h5p id=”1″]
Students do not see the colour-coded marked areas as part of the question, the image and labels in the example above is to illustrate how the question is set up on the system. Drag and drop questions also work well for an image of text such as matching a step in a process with a description. It is important to note that the labels at the bottom of a drag and drop questions should be short (preferably one or two words) as it must fit in next to each other at the bottom of the image. This question type should be used with caution if you have students in your class who are visually impaired as the software on their computers typically cannot read an image. This question type is not commonly available on all online assessment platforms but it is available on Questionmark.
Developing good multiple choice questions
It is important that a multiple-choice question is constructed well, before focusing on constructing questions that test higher order thinking. A multiple-choice question consists of a question (sometimes also referred to as the stem) and a number of options, of which only one is correct. Both the question and the options need to be constructed well to ensure the quality of a multiple-choice question. Since multiple-choice questions are often re-used, it is worth setting up good quality questions from the start. These questions can often be adapted to build a large question bank over time. The following guidelines are useful to consider in setting up multiple-choice questions.
| Guideline | Example of a poor question | Suggestion for improving the question |
|---|---|---|
| Avoid unnecessary and irrelavant material in the question to avoid confusion and misdirecting the student. | Paul Muldoon, an Irish post-modern poet who uses experimental and playful language, uses which poetic genre in “Why Brownlee Left”? a. Sonnet b. Elegy c. Narrative poem d. Dramatic monologue e. Haiku |
Paul Muldoon uses which poetic genre in “Why Brownlee Left”? a. Sonnet b. Elegy c. Narrative poem d. Dramatic monologue e. Haiku |
| Use clear and straightforward language in the question. Avoid making the question a reading comprehension rather than an assessment of the subject matter. | As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation? a. A decrease in the labour force participation rate of women b. A dispersing effect on population concentration c. A downward trend in the youth dependency ratio d. A broader base in the population pyramid e. An increased infant mortality rate |
A major decline in fertility in a developing nation is likely to produce which of the following? a. A decrease in the labour forces participation rate of women b. A dispersing effect on population concentration c. A downward trend in the youth dependency ratio d. A broader base in the population pyramid e. An increased infant mortality rate |
| Use negatives sparingly. If negatives must be used (as you may sometimes want to test knowledge about exceptions to a rule), capitalize, underscore, embolden or highlight it. | Weak question: Which of the following is not a symptom of osteoporosis? a. Decreased bone density b. Frequent bone fractures c. Raised body temperature d. Lower back pain |
Which of the following is NOT a symptom of osteoporosis? a. Decreased bone density b. Raised body temperature c. Hair loss d. Painful joints |
| Put as much of the question wording in the stem as possible, rather than duplicating material in the options, which is difficult to read and makes the question look unwieldly. | Theorists of pluralism have asserted which of the following? a. The maintenance of democracy requires a large middle class. b. The maintenance of democracy requires autonomous centres of countervailing power. c. The maintenance of democracy requires the existence of a multiplicity of religious groups. d. The maintenance of democracy requires a predominantly urban population. e. The maintenance of democracy requires the separation of governmental powers |
Theorists of pluralism have asserted that the maintenance of democracy requires which of the following? a. A large middle class b. Autonomous centres of countervailing power c. The existence of a multiplicity of religious groups d. A predominantly urban population e. The separation of governmental powers |
| Avoid giving grammatical clues to the correct answer. As the example of the poor question shows, the question ends with the word ‘an’. Since only one of the options start with a vowel, it is the only plausible answer. | A fertile area in the desert in which the water table reaches the ground surface is called an a. Mirage b. Oasis c. Water hole d. Polder |
What is a fertile area in the desert in which the water table reaches the ground surface called? a. Mirage b. Oasis c. Water hole d. Polder |
Other guidelines to consider:
- Avoid ‘all of the above’ and ‘none of the above’ options – It is often an easy option to eliminate. For instance, if students can eliminate one of the other options then they can also immediately eliminate ‘all of the above’. If you do want to use these options in some of your questions, ensure that it is sometimes also the correct option. Do not only use it to ensure that you have a certain number of options (e.g. 4 options or 5 options).
- Statements that are correct but that do not answer the question can be strong distractors.
- Avoid ‘always’ and ‘never’ options – these are easy options to rule out as extreme statements like this are often incorrect. Few things are universally true or false.
- Do not create distractors that are so similar to the correct choice that it confuses students who really do know the answer. Distractors should be clearly different from the correct answer and not only in how it is phrased or what is emphasised.
- Ensure that your options are mutually exclusive (that they do not overlap) – An example of overlapping options is:
How long does an annual plant generally live?
- It dies after a year
- It lives for many years
- It needs to be replanted each year
- It lives for 3 months
In this example, options a and c are not mutually exclusive. If a plant dies after a year it implies that it needs to be replanted every year. If this is the correct answer (as in this example) it technically means that both answers are correct. If both a and c were incorrect, it would have meant that a student would be able to eliminate both of these options and would only have two options left to consider.
- Keep your options as short as possible – often, the more difficult a multiple-choice question becomes (see next section on creating questions that test higher order thinking) the longer the stem becomes. It is best practice to keep the options as short as possible to avoid the question becoming unwieldly.
You can use the below checklist (which includes the guidelines discussed in this section) before setting up multiple-choice questions. This checklist can also be used to assure the quality of your exiting question bank:
- There is no irrelevant material in the question (stem) ☐
- The question is asked in clear straightforward language ☐
- The question is stated positively (or if not the negative part of the question is highlighted/ emboldened/ underlined so as to sufficiently stand out) ☐
- As much of the question wording as possible is in the stem and not repeated in the options ☐
- There are no grammatical clues in the question ☐
- ‘All of the above’ or ‘none of the above’ options are avoided as far as possible (if not they are feasible options) ☐
- ‘Always’ and ‘never’ options are avoided ☐
- Distractors are not so similar to the correct option that it would confuse a student who knows the correct answer ☐
- All options are mutually exclusive (none of the options overlap with any of the other options) ☒
- Options are not unnecessarily long ☐