Why I Hate Multiple Choice Question Papers and the Revised Bloom’s Taxonomy

In one sentence I have put two things that I hate. By this, do I mean that the two are related, parallel or orthogonal or in some oblique angle (you do not mention two things in one sentence unless there is a commonality or there are identifiable distinctions between the two). This post is about the commonality, in the main, between Multiple Choice Question Papers (MCP) and Revised Bloom’s Taxonomy (RBT).

Both MCP and RBT have found their way into the assessment of students in a class, though RBT is perhaps a step removed. Yet they come from the same genome. The name – assessing faculty performance.

When a question paper is submitted to the examination cell in a university, faculty members are to assign the RBT level for each question. It so happened that during the COVID 19 pandemic, there were no classroom instructions, and Zoom or Google Class Rooms ran away with the loot.

For the easily perceptible reason – a long answer typed out/written on a paper by a student, then photographed/scanned on the student’s mobile will consume so much time. There was no consideration for this time factor, at least formally.

On the side of the faculty, the additional effort was wider. From end to end, if it was decided that all the questions will be in MCQ format, and there would be no relaxation in assigning an RBT level to each question (though students are not to be aware of this).

The above is to set the context in which MCQs were to be operable. The following takes the second topic I hate, RBT, first. It is very strongly associated with the term, “Learning Outcome”, obviously created by a management guru.

The word “Taxonomy” comes from biology/zoology, meaning something like “Law of Arrangement”.  In 1956, a group of educators felt the need to create a model of comparison of graduates from different universities as regards their potential professional performance. This group was headed by one Benjamin Bloom; hence, Bloom’s Taxonomy

I have a technical paper from the American Society of Civil Engineers authored by three professors teaching a course on concrete design with same syllabus. One must understand that designs in civil engineering are strait-jacketed as the consequences of a failure are bound to be unimaginably high and varied. Just imagine the Three Gorges Dam in China, that has a 600 km long reservoir failing! Therefore, students’ answers to the questions cannot differ very much (you have to arrive at the optimum design – a commercial consideration and not an academic one). But, in assessing the RBT level of each question (identical question papers were given to the three classes), the three academicians did assign different RBT levels o the same questions, varying, if I remember right, two to four. That is, RBT is subjective. The same point would be made from a different perspective later.

One easily notices that the diagrammatic representation is a pyramid, and more importantly the objectives are nouns, like “knowledge”, “comprehension”, “application” ... There is nothing wrong with that. Yet, as the teacher says here, it is better to use the verb as the anchor in the sentence that even native English speakers do not do as much as they should. Therefore the taxonomy of learning has to be changed, wholesale. This change is indicated in the figure below:

“Knowledge” has become “Remember”, “Comprehension” is changed to “Understand”, “Analysis” to “Analyze” and so on; yet, note the position reversal between ”Synthesis” and above it, “Evaluation” in the original taxonomy. In the revised verb oriented taxonomy, “Synthesis” at the level below the crest in the original becomes “Create” at the apex in the revision. This forces “Evaluation” to take on the action-role of “Evaluate”, a down-grade. In general, though nouns are taught in English 101 before verbs, in the RBT, it is verbs that take the pole position.

The figure below shows RBT and the supposedly corresponding verbs.

This is where, when RBT was discussed over two days (simultaneously in more than 20 sections with two people in charge of a room), the level of comprehension – sorry, “understand”ing – of the faculty members in the section that I was in charge of came to light.

To set the stage, the question was to name the agency that is the predecessor of the current agency, NASA. The answer, in my words, is the National Advisory Committee for Aeronautics (NACA), transmuting itself to the National Aeronautics and Space Administration. I believe this change enabled the agency to be one with executive power and not merely one of advising, under the Department of Defence perhaps. But, what do I know?

One faculty member had written down the question in which the verb was at the level of “remember”, the lowest. He, following the suggested list of verbs at level III, and used interpret. The altered question, as I recall some six or seven years later, was Interpret NASA. And, I am not joking. What is there to interpret in a proper noun? It is like asking anyone to interpret my name, Raghuram Ekambaram! What I do can, of course, be interpreted but I myself cannot be. It is ever changing. The moment I hit the “Create” button in my blog space, I would be a changed “I”!

One more point that I am posing as a question to the readers. Are double promotions in RBT allowed? There are faculty members, most senior and the next level down who think they can! Remember, the bottom most level is “Remember”. The next higher level is “Understand” and the third level is “Apply”. If a problem demands nothing more than solving a simple equation with only one unknown, can it ever be anything other than “Remember”?

I have another bone to pick with RBT. It does not take into consideration the time taken to answer a question. I showed in that session a problem and told the attendees that if the student “understands” the problem, she would take less than one minute. YES. If a student used the appropriate formula, it may take as much as 15 minutes (because the equation contains trigonometric function at inconvenient places!). The question cared two hoots for the verbs suggested. So, I asked the faculty whether and how they would differentiate the two ways to the answer. It was silence, all around.

The point that I am making is, if the question paper setter has an idea in mind to be transformed into an MCQ and the correct response is also known, is it easy to compile the three wrong answers without inducing ambiguity or obviousness and demoting the RBT level? The answer is a resounding NO.

If there are four possible answers, one may allow one answer to be perceived to be wrong immediately. But, creating the other two possible answers have to give no clue as to their wrongness. Even, if the right answer appears correct, the remaining two statements cannot appear to be wrong without brain work. Then, how would you rank this MCQ on RBT? By the correct answer, by the two brain-twisting yet wrong answers or the obviously wrong answer (if it is a choice)?

RBT is not objective as it cannot be independent of the student’s mental processes, which show up as the time factor. What, the, is the use of RBT? I find none.

Now I come to MCQs. Unless the teacher wants to make it easy for the student, framing the question, including the choices to be given, is not an easy task. One can make an MCQ easy to answer in two ways: one, refer to any of the many books and memorize the short-answer questions and the answers given at the end of a chapter while preparing. There was one such question paper in which out of 13 questions, twelve required at the most one equation (including the 10 mark questions) and included only one unknown. All the questions were given by the question paper setter an RBT level of “Analyze”, tut, tut...

Two, make the wrong answers appear to be correct. The student would then tell herself that she could not spend more than three minutes and 36 seconds for a two-mark question (assuming 100 marks to be answered in 180 minutes). That is, change the metric to include a time component. This realization would fire a newer set of neurons in distinctly different directions and the answer would emerge as it did to Saul on the Road to Damascus. Students’ brains, particularly their synapses, are ever changing. This should be the purpose of teaching in the class. Creating newer pathways for communication between students’ synapses. Perhaps this has already been done, but not in the university I was employed.

Questions demanding calculations or longer explanations (as I did for a test in Transport Economics), the solution lies in burying the substantive part of the question in the woods of unnecessary statements in the question. (The questions were 30 words long.) It is only the well prepared student who would catch the crux and respond appropriately and within the time. The others will be lost in the woods, perhaps forever.

The question paper has to consider the above while making a question as MCQ. This aspect of setting of a question paper for an MCQ was not discussed except in the set of faculty I was interacting with (don’t complete a sentence with a preposition. I felt a numbing pain in my knuckles when I finished my sentence).

It is not my loss, but the students would feel the loss. And I know one instance, the faculty member could not convince some students of his class that their answer is wrong and succumbed to granting the undeserved additional marks. He failed in differentiating between the students to some extent.

What is the obligation of the teacher to the profession? There are two: differentiate students based on the knowledge they have absorbed and be able to use in a given situation. Try one’s level best to reduce the level difference between the students’ comprehension.

With the above, I believe, I have convinced myself that my arguments are water-tight. I am under no obligation to convince others; they can go their own way.

Raghuram Ekambaram