THINK INSIDE THE BOX · TOOL 15 · B2 · 50 min

The Examiners' Ladder

In 1948 a group of university examiners wanted to save themselves work. Their dull-sounding classification became one of the most famous diagrams in education.

← The Five Whys  ·  All the tools  ·  The Food Pyramid →

Today: you'll learn how a committee of examiners sorted thinking into levels, use the pyramid to classify real questions, and practise subject and object questions: Who wrote it? What did they write?
Before we start — talk these through:
  • Think of the last exam or test you took. Did it ask you to remember, or to think?
  • What is the difference between knowing a fact and understanding it?
  • Which is harder for you: analysing someone else's idea or creating your own?

The vocabulary of this story

Ten items. Say your answer out loud before you reveal it.

A · What do they mean here? Work it out from the context

Read the sentence, guess the meaning of the word in bold, then reveal.

The examiners wanted to classify every question by the kind of thinking it required.
classify — to put things into groups according to their type. (noun classification; adj. classified)
Most of the test only asked students to recall facts they had learnt the week before.
recall — to bring something back into your mind; to remember. (also a noun: instant recall)
The manual was revised decades later, and some of the categories were renamed.
revise — to change a text or plan to improve it. (noun revision; in British English also: to study again before an exam)
The taxonomy gave teachers a common framework for planning lessons and writing tests.
framework — a structure of ideas used to plan or judge something. (a legal framework, a theoretical framework)

B · Which word matches the meaning?

The three words are: evaluate · apply · memorise. Read each meaning, say the word, then reveal.

(verb) To use a rule, method or idea in a real situation.
apply — “Can you apply the formula to a problem you haven't seen before?”
(verb) To learn something so well that you can repeat it exactly.
memorise — “She had memorised the dates but couldn't explain what happened.”
(verb) To judge how good, useful or successful something is.
evaluate — “Evaluate the two plans and recommend one.” (noun evaluation)

C · Which word completes all three phrases?

One missing word fits every phrase in the group. Say it, then reveal.

critical ___  ·  higher-order ___  ·  wishful ___
thinking — (critical thinking, higher-order thinking skills, wishful thinking)
___ skills  ·  ___ a habit  ·  ___ confidence
develop — to make something grow stronger. (build also works; noun development)
draw a ___  ·  make a ___  ·  a fair ___
comparison — (draw / make a comparison; a fair comparison; in comparison with; verb compare)

Now use them — out loud

Before we read:
  • What did you have to memorise at school? Was any of it useful?
  • Describe a time you had to apply something you had learnt to a real problem.
  • Which is easier for you to recall: names, numbers or faces?

The story, in three parts

Read each part. After it, answer the quick check from memory, then discuss one question.

Part 1 — A meeting in Boston

In 1948, at a convention of the American Psychological Association in Boston, a small group of college examiners met informally. Their problem was practical. Every year their universities set long comprehensive exams, and every year writing them took an enormous amount of work.

One of them was Benjamin Bloom, associate director of the Board of Examinations at the University of Chicago. His idea was simple: if test questions could be classified by the kind of thinking they required, universities could share their banks of questions instead of each writing them from scratch.

The group met about twice a year. In 1956 it published Taxonomy of Educational Objectives, Handbook I: Cognitive Domain. It sorted thinking into six levels, from the simplest to the most complex: Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation.

Quick check — from memory, then reveal:

1. What practical problem did the examiners have?

2. What did Bloom hope universities could share?

3. What were the six original levels?

1. Writing their long annual exams took an enormous amount of work.

2. Their banks of test questions.

3. Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation.

Discuss:
  • Why might a boring, practical problem lead to an important idea?

Part 2 — An unfamiliar word

At first, hardly anyone noticed. “Taxonomy” was an unfamiliar word, and the book was written for examiners rather than for classroom teachers. But over the following decades it spread, and it was eventually translated into 22 languages.

When researchers used the framework to analyse real courses and tests, the results were uncomfortable. The questions were weighted heavily towards the bottom level: recognising and recalling information. The framework made it easy to see what was missing — the levels where students apply, analyse and judge.

In 2001 a team led by Lorin Anderson and David Krathwohl, who had worked on the original handbook, published a revision. The levels became verbs — Remember, Understand, Apply, Analyse, Evaluate, Create — and the top two changed places: Synthesis, renamed Create, moved above Evaluate. The revision also relaxed the idea of a strict ladder: the levels were allowed to overlap.

Quick check — from memory, then reveal:

1. Why did the book get little attention at first?

2. What did analyses of real tests show?

3. Name two changes in the 2001 revision.

1. “Taxonomy” was an unfamiliar word, and the book was written for examiners.

2. Most questions only asked students to recognise or recall information.

3. Any two: the levels became verbs; Synthesis became Create and moved to the top, above Evaluate; the strict hierarchy was relaxed.

Discuss:
  • Do you think most tests today still check memory more than thinking?

Part 3 — The trouble with the bottom

Bloom's taxonomy, usually drawn as a pyramid, is now part of teacher training in many countries, and teachers use its verbs — explain, compare, judge, design — to write lesson aims. But it can have a side effect. If remembering is the “lowest” level, it can start to look unimportant.

In 2016 the biologist Susan Lawler, of La Trobe University in Australia, described what had happened in her own field. Teaching students to identify plants and animals had been dropped from courses as too simplistic — the bottom of the pyramid. “The problem is that identifying a plant or insect is not simple at all,” she wrote. To name an insect correctly, you need to know what a mandible, a maxilla and a rostrum are. Meanwhile, people from industry were complaining that graduates couldn't identify common plants and animals.

Her argument was that the higher levels depend on the lower ones: you cannot analyse, evaluate or create in a subject you know nothing about. The pyramid is a way of seeing different kinds of thinking — not a reason to stop remembering.

Quick check — from memory, then reveal:

1. What do teachers use Bloom's verbs for?

2. Why had identification been dropped from Lawler's courses?

3. Why, according to Lawler, is identification not simple?

1. To write lesson aims.

2. Because it was seen as too simplistic — the bottom level of the pyramid.

3. Because it needs a lot of detailed knowledge — for example, what a mandible, a maxilla or a rostrum is.

Discuss:
  • Is memorising underrated in schools today?

How the tool works

Six levels of thinking, from the simplest at the bottom to the most complex at the top (the 2001 version). Each level has its own verbs — which makes it easy to check what a question really asks.

CreateEvaluate · judgeAnalyse · organiseApply · use, carry outUnderstand · explain, compareRemember · recall, recognisemore complex thinking

Using it, step by step

  1. Read the question or task and find the verb: list, explain, compare, judge, design…
  2. Ask what students must actually do in their heads — the verb can mislead (“explain” is only Remember if the explanation was learnt by heart).
  3. Place each task on the pyramid. Then look at the whole test or lesson: are all the tasks on one level?
  4. Add tasks at the missing levels — but keep the bottom: higher thinking needs knowledge to work on.

Try it yourself

A teacher has written six questions about a short story. Which level of the 2001 taxonomy does each one mainly ask for?

1. Write a new ending for the story, told by the dog.
2. What is the name of the main character's sister?
3. Explain in your own words why the father leaves home.
4. Was the mother right to keep the secret? Give your reasons.
5. Which details in the first paragraph prepare us for the ending?
6. Use the story's advice to plan what you would say to a friend in the same situation.

1. Create — producing something new.

2. Remember — recalling a fact from the text.

3. Understand — explaining the meaning shows understanding.

4. Evaluate — making a judgement and supporting it.

5. Analyse — breaking the text into parts and seeing how they connect.

6. Apply — using an idea in a new, real situation.

Where the tool breaks

Verbs aren't levels. “Explain” can be pure memory if the explanation was learnt by heart.

It can make knowledge look cheap. The bottom of the pyramid is the foundation, not the least important part.

Thinking isn't a ladder. Real tasks mix levels; even the 2001 revision relaxed the strict order.

Talk it through:
  • Think of a test or textbook page you know well. Which levels does it use most?
  • Should every lesson reach the top of the pyramid?

Its cousins

Maslow's Hierarchy of NeedsAnother famous pyramid — of needs, not thinking. Open the lesson →
Kolb's Learning CycleLearning as a circle of experience and reflection, not a ladder. Open the lesson →
The Ladder of InferenceHow we climb from what we see to what we conclude — often too fast.
The Forgetting CurveWhy the bottom level needs regular practice.

Grammar — subject and object questions

Bloom's bottom level is full of questions like Who wrote it? and What did they write? They look similar, but one uses did and the other doesn't. It depends on whether the question word is the subject or the object.

Object questions ask about the object of the verb. Use do/does/did and question word order.
“Bloom met the examiners.” → “Who did Bloom meet?” (answer: the examiners)
Subject questions ask about the subject. No do/does/did — keep statement word order.
“Bloom met the examiners.” → “Who met the examiners?” (answer: Bloom)
This works with who, what, which, whose, how many.
“What happened in 1956?”  “How many people came?”  “Which level comes first?”
Other auxiliaries stay in their normal place in subject questions.
“Who has read the report?”  “What can go wrong?”
Watch out: ~~Who did write the book?~~ → “Who wrote the book?”   ~~What did happen?~~ → “What happened?”

1. Subject or object?

Write a question for the bold answer. Decide whether you need did.

1. Our teacher marked the tests last night. → Who ?

2. The new manager changed the whole timetable. → What ?

3. A burst pipe flooded the exam hall. → What ?

4. She asked her colleague for help. → Who ?

5. The students chose the second topic. → Which topic ?

1. …marked the tests last night? (subject)

2. …did the new manager change? (object)

3. …flooded the exam hall? (subject)

4. …did she ask for help? (object)

5. …did the students choose? (object)

2. Correct the questions

Three questions have a mistake with did; one is correct. Write the correct version.

1. Who did invent the telephone? →

2. What you remembered from the lecture? →

3. Who did you sit next to in the exam? →

4. How many students did pass the test? →

1. Who invented the telephone?

2. What did you remember from the lecture?

3. Correct — you is the subject and who is the object, so did is needed.

4. How many students passed the test?

Now write two questions about this lesson for your teacher — one subject question and one object question — and say which level of the pyramid each one tests.

Speak

Climb the pyramid out loud. Choose a task, start the timer, and talk.

Useful language: “That's just remembering.” · “This one asks you to judge …” · “Who decided …?” · “What did they change?”
Time remaining: 4:00
Task 1 — Six questions

Choose a film, book or news story everyone knows. Ask one question at each level of the pyramid, from Remember to Create.

Task 2 — Your best and worst test

Describe a test that only asked you to remember, and one that made you really think. Which did you learn more from?

Task 3 — Defend the bottom

Argue that knowing facts is as important as creative thinking. Use an example from your job or studies.

To finish — the harder questions

  1. Should exams test more creativity — even though it is harder to mark fairly?
  2. Is it possible to be creative in a subject you know very little about?
  3. Who should decide what counts as “higher” thinking?
  4. From memory: ask four questions about the story of the taxonomy — two subject questions and two object questions.