The illusion of learning · TaiLORED IDEAS

We judge whether we are learning by how the learning feels, and that feeling is often manufactured by things that have almost nothing to do with whether anything is being learned.

Think of the last TED talk you watched. The speaker walks out onto that red circle and takes their time. The images behind them are gorgeous and arrive exactly when they are needed. Complex ideas are unpacked one clean step at a time, each one landing just before you have to reach for it, and every so often the speaker pauses, holds the room, and lets a line settle. The whole thing washes over you. By the end you feel like you understand something you did not understand eighteen minutes ago.

Now try to explain the mechanism they described to somebody else. Not the topic, the mechanism: how the thing actually works. Most of us get a sentence in and stall. The talk was fluent, and fluency felt like comprehension, but almost nothing was retained that could be produced on demand.

The reason we get caught this way is that we have no direct read on our own memories. Judging what you know is not like a thermometer, which measures temperature itself. It is more like a speedometer, which measures how fast the wheels are turning and infers the rest. Every judgment about your own learning is an inference drawn from whatever cues happen to be available (Schwartz et al., 1997).

The cue we lean on hardest is ease. If material goes down smoothly, or an answer arrives quickly, we treat that as evidence. As a rule of thumb it is not unreasonable. The problem is that ease is produced by all sorts of things unconnected to learning. Words shown in a large font are judged more memorable than the same words shown small (Rhodes & Castel, 2008). In one experiment, learners’ predictions about their own recall moved with font size, which had no effect whatsoever on how much they actually remembered, and barely moved with whether they would get another study opportunity, which had a large effect (Kornell et al., 2011).

Speed deceives the same way. When people answered trivia questions and predicted which answers they would recall later, the answers that came fastest drew the highest confidence and were the least likely to survive. The ones they had laboured over were the ones that stuck (Benjamin et al., 1998).

That inversion is the whole problem. Conditions that make learning feel quick and smooth often produce less that lasts, while conditions that introduce difficulty slow visible progress and improve what remains. This is the illusion of learning, and it is not a failure of effort or attention. It is a gauge that reads confidently and reads wrong.

If ease is what produces the illusion, then friction is the antidote. Conditions that feel effortful, that make you work on the problem rather than follow along with it, are known as desirable difficulties (Bjork, 1994). They feel like slower progress and they are not. Spacing and retrieval practice are the two best understood of them, and both share the same signature: harder in the moment, better a month later.

In practice

What this means for AI capability training

This is the mechanism behind a great deal of training that everyone enjoys and nobody uses. Picture the polished in-person session: clearly sequenced, smoothly delivered, every step visible on the screen at the front, the room nodding along. Or picture its more common cousin, the library of online training videos, where a confident presenter demonstrates the tool at a comfortable pace while the learner watches, agrees, and ticks the module off. Both generate real fluency in the moment. Neither asks the learner to produce anything.

That is the whole problem, stated plainly. Any training whose central experience is observation, however well made, is unlikely to produce capability that survives the week. Watching someone else solve a problem builds a feeling of competence and almost none of the thing itself. What builds capability is being made to work on the material directly: to attempt a task from your own role with the ideas just covered, get a mediocre result, work out why it went wrong, and go again. The problems have to look like your problems, not tidy demonstration cases, because that wrestle with your own messy context is the part that does the learning.

There is a second trap waiting at the end. Most training closes with a satisfaction survey, which asks in some form how much people enjoyed the session and how confident they now feel. That question is answered from exactly the cue this card is about. A smooth, enjoyable, well-delivered session scores highly whether or not it changed anything, and a session that made people struggle productively can score worse while having taught considerably more. An organisation can run a full training series, collect excellent scores, feel entirely satisfied, and be no more capable than it was in June.

So the question worth asking is not whether people enjoyed it or felt confident at the end. It is what they can do a month later that they could not do before. Comfort is not evidence, and it is not free.

The sources

Read it for yourself.

The papers this piece draws on, and what each one shows.

Judgments about our own learning are inferences from cues, not direct readings of memory.

Schwartz, B. L., Benjamin, A. S., & Bjork, R. A. (1997). The inferential and experiential basis of metamemory. Current Directions in Psychological Science, 6(5), 132–137.
Predictions of recall tracked font size, which changed nothing, and ignored extra study, which changed a great deal.

Kornell, N., Rhodes, M. G., Castel, A. D., & Tauber, S. K. (2011). The ease of processing heuristic and the stability bias: Dissociating memory, memory beliefs, and memory judgments. Psychological Science, 22(6), 787–794.
Words presented in larger type are judged more memorable than they actually are.

Rhodes, M. G., & Castel, A. D. (2008). Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions. Journal of Experimental Psychology: General, 137(4), 615–625.
The answers that came to mind fastest drew the most confidence and were the least likely to be recalled.

Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998). The mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index. Journal of Experimental Psychology: General, 127(1), 55–68.
Conditions that slow visible progress during learning can improve what survives it.

Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185–205). MIT Press.

From the science to the work

See how the science shapes the training.

Every engagement begins with a conversation about what you’re trying to change.

Book a conversation Why the training sticks