Key takeaways
- In a multiple-choice question, difficulty comes mostly from the quality of the wrong answers rather than from the subject.
- A plausible distractor is one someone with partial knowledge would genuinely consider; an implausible one can be eliminated without knowing anything.
- Recognition is easier than recall, which is why an open-answer version of a question is far harder than the multiple-choice version.
- A fair hard question tests knowledge; an unfair one tests obscurity, ambiguity or how carefully you read the wording.
- Good difficulty progression increases the precision required rather than the obscurity of the subject.
In a multiple-choice question, the difficulty lives in the wrong answers. The same question about the same fact can be trivial or genuinely hard depending only on what it is offered against.
That is the single most useful thing to know about quiz design, whether you are writing questions or trying to get better at answering them.
Distractors do the work
Take any factual question. Offer the correct answer alongside three obviously implausible options and it is easy — you can eliminate the wrong ones without knowing the subject at all.
Offer the same correct answer alongside three options that someone with partial knowledge would genuinely consider, and it becomes a real test.
Good distractors are usually one of:
- The common misconception. The thing people believe that is not true.
- The adjacent fact. Correct about something else, in a way that requires precision to reject.
- The plausible alternative. Consistent with everything the reader knows except the specific detail being tested.
Bad distractors:
- Obviously wrong to anyone.
- Wrong in category — an option that is not even the right kind of thing.
- So similar to the correct answer that both are defensible.
That last one is worth separating, because it is the boundary between hard and unfair.
Recognition versus recall
The same fact tested in two formats produces two very different difficulties.
| Format | Cognitive demand | Relative difficulty |
|---|---|---|
| Multiple choice | Recognise the right option | Lower |
| Open answer | Produce it from memory | Higher |
| Multiple choice with "none of the above" | Recognise, plus verify | Between |
This is why a quiz can be made harder without changing its subject at all: remove the options.
It is also why retrieval practice is such an effective study technique — producing an answer from nothing does more for memory than recognising one does. The Dunlosky review of learning techniques rates practice testing among the most effective methods, and this is the mechanism. Memory techniques that actually work covers it further.
Construction tells
Poorly written multiple-choice questions leak the answer through their structure. Recognising these makes you better at quizzes and better at writing them:
The longest option. Writers frequently add qualifying detail to the correct answer to make it precisely true, which makes it longer.
The grammatically consistent one. If three options do not fit the stem grammatically and one does, the one that fits is likely correct.
Absolutes. "Always", "never", "all", "none" are rarely correct in questions about complicated subjects, because reality is usually hedged.
The middle values. In numeric questions, extremes are more often distractors.
None of these is reliable individually. Collectively they let you outperform your knowledge, which is a sign of question construction rather than of your ability.
Fair difficulty versus unfair difficulty
The distinction that matters most in designing a quiz people want to keep playing.
Fair hard question: tests whether you know something precisely rather than approximately. You either know it or you do not, and finding out is informative.
Unfair hard question: tests something other than knowledge.
- Obscurity. A detail nobody would encounter in ordinary engagement with the subject.
- Ambiguity. Two defensible answers, one arbitrarily marked correct.
- Trick wording. Punishes reading speed rather than understanding.
- Stale facts. Answers that were correct when written and are not now.
A quiz full of unfair hard questions is not challenging, it is annoying — and the difference is legible to players almost immediately.
How difficulty should progress
The right axis is precision required, not obscurity of subject.
An easy question about a topic asks for a broad fact. A hard question about the same topic asks for a specific one. That progression keeps the subject familiar and increases the demand, which is what a difficulty curve should do.
The wrong progression is easy questions about well-known subjects and hard questions about obscure ones. That does not increase the challenge, it changes who can answer at all — and it stops rewarding the knowledge the player has been building.
Themed quizzes and vocabulary
A themed quiz restricts the answer space to one subject, which changes both halves of the problem.
The domain vocabulary becomes the barrier. For an anime quiz, that means demographic labels, format terms, production vocabulary and genre categories — anime terminology: a glossary covers the common set, and the Anime News Network lexicon is a broader reference.
Distractors also get better, because a themed subject supplies genuinely adjacent facts — the other work by the same studio, the other series in the same season. That is why themed quizzes can be harder than general ones about the same difficulty of fact.
Where our app fits
Anime Trivia & Word Games combines quiz questions with word puzzles, so both the recognition and the production formats above are in play.
It is free to download on Google Play, supported by ads, with optional in-app purchases and an Everyone content rating.
Our anime trivia guide covers approaching the quiz side, and word game strategy covers the puzzle side, where the search techniques matter more than recognition. Everything we make here is under anime and creative games.
Good to know
Frequently asked questions
What makes a quiz question hard?
What is a distractor?
Why are some multiple-choice questions easy to guess?
Why is an open-answer question harder?
What makes a hard question unfair?
Sources
- Improving Students' Learning With Effective Learning Techniques (Dunlosky et al., Psychol Sci Public Interest 2013;14(1):4–58) (opens in a new tab)PubMed, U.S. National Library of Medicine — accessed
- Anime News Network Lexicon (opens in a new tab)Anime News Network — accessed
