Skip to content

Abstract Reasoning Tests: How the Items Are Built

Abstract reasoning items are engineered objects: a rule, a key and distractors chosen to match specific errors. Knowing the anatomy makes practice sharper.

By Published Updated 5 min read

Reign Creative LLC makes IQ Test & Brain Training Games, which this article links to, so we have a commercial interest in it. How we write and check articles.

Key takeaways

  • An abstract reasoning item has four parts: a stem of figures or numbers, a hidden transformation rule, one correct key and a set of distractors built to match specific plausible errors.
  • Difficulty is engineered, not accidental. Published research on generated number-series items found that items needing two arithmetic steps were far harder than one-step items, and that item features predicted most of the variance in difficulty.
  • Modern item banks are increasingly produced by automatic item generation, in which a template specifies which features can be varied and an algorithm produces many items targeting the same construct.
  • Because distractors are designed around common errors, a near-miss answer is usually evidence of a specific misconception rather than carelessness.
  • Our own app, IQ Test & Brain Training Games, has logic games built on patterns and rule discovery and an IQ-style test that includes patterns and spatial reasoning; its listing calls the results estimates based on app activities, not a professional assessment, so read them as a record of how you did on the app's own questions.

An abstract reasoning item is an engineered object with four parts: a stem, a hidden rule, one correct key and a set of distractors. Nothing about it is arbitrary. The rule is chosen from a short catalogue, the difficulty is tuned by how many rules operate at once, and the wrong answers are selected because real candidates produce exactly those errors. Once you can see the construction, practice stops being a guessing game and becomes a search with a short, finite checklist.

The four parts of an item

The stem is the material you are shown: a row of frames, a grid with a gap, or a run of numbers. The rule is the transformation that turns one element into the next. The key is the single option consistent with that rule. The distractors are the other options.

The rule catalogue for figural items is small. Elements rotate, reflect, move around a frame, change shading, change size, change in number, or appear and disappear. A hard item combines two of those on different features — shape count increasing while shading alternates, say — so that spotting one rule is not enough to eliminate the near-misses.

Numeric items work the same way with arithmetic in place of geometry. A published study of automatically generated number-series items describes three stimulus features that drive the design: the type of number element, the type of arithmetic operation, and the number of arithmetic operations. The operations used were arithmetic progressions with a constant increment, geometric progressions with a constant ratio, Fibonacci-style addition sequences, and multiplicative sequences.

A worked example, and what the wrong answers mean

Take the series 3, 6, 11, 18, 27, ?

Work the first differences: 6 − 3 = 3, 11 − 6 = 5, 18 − 11 = 7, 27 − 18 = 9. The differences themselves rise by 2 each time, so the next difference is 11 and the answer is 27 + 11 = 38. The same series has a second description — each term is a square plus two: 1 + 2 = 3, 4 + 2 = 6, 9 + 2 = 11, 16 + 2 = 18, 25 + 2 = 27, 36 + 2 = 38. Two descriptions of one rule is a good sign you have found the real structure rather than a coincidence.

Now consider three plausible wrong answers and what each one means:

  • 36 — you reused the last difference (+9) instead of extending the ladder. This is the single most common sequence error.
  • 37 — you extended the ladder by 1 rather than 2, giving +10.
  • 54 — you abandoned the difference structure and doubled the last term.

An item writer who includes 36 and 37 is not padding the options. Each one catches a named misconception, which is why a near-miss on a real test usually tells you something specific about your method.

A two-rule version looks like this: 2, 6, 5, 15, 14, 42, 41, ? Here the operation alternates. Multiply by 3, then subtract 1: 2 × 3 = 6, 6 − 1 = 5, 5 × 3 = 15, 15 − 1 = 14, 14 × 3 = 42, 42 − 1 = 41, so the next term is 41 × 3 = 123. Nothing in it is harder arithmetic than the first example; the difficulty is entirely in noticing that two rules take turns.

Difficulty is designed, and it is predictable

The same number-series study quantified the effect. Items requiring two steps of arithmetic operation had an average difficulty of 1.28, against minus 0.70 for one-step items, and using fractions or irrational numbers pushed difficulty up further compared with integers. Most usefully, the authors reported that "77.4% of the variance in item difficulty... was explained by the stimulus features implemented in the automatic item generator."

That last figure is the practical takeaway for anyone preparing. Difficulty comes overwhelmingly from structural features you can learn to recognise — how many rules, how many steps, what kind of number — rather than from some ineffable quality of the item. The checklist that follows falls straight out of it: count the terms, take first differences, take ratios if the differences are not stable, check whether the operation alternates, and check whether the series interleaves two independent sequences.

How item banks are made now

Large-scale testing needs many items that target the same construct without repeating, and hand-writing them is slow. Automatic item generation addresses this. One review defines it as "the process of generating items/questions from various inputs, including models, templates, or schemas", where an item model is "a template that specifies the features in an item that can be manipulated". Subject-matter experts build a cognitive model of the skill, express it as a template, and an algorithm varies the manipulable features to produce a bank.

The listed promises include reduced item-writing time and cost, support for continuous development of large item pools, and the ability to tailor items for learning. For a candidate this has one direct consequence: memorising specific items is close to worthless, because the bank is large and the surface features vary while the rule family does not. Learning the rule family is the only portable skill.

It is also worth carrying over a caution from the fairness literature. Research on cross-cultural testing has found consistent group differences on visually mediated tasks that are widely assumed to be neutral, which is covered in more detail in nonverbal reasoning tests. A generated bank inherits whatever assumptions the template encodes.

Practising the move without pretending it is a test

If you want to practise rule discovery, practise the search procedure rather than collecting answers. Run the checklist in a fixed order, and when you get one wrong, name the misconception before moving on. That habit is the difference between repetition and useful practice.

IQ Test & Brain Training Games is our own app, built by Reign Creative. Its logic games include patterns and rule discovery, and its IQ-style test covers patterns and spatial reasoning alongside numerical reasoning, logic, vocabulary, working memory and processing speed; its Google Play listing does not say whether any of those items use the figural formats described above. Practising rule discovery there is an analogue of the task, not an equivalent, and the listing describes the test's results as estimates based on app activities, not medical, psychological, clinical or professional diagnostic assessments. The app keeps personal bests, accuracy, streaks, response speed and a history of past results, so what it tracks is your record against yourself.

Faster arithmetic does make the numeric variety of these items less punishing, simply because less of your attention goes on computing and more is available for the rule — the methods in mental math tricks that work are the relevant ones. And because rule search loads working memory hard, working memory and training is worth reading before you conclude that a plateau means anything about you.

For the construct these tests are supposed to sample, start with what IQ tests actually measure. The rest of the material sits in the IQ tests and reasoning hub and the wider education and brain category, and the app itself is described on the IQ Test & Brain Training Games app page.

Good to know

Frequently asked questions

What is an abstract reasoning test?
It is a timed test of rule discovery. You are shown a series or grid of figures, or sometimes numbers, that changes according to a rule you are not told, and you choose the option that continues it. Employers often call the same thing inductive or diagrammatic reasoning. The construct being targeted is the ability to find structure in unfamiliar material rather than recall of anything you have learned.
What makes one abstract reasoning item harder than another?
Mostly the number of rules you must hold at once and how many steps separate one term from the next. In one published study of automatically generated number-series items, items requiring two arithmetic operations had an average difficulty of 1.28 against minus 0.70 for one-step items, and items using fractions or irrational numbers were harder than those using integers. Visual clutter matters far less than rule depth.
Are the answer options random?
No. Distractors are chosen deliberately, and good item writers build them from the errors real candidates make — continuing a difference instead of extending it, applying a rule to the wrong feature, or stopping one step early. Some automatic generation systems now produce distractors from documented error patterns. If you are consistently landing on the same kind of wrong answer, that pattern is diagnostic.
Does IQ Test & Brain Training Games contain abstract reasoning items?
Its Google Play listing describes logic games built on patterns and rule discovery, and an IQ-style test that covers patterns and spatial reasoning alongside numerical reasoning, logic, vocabulary, working memory and processing speed. The listing does not say whether any of those items use the figural series or grid formats described in this article. It also describes the test's results as estimates based on app activities, not a professional assessment, so read a result there as a record of how you did on the app's own questions rather than as a score on a standardised abstract reasoning test.

Sources

  1. Automatic item generation: foundations and machine learning-based approaches for assessments (opens in a new tab)Frontiers in Education — accessed
  2. Automatic Generation of Number Series Reasoning Items of High Difficulty (opens in a new tab)Frontiers in Psychology — accessed
  3. Reasoning tests - numerical, verbal, logical and diagrammatical (opens in a new tab)University of Manchester Careers Service — accessed
  4. Origins Matter: Culture Impacts Cognitive Testing in Parkinson's Disease (opens in a new tab)Frontiers in Human Neuroscience — accessed

About the author

— The independent app studio that builds the apps covered on this site and publishes every article here. On Google Play it publishes as Reign Collective Apps.

We use AI tools to help research, draft and edit our articles. Each article lists the sources its factual claims rely on, and details about our apps come from their live Google Play listings. No article on this site has been reviewed by an independent expert unless the reviewer is named on it.

Spotted an error? Tell us — how corrections are handled is set out in our editorial policy.

Keep reading