Key takeaways
- An online test can be well written and still produce an uninterpretable number, because accuracy depends on the conditions and the comparison group rather than on the questions.
- The four failure points of unsupervised self-administration are conditions you control, a comparison group nobody documented, no observer, and no enforced retest interval.
- Proctoring is not uniformly decisive. A 2025 study of 226 participants in a low-stakes setting found no overall performance difference by proctoring status, but memory tasks behaved differently across conditions and observed cheating in proctored sessions was rare, at around 4 per cent.
- Repeat exposure inflates scores on its own. Retest effects are largest between the first and second sittings — one analysis reported an effect size of 0.72 — and remain positive through a fourth administration.
- Regulators have acted on unsupported cognitive claims. In January 2016 the US Federal Trade Commission announced a $2 million settlement over brain-training advertising that claimed scientific studies proved benefits they could not substantiate.
- Our own app, IQ Test & Brain Training Games, includes an IQ-style test, and the same limits apply to it: it is taken unsupervised on a phone, and its Google Play listing describes the results as estimates based on app activities, not a clinical or professional assessment.
An online IQ test is not accurate in the sense people want, and the reason has very little to do with whether its questions are any good. Accuracy in testing is a property of the entire procedure — the conditions, the comparison group, the administration and the interval between sittings. An unsupervised web test cannot control any of those, so its output has no defined meaning even when the items themselves are competently written.
That is a different complaint from the one usually made. This article is about delivery, not about the honest limits of a well-administered instrument, which are covered separately in the limitations of IQ testing.
Accuracy is a property of the procedure, not the questions
ETS defines reliable scores as those that are consistent across different occasions of testing, across different editions of a test, and across different raters scoring the same responses. Everything in that definition is about circumstances.
A normed instrument exists as a package: a fixed item set, a scripted administration, a trained administrator, a documented standardisation sample, and published evidence about consistency. Remove the packaging and the items are just questions. Copying the questions into a web form reproduces the least important part — as our guide to what IQ tests actually measure sets out, the items are the visible surface of a much larger apparatus.
The four things an unsupervised test cannot do
It cannot fix the conditions. The norms were collected under one set of conditions. At home there is no way to hold timing, interruptions, note-taking, a second screen or a second attempt constant. A score produced under unknown conditions cannot be compared with one produced under known conditions.
It cannot document the comparison group. A percentile means something only against a stated sample. Free web tests are typically ranking you against the people who chose to take that test on that site, which is self-selected in ways nobody has characterised. A rank against an undescribed group is not a rank.
It cannot observe. A live administrator does more than police cheating. They note whether the person understood the task, whether attention lapsed, whether an answer was a guess or a considered response — context that determines whether the numbers are even usable.
It cannot enforce a retest interval. Repeat exposure produces real score gains, discussed below, and nothing stops a web test being taken repeatedly until a pleasing number appears.
What the proctoring research actually found
It would be convenient if supervision were a single switch. The evidence is more interesting than that.
Scherrer and colleagues, writing in the Journal of Intelligence in 2025, compared three conditions across 226 participants in Germany: webcam-proctored testing, unproctored testing by random assignment, and unproctored testing that participants chose. The setting was low-stakes research rather than selection or diagnosis.
They found no significant overall performance difference by proctoring status. Proctored participants did slightly better on reasoning and divergent thinking, and slightly worse on memory tasks. Memory tasks also showed measurement invariance violations across conditions — meaning the tasks were not behaving identically in each setting, which is a measurement problem rather than a score problem. Observable cheating in the proctored sessions was rare, around 4 per cent, mostly note-taking or photographing the screen. The authors' conclusion is the practical one: supervision matters most for tasks that are easy to cheat, such as memory tasks, and less for complex reasoning in low-stakes conditions.
The honest reading is therefore narrower than "unproctored tests are inflated". It is that the effect of supervision depends on the task and the stakes, and that you cannot know which applies to a random web test because nothing about it is documented.
The retest problem, which applies even to an honest taker
Scores rise with repetition, and the rise has nothing to do with ability changing.
Jendryczko, Scharfen and Holling, in the Journal of Intelligence in 2019, examined performance across repeated administrations of cognitive ability tasks. The largest retest effect appeared between the first two sessions, with an effect size of 0.72, and effects remained positive and significantly different from zero through the fourth administration before levelling off — a pattern the authors relate to the power law of practice.
An effect of that size is large enough to swamp whatever the test was trying to detect. Supervised assessment handles it with enforced intervals and alternate forms. A web test handles it by letting you press retry. The wider question of what these gains do and do not mean is covered in can practice raise an IQ score.
Why the claims around these products attract regulators
Cognitive-benefit marketing has a documented enforcement history. On 5 January 2016 the US Federal Trade Commission announced a $2 million settlement with the maker of a widely advertised brain-training program over claims that it improved performance on everyday tasks at school, at work and in athletics; that it delayed age-related cognitive decline and protected against mild cognitive impairment, dementia and Alzheimer's disease; and that it reduced cognitive impairment associated with a range of health conditions. The agency stated that the company had falsely claimed scientific studies proved those benefits. A larger judgment was suspended based on the company's financial condition.
That case was about training claims rather than testing claims, but the principle transfers: a cognitive claim needs substantiation proportionate to the claim, and "our users score higher" is not substantiation.
Where our own app fits
IQ Test & Brain Training Games is our own app, built by Reign Creative, and it includes an IQ-style test — so this article's argument applies to it as much as to any web quiz.
Its Google Play category is Educational game and its content rating is Everyone. Its IQ-style test covers numerical reasoning, logic, patterns, vocabulary, working memory, spatial reasoning and processing speed, and the app also has a memory test, a reaction time test, a personality questionnaire, and memory, logic, focus, vocabulary and mental math games. It keeps personal bests, accuracy, streaks, response speed and a history of past results.
The listing is explicit about what those results are: IQ test, personality test, memory test and other results are estimates based on app activities, not medical, psychological, clinical, educational-placement, employment or professional diagnostic assessments. The four failure points above apply to it in full: you set the conditions, the listing documents no comparison group, nobody observes the session, and the app invites you to revisit the test and compare against your own history, which is exactly where the retest effect shows up. The four categories this series keeps apart still sort it cleanly: the app is practice, entertainment and educational activity, and it is not standardised psychological assessment. Our broader position on what practice products can claim is set out in do brain training apps work.
If you want a number that means something
There is exactly one route: a qualified administrator using a current, normed instrument under its specified conditions, with a report that includes a confidence interval. That is slow and it costs money, and those two facts are why free alternatives exist.
If you are curious rather than making a decision, a quiz is fine — provided you treat the output as entertainment. The checklist separating one from the other is in fun IQ quizzes and standardised assessments, with the rest of the series in the IQ tests and reasoning hub and the wider education and brain training section.
Good to know
Frequently asked questions
Are online IQ tests accurate?
Why do online tests so often return a flattering score?
Does supervision actually change results?
Does the IQ Test & Brain Training Games app contain an IQ test?
Sources
- Effects of Proctoring on Online Intelligence Measurement: A Literature Overview and an Empirical Study (opens in a new tab)Journal of Intelligence (Scherrer, Petry, Breit, Urban, Preuß & Preckel, 2025), via PubMed Central — accessed
- The Impact of Situational Test Anxiety on Retest Effects in Cognitive Ability Testing: A Structural Equation Modeling Approach (opens in a new tab)Journal of Intelligence (Jendryczko, Scharfen & Holling, 2019), via PubMed Central — accessed
- Lumosity to Pay $2 Million to Settle FTC Deceptive Advertising Charges for Its "Brain Training" Program (opens in a new tab)U.S. Federal Trade Commission — accessed
- Test Reliability — Basic Concepts (Research Memorandum RM-18-01) (opens in a new tab)Educational Testing Service (Samuel A. Livingston, 2018) — accessed
About the author
Reign Creative LLC — The independent app studio that builds the apps covered on this site and publishes every article here. On Google Play it publishes as Reign Collective Apps.
We use AI tools to help research, draft and edit our articles. Each article lists the sources its factual claims rely on, and details about our apps come from their live Google Play listings. No article on this site has been reviewed by an independent expert unless the reviewer is named on it.
Spotted an error? Tell us — how corrections are handled is set out in our editorial policy.