How to Track Practice Progress Without Fooling Yourself

A practice record is only evidence if the comparison is held fixed. Same mode, same difficulty, same length, and ideally the same time of day — change any of those and the difference you are reading belongs to the setting rather than to you. After that, the order to read the numbers in is accuracy, then median answer time, then score, and the most flattering numbers on the screen are the ones to trust least.

Measurement hygiene sounds like a dull subject next to the practice itself. It is the difference between a record that tells you when to change something and a record that tells you a pleasant story.

Compare like with like

Progress monitoring is a recognised part of structured practice, not an optional extra: the IES practice guide Assisting Students Struggling with Mathematics devotes a whole recommendation to monitoring the progress of learners receiving intervention. What it assumes, and what a self-directed practice record usually lacks, is a consistent measure.

The comparability trap has three common shapes.

Different mode. A mixed-operation round contains a step that a single-operation sprint does not: deciding which operation the question is asking for. Its answer times are structurally longer and its accuracy structurally lower. Moving between the two and calling the difference progress is a category error.

Different difficulty. Difficulty labels in a drill app change what is generated — the size of the numbers, whether a borrow is involved, how long a memory item stays on screen. A score that fell after you moved up a rung has told you that the rung is harder.

Different length. Longer rounds contain more fatigued answers, and fatigue lengthens the slow tail more than it moves the typical answer. How long a brain training session should be covers what that does to the practice; here the point is narrower: a ten-minute round and a two-minute round are not the same instrument.

National statistics agencies take the choice of measure seriously for the same reason. The Office for National Statistics's quality and methods guide for household wealth states that "wealth data are highly skewed, so these statistics use medians rather than means to reflect central tendency" — the summary statistic is chosen to fit the shape of the data, and then used consistently. A personal practice record deserves the same discipline on a much smaller scale: pick the measure that suits the data, then stop changing it.

Read accuracy before score

A score in a drill game is a composite. It typically folds together how many you got right, how fast you answered, how long your streak ran, and a multiplier for the difficulty you chose. That makes it responsive — it moves a lot, which feels informative — and unspecific, because it moves for several unrelated reasons.

Accuracy answers one clean question: of the questions you were asked, what proportion did you get right? Median answer time answers a second: how long did a typical answer take? Between them they cover almost everything a score does, and they cover it separably, so you can see when a rising score is actually a speed-for-errors trade rather than a gain. That trade is the subject of accuracy before speed.

Use the median rather than the mean answer time. A single long think pulls an average up sharply while leaving the median where it was; reading your response time numbers works through an example where one stalled answer moves the mean by almost a second and the median by nothing at all.

Three metrics that flatter you

Personal best. It is the maximum of a noisy series, so it only ever ratchets upward and it never describes a typical session. It is also sensitive to luck: a round that happens to generate familiar questions produces a better run than the same skill would on a harder draw. Enjoy it, do not trend it.

Streak. A streak counts days on which you opened the app. That is adherence, which is genuinely worth having and worth protecting — building a daily practice habit that survives a bad week covers how to keep one without it turning into a liability — but it contains no information about whether the practice is working.

End-of-session performance. The last round of a session is your warmest, most primed performance of the day, and it is the one you remember. Soderstrom and Bjork's review Learning Versus Performance makes the general case that performance during training "is often an unreliable index of whether the relatively long-term changes that constitute learning have taken place", and observes that people "often mistakenly interpret their performance during acquisition as a reliable guide to long-term learning".