What Is OCR? How Scanned Text Becomes Searchable

A scanned page looks like text, but to a computer it is a picture. You cannot search it, copy a sentence from it or have it read aloud until software works out which shapes are which letters. That software is OCR. This guide explains how it works, how accurate it is and how to get better results, drawing on Wikipedia's article on optical character recognition.

What OCR does

Wikipedia defines optical character recognition as the conversion of images of typed, handwritten or printed text into machine-encoded text, whether from a scanned document, a photo of a document, a photo of a scene such as a street sign, or subtitles on a video. Its uses include data entry from invoices, receipts and passports, and making scanned documents searchable by converting them to PDFs. Google's help for the Google Drive scanner, for example, says it saves receipts, letters and billing statements as searchable PDFs.

Wikipedia notes that more sophisticated OCR systems can preserve the page layout and produce a PDF that includes both the original image of the page and a searchable textual representation. You see the scan; search and copy use the text.

How it works, step by step

Wikipedia's article breaks the process into stages:

1. Pre-processing. Wikipedia says OCR software often pre-processes images, and lists techniques including de-skewing a tilted page so lines run straight, despeckling spots, binarisation (turning the image black and white to separate text from background), removing lines and boxes, and layout analysis, which identifies columns, paragraphs and captions.
2. Line, word and character detection, including separating characters that touch and joining ones broken by artefacts.
3. Recognition. The software compares each shape with known characters, either by matching patterns or by extracting features such as lines, loops and intersections. Wikipedia notes that some engines, such as Cuneiform and Tesseract, take a second pass, using letters recognised confidently the first time to read the rest.
4. Post-processing, where word lists and context can correct likely mistakes. Wikipedia notes that constraining the output to a lexicon, a list of words allowed to appear, can raise accuracy, but can cause problems when a document contains words outside it, such as proper nouns. That is a good reason to check surnames and place names in OCR output by eye.

How accurate is it?

Not perfect. Wikipedia notes that recognising typewritten Latin-script text is still not 100% accurate even with clear images, and cites a study of 19th- and early 20th-century newspaper pages that found character-by-character accuracy for commercial OCR software ranging from 81% to 99%. Wikipedia adds that recognition of hand printing, cursive handwriting and printed text in some other scripts, especially East Asian characters with many strokes, is still the subject of active research.

How accuracy is measured matters too. A worked example: a typed page of 2,000 characters read at 99% character accuracy contains about 20 wrong characters. If words average five letters plus a space, the page has about 333 words, and those 20 errors can spoil up to 20 of them, about 6%. Wikipedia makes the same point: without a lexicon to correct non-existent words, a 1% character error rate can become a 5% or worse word error rate when every word must be perfect.