OCR: how a picture of a page becomes text

The useful thing to understand about OCR is that it is a chain of steps, and the last step gets blamed for failures created several steps earlier. When OCR output is poor, the cause is usually the scan, not the recognition engine.

The pipeline

1. 1

   Image capture

   Everything downstream inherits this. A skewed, low-contrast or under-resolved image cannot be rescued by better software. Capture settings are chosen for the material: bitonal is fine for clean black print, greyscale preserves faint carbon copies, colour is needed when colour carries meaning such as a stamp or a signature.

2. 2

   Pre-processing

   The image is straightened, borders and punch holes are cleared, speckles from photocopying are removed and contrast is normalised. Aggressive cleanup here can thin faint strokes to nothing, so it is tuned per batch rather than applied globally.

3. 3

   Layout analysis

   The engine segments the page into regions: body text, columns, tables, headers, images, signatures. This is where multi-column documents and dense tables commonly go wrong, producing text that is individually correct but scrambled in reading order.

4. 4

   Character recognition

   Each character shape is classified. Modern engines are trained models rather than shape-matching templates, which is why they cope with varied fonts but still stumble on unusual typefaces and degraded print.

5. 5

   Language modelling

   Results are checked against dictionaries and character-sequence patterns for the language in use. This is why declaring the correct language matters so much, and why a page of product codes or account numbers can score worse than a page of prose. The language model has nothing to help it.

6. 6

   Confidence scoring and output

   Each word carries a confidence value. Text is written back as an invisible layer behind the image in a searchable PDF, or exported as data, or both. Confidence is the hook that routes doubtful results to a human instead of letting them through unnoticed.

That final point is the one that separates a controlled process from an uncontrolled one. OCR always produces an answer. It does not always produce a correct one, and confidence scoring is how you tell the difference at scale.

Where OCR fails, specifically

• Handwriting. Standard OCR is built for print. Handwriting needs ICR, and even then cursive free text remains unreliable. Hand-printed characters in boxed form fields do considerably better.
• Anything printed over text. Stamps, signatures and highlighter over words break character shapes in ways the engine cannot resolve.
• Faded and thermal originals. Old fax rolls and receipt paper lose contrast until characters become suggestions.
• Carbon copies. Low contrast plus offset registration plus stray pressure marks.
• Dot-matrix print. Characters are made of separated dots that pre-processing may thin away.
• Dense tables and multi-column layouts. Recognition can be near-perfect while reading order is wrong, which quietly corrupts any extraction that depends on position.
• Skew and curl. Pages scanned from bound volumes curve near the spine, and curved baselines lower accuracy across the whole region.
• Unusual and decorative fonts, condensed print, and text below the resolution the scan was captured at.
• Mixed-language lines, where a single line switches script mid-sentence.

None of these are exotic. A typical corporate archive contains all of them, which is why an honest OCR conversation is about which parts of the archive are reliably recognisable rather than about a single accuracy figure applied to everything.

Arabic and English in one archive

Almost every UAE archive is bilingual, and Arabic is genuinely harder for OCR than English. The reasons are structural, not a matter of vendor quality.

• Arabic script is cursive. Letters connect, and a letter's shape changes depending on whether it is initial, medial, final or isolated. The engine has to segment a connected word before it can classify anything.
• Dots and diacritics distinguish otherwise identical letter shapes. A speck of photocopier noise, or an over-aggressive despeckle, can change one letter into another.
• Text runs right to left while embedded numbers, product codes and English terms run left to right. Reading order in mixed lines is a recurring source of scrambled output.
• Bilingual forms often place Arabic and English in parallel columns, so layout analysis has to keep two languages and two directions apart on the same page.

The practical consequences are worth planning for. Set the language per document type rather than per project, because auto-detection struggles on short or mixed text. Expect Arabic content to need a higher share of human verification. Capture bilingual material at a resolution and colour depth that preserves diacritics rather than the minimum that makes English readable. And when a field must be right, such as an Emirates ID number or a trade licence number, verify it rather than trusting the extraction.

OCR, ICR and IDP are not the same thing

What to do with output you cannot fully trust

Treating OCR as either perfect or useless are both mistakes. The workable position is to grade it.

1. 1Decide which fields are critical. A wrong searchable word is an inconvenience. A wrong contract number is a lost document.
2. 2Set a confidence threshold per field type, and route anything below it to human verification rather than accepting it silently.
3. 3For the fields that must be right, use double-key entry or a validation rule such as a checksum, a date range or a lookup against an existing master list.
4. 4Keep the image as the record of truth. The text layer is an access aid, not a replacement for the document.
5. 5Measure accuracy on a defined sample with a defined method, and write that method into the contract. Accuracy with no stated measurement is not a commitment.