Intelligent data capture: reading handwriting and form fields

Most archives contain two different problems wearing the same cover. There is printed matter, which a recognition engine handles well, and there is everything a person wrote by hand into a box on a form. Intelligent data capture in the UAE is usually bought because of the second category: the values somebody handprinted into a claim form, an application or a site sheet, which currently reach your systems only when a member of staff retypes them.

ICR, OCR and IDP are not the same thing

These three get used interchangeably by vendors, which makes buying harder than it should be. They sit at different levels. One reads characters, one reads handwriting inside known field positions, and one makes decisions about the document as a whole.

What handprint recognition can and cannot read

ICR is trained on separated, handprinted characters: block capitals in individual boxes, digits in a combed field, a date written one character per cell. Given that, it performs. Given continuous cursive, it does not, and no amount of tuning changes the fact that joined-up handwriting has no reliable character boundaries to find.

• Reads well: block capitals in printed boxes, digits in combed date and amount fields, tick boxes and shaded response marks.
• Reads with verification: unboxed handprint on ruled lines, mixed upper and lower case, characters that overflow their box.
• Does not read reliably: cursive script, signatures, handwritten Arabic, overwritten corrections, anything crossed out and rewritten.
• Reads better than expected: numeric-only fields, because the character set is small and check digits give the validation step something to test against.

The form itself decides much of this. Comb boxes, drop-out colours that vanish under the scanner lamp, generous field spacing and a clear instruction to use capitals will each do more for capture quality than any software setting. Where a client is redesigning a form anyway, an hour spent on capture-friendly layout pays for itself for years.

Confidence scores are the point, not a footnote

Every field ICR returns carries a score describing how sure the engine is. That number is the mechanism that makes automated capture safe to use, because it separates the values you can accept untouched from the values a person needs to look at.

You set the threshold, and you set it per field rather than per form. An optional comments box and a national identifier do not deserve the same treatment. In practice we set identifiers, amounts, dates and anything with a downstream financial or legal consequence to a strict threshold, and let descriptive fields run looser. A system tuned so that nothing is ever flagged is not accurate, it is untested.

Human-in-the-loop verification

1. 1

   Capture and score

   The engine reads each defined field and attaches a confidence value to the result.

2. 2

   Route by threshold

   Fields above the threshold pass through. Fields below it queue for verification, together with anything a validation rule rejected.

3. 3

   Verify against the image

   The operator sees a cropped image of that field beside the proposed value, not the whole page. Correcting a single field takes seconds and keeps the reviewer looking at exactly the thing in doubt.

4. 4

   Double-key the critical fields

   Nominated fields are keyed independently twice and compared. Where two operators disagree, the record escalates. This is how identifiers and amounts are protected.

5. 5

   Validate as a record

   Cross-field checks run last: does the date make sense, does the total match the lines, does the reference exist in your master data.

6. 6

   Release and report

   Verified data is exported, and the batch report shows how many fields needed intervention and which ones. Those figures are how the next batch gets configured better.

When ICR is worth it, and when it is not

ICR earns its place where the same structured form arrives repeatedly and in volume. Ten thousand identical claim forms is exactly the shape of problem it was built for. A few hundred one-off handwritten letters is not: setting up field templates for material that never repeats costs more than typing it. We will say so rather than sell the licence.

Legacy archives sit somewhere in the middle. Where a historical form was used unchanged for years, a single template can unlock decades of boxes. Where the form was revised every eighteen months, each version needs its own template, and the sensible answer is often to capture only the handful of fields you will actually search on and leave the rest as a searchable image.