The equipment and software behind a digitization project

Buyers are often shown a technology list before anyone has looked at their documents. It reads well and it decides nothing. The useful conversation runs the other way: what is the material, what condition is it in, what has to be findable afterwards, and what system is it going into. Equipment and software follow from those answers. This page describes the categories we work with and what each one is actually for.

Capture hardware

One scanner class cannot cover a mixed archive. Sending bound volumes through a sheet-fed transport destroys them, and running loose A4 across a book cradle wastes most of a working day. Real projects sort material by physical characteristics during preparation and route each stream to the right device.

Capture settings are decided per record class rather than set once for a whole project. Resolution, bit depth and colour mode are chosen against what has to remain legible and what the recognition stage needs downstream, then validated on a sample batch before volume production begins. Faint carbon copies, coloured forms and pencil annotations all behave differently, and the sample is where that shows up.

Image processing

Raw output from a scanner is rarely the image you want to keep. Processing sits between capture and recognition, and it is where most of the difference between a usable archive and a frustrating one is made. Applied too lightly, text stays broken and OCR guesses. Applied too heavily, thin strokes and light pencil disappear permanently, which is why processing settings are proved on a sample and then locked.

• Deskew, which straightens a page that entered the transport at an angle. Recognition accuracy falls quickly on skewed text
• Despeckle, which removes the scattered noise that photocopying and ageing paper introduce, without eating punctuation
• Adaptive binarisation, which sets the black-and-white threshold region by region rather than once for the whole page. This is what rescues a document with a coffee stain, a shadowed gutter or uneven ageing, where a single global threshold would wash out one half of the sheet
• Colour dropout, which removes a form's pre-printed grid or coloured background so only the entered data survives into the image. It makes field extraction dramatically more reliable on structured forms
• Automatic cropping and border removal, so pages are not delivered inside a black scanner frame
• Blank page detection and page orientation correction, both of which reduce manual handling more than they sound like they should

Recognition: OCR, ICR and IDP

These three terms get used interchangeably by suppliers and they should not be. They describe different things, and confusing them is how expectations get set wrongly at the start of a project.

1. 1

   OCR: optical character recognition

   Converts printed and typed characters in an image into machine-readable text. It is what makes a scanned page full-text searchable. It behaves well on clean printed originals and gets progressively less reliable on faint, skewed, stamped or heavily photocopied material.

2. 2

   ICR: intelligent character recognition

   Extends recognition to hand-printed characters, most usefully in constrained boxes on structured forms. Free-flowing cursive on unstructured paper remains hard for any engine, and any project relying on it should be tested against a real sample before it is priced.

3. 3

   IDP: intelligent document processing

   The layer above recognition. It classifies a document by type, locates the fields that matter, extracts their values, and validates them against business rules or a reference system. The output is structured data, not a page of text, and that is the difference that lets a document post into a business system.

Arabic and English archives sit side by side across the UAE and both are handled, including bilingual index fields. Arabic script is genuinely harder for recognition engines than Latin script, particularly on handwriting and degraded originals, so we run a representative sample first and set expectations from the result rather than from a brochure figure.

Output formats

The format decision looks technical and is really a governance decision. It determines whether the file opens in fifteen years, whether the text is searchable, and whether anything downstream can consume it.

Many projects take two outputs: a preservation master and a lighter access copy for daily use. It costs little at capture time and it is expensive to recreate later, because the paper has usually gone by then.

Integration targets

Delivery into a folder is the least useful outcome available. The point of indexing is that a system can consume it, and that system is normally already chosen before we arrive. We build the output to fit what you run.

• SharePoint and Microsoft 365 environments, with index values mapped to site columns and content types rather than dumped as filenames
• ECM and EDMS platforms, using their own import format so that metadata, version history and permissions land correctly
• Cloud object storage, with a manifest and a folder convention agreed in advance so the structure is predictable and scriptable
• ERP, finance and line-of-business systems, which want validated structured data rather than images
• Case, patient and student record systems, where the match key back to an existing record matters more than the image itself
• A plain, well-structured file and folder delivery with an index file, for organisations that have not yet chosen a repository

We do not resell any of these platforms and we do not claim a partnership with their vendors. Integration means we produce output in a shape the target system accepts, and we test that ingestion on a sample before the bulk of the archive is processed.

How the stack gets chosen

In the order that keeps the cost down. A physical survey of the material comes first, then the retention and classification position, then the retrieval requirement, then the target system, and only then the equipment and recognition approach. A sample batch runs end to end through the proposed configuration, you look at the result, and the settings are fixed before volume work starts.