Recognition tells you what characters are on a page. Extraction tells you what a field says. Neither answers the question an operations team actually has, which is: what is this document, does it belong to us, is it complete, and who should be dealing with it? Intelligent document processing in the UAE is bought to answer that question at volume, on batches that arrive mixed, unsorted and occasionally in the wrong order.
The stack, in the order it runs
- 1
Ingest
Documents arrive from scanners, monitored email inboxes, portal uploads or system folders. Everything is stamped with an intake reference at this point, because the audit trail has to start before any decision is made about the content.
- 2
Separate
A scan batch is a continuous stream of images. The system decides where one document ends and the next begins, using barcode separators, blank pages, page counts or layout change. Get this wrong and every downstream step inherits the error.
- 3
Classify
Each separated document is assigned a type: invoice, delivery note, passport copy, tenancy contract. Classification uses layout signatures, keyword patterns and the recognised text together, because layout alone breaks the moment a supplier redesigns their stationery.
- 4
Extract
Fields are pulled according to the rules for that document type. Structured forms use fixed field positions. Semi-structured documents such as invoices are located by their labels and their spatial relationships, which is why an invoice from a new supplier can be processed without anyone building a template for it first.
- 5
Validate
The extracted record is tested. Do the line items sum to the total? Is the tax registration number in the expected format? Does this supplier exist in your master data? Is this invoice number already in the system? Validation is where most genuine errors are caught, not extraction.
- 6
Route
Clean records go straight to the destination system. Anything that failed classification, extraction or validation goes to the review queue with the reason attached, and the document image beside it.
- 7
Release and learn
Approved data is exported to the EDMS, ERP or workflow platform. Corrections made in review are collected and fed into the next model review cycle, so the same failure does not keep recurring.
Classification is the step people underestimate
Extraction gets the attention, but a misclassified document is a worse outcome than a badly extracted one. A wrongly extracted invoice total fails a validation rule and lands in review. An invoice classified as a delivery note goes quietly into the wrong process, where it may sit until somebody chases the payment.
Classification therefore gets its own confidence threshold and its own exception path. If the system is not sure whether something is a contract variation or a new contract, the right answer is to ask, not to guess and continue.
Straight-through processing, described honestly
Straight-through processing is the proportion of documents that go from intake to release without a person touching them. It is the number everybody wants quoted up front, and it is the number nobody can honestly quote before seeing your documents. It varies by document class, by supplier, by how consistent your inbound paper is, and by how strict you set your own validation rules.
What can be said in advance is what moves it. Consistent inbound formats raise it. A long tail of small suppliers each using their own invoice layout lowers it. Strict validation lowers the straight-through rate and raises data quality, which is usually the right trade. Loose rules do the opposite and produce a flattering statistic. We report the rate per document class, from your own volumes, so that improving it means something specific rather than nudging an average.
Exception handling is a design decision
Every IDP implementation produces exceptions. The ones that fail do so because nobody designed what happens next, and the queue silently grows into a backlog worse than the manual process it replaced. Exceptions need owners, routes and a target for how long they may sit.
| Exception | Usual cause | Who resolves it |
|---|---|---|
| Unrecognised document type | A format the classifier has not seen, or a new supplier template | Capture team, who confirm the type and add the example to the training set |
| Low-confidence field | Poor scan quality, handwriting, or an unusual label wording | Verification operator, working from the cropped field image |
| Failed business rule | Totals that do not reconcile, a date outside the permitted range, a missing mandatory field | The business owner of that process, not the capture team |
| Unmatched reference | A supplier, contract or customer that does not exist in master data | Whoever maintains that master data, with the document attached to the request |
| Suspected duplicate | The same invoice number and supplier already released | Finance, before payment, which is the entire point of catching it here |
| Illegible original | Damaged, faded or partially missing paper | Back to source: the document is requested again rather than guessed at |
Where the data goes
Extraction that ends in a spreadsheet has moved the work rather than removed it. Released records are written into the system that owns the process: an EDMS or SharePoint repository for the document and its index, an ERP for the transaction, a workflow platform for the approval. The document image travels with the data, so an approver can see the original without leaving their screen.
Because the processing history is retained at document level, you can answer the audit question that matters later: what was extracted, what did the rules say, who changed it, and when it was released.
Frequently asked questions
What is intelligent document processing?
IDP is the layer that identifies a document, extracts its key data, checks that data against business rules, and routes the result onward. It uses OCR and ICR underneath for reading, but its unit of work is the whole document and the decision about what happens to it next.
How is IDP different from OCR?
OCR converts an image of text into characters and stops there. IDP decides what the document is, which fields matter for that document type, whether the extracted values are plausible, and where the document should go. OCR is a component inside IDP rather than an alternative to it.
What straight-through processing rate should we expect?
It depends on your document mix, the consistency of your inbound formats and how strict your validation rules are, so quoting a figure before seeing your documents would be guesswork. We measure it per document class during a pilot on your own volumes and report it that way rather than as one site-wide average.
Why do we still need people if the process is automated?
Because some documents will always be damaged, ambiguous or genuinely new, and a system with no review queue has nowhere to put them except through. Review shrinks from keying every document to handling only flagged ones, with the reason and the page image supplied.
Can IDP handle invoices from suppliers we have never seen?
Yes. Semi-structured extraction locates fields by their labels and spatial relationships rather than by fixed coordinates, so a new supplier layout does not require a template to be built first. New formats do tend to produce more low-confidence fields until corrections from review feed back into the model.
Does IDP replace our document management system?
No. IDP prepares documents and data; the EDMS, ERP or workflow platform stores them and runs the process. Released records and their images are written into whichever of those systems owns the process, along with the index values that make the document findable afterwards.
What audit trail does the process produce?
Each document carries an intake reference from the moment it arrives, then a record of its classification, extracted values, rule results, any operator correction, and the release event with timestamps. That history is what allows a later audit question about a single document to be answered specifically.
Related reading
- ICR and intelligent data captureThe field-level capture and verification that feeds the IDP pipeline.
- Document indexing and metadataExtracted values are only useful if they land in the right index fields.
- Compliance and audit trailsHow processing history is retained for later inspection.
- OCR and text conversionThe recognition layer everything above it depends on.
- Athena Global Technologies, our parent companyWorkflow automation, RPA and the wider enterprise automation practice.