Plenty of digitization projects produce good images and still fail. The scanning was fine. The problem is that the output arrived in the enterprise document management platform as a pile of files with meaningless names, in folders nobody agreed, visible to people who should not see them. Integration is the discipline of preventing that, and it is decided long before the first batch is delivered.
Start from the retrieval question
Before any structure is drawn, write down the ten sentences that begin "I need to find..." as your staff would actually say them. Find the tenancy contract for unit 402. Find every invoice from this supplier in 2024. Find the signed variation for this project. Those sentences dictate the index fields, because each one has to be answerable by a field that exists and is populated. Design the structure first and you will discover months later that the one field everybody searches on was never captured.
Field mapping is where projects quietly fail
Mapping looks trivial on a whiteboard and is rarely trivial in practice. The target system has opinions: about types, about what is mandatory, about how long a value may be. Each of those opinions can reject a batch, or worse, accept it after silently truncating something.
- Type mismatches, where a reference captured as text has to enter a numeric or lookup field
- Mandatory fields in the target that simply do not exist in the source records, forcing a default value nobody agreed
- Choice and lookup fields with a fixed value list, which reject anything outside it and stop the load
- Ambiguous date formats, where day-first and month-first values sit in the same legacy export
- Arabic text and encoding, which needs checking on a real sample rather than assumed to work
- Multi-value fields flattened into one string, losing the ability to filter on them afterwards
- Character limits and forbidden characters in names and paths, which the source data has never had to respect
- Duplicate handling, where the same document exists in three departments and no unique key has been agreed
Folders, metadata, or both
Folder trees mirror how an organisation was structured on the day the tree was drawn, which is why they age badly. Metadata-driven repositories let the same document be found down several routes at once, and let a reorganisation happen without moving a single file. In practice most implementations need a shallow folder structure for orientation, with the real work done by metadata and saved views. The failure mode to avoid is deep nesting that encodes a department name into the path, because the next restructure invalidates every link anyone has saved.
Permissions have to be exported, not remembered
Access rules are the part of a migration most often rebuilt from memory, which means quietly widened. Extract the permissions from the source system rather than reconstructing them in a workshop, then map them to groups in the target rather than to named individuals. Two questions decide the design: which records are restricted at document level rather than folder level, and who is allowed to see that a restricted document exists at all. Those are different controls, and platforms handle them differently.
How the content actually gets in
The delivery mechanism is a practical choice, driven by what the platform exposes, how much content is moving and how often it will move again after go-live.
| Delivery method | Where it fits | What to watch |
|---|---|---|
| Vendor bulk import tool | One-off back-file loads into a mainstream platform that ships its own migration utility | Support for your specific metadata types varies. Test permission and version handling on a pilot batch |
| Staged file share plus manifest | Any platform, and the safest fallback when the API is limited or undocumented | The manifest becomes the contract. Agree its columns, encoding and date format in writing before the first batch |
| API or SDK integration | Ongoing day-forward capture, and any flow that has to keep running after the project ends | Rate limits, throttling, authentication token lifetimes, and error handling for the items that fail mid-run |
| Manual upload by your team | Small volumes, or a pilot where you want hands-on familiarity with the platform | Does not scale, and index consistency depends entirely on individual discipline |
Proving the load worked
A load that reports success at batch level has told you almost nothing. Verification should be specified up front and evidenced at handover.
- Item counts reconciled between what was delivered and what the platform reports as stored
- A sample opened and rendered inside the live system, not merely listed in a results file
- Index values spot-checked against the source document rather than against the manifest that produced them
- Permissions tested by signing in as a restricted user and confirming what is and is not visible
- A named rollback point, so a failed batch can be removed cleanly instead of deduplicated by hand
If your target is SharePoint
Microsoft 365 is the most common destination we deliver into in the UAE, and it has enough of its own behaviour around content types, managed metadata and permission inheritance to justify a separate treatment. If SharePoint is where this content is going, read the SharePoint migration page next rather than applying generic guidance to it.
Frequently asked questions
What does DMS integration include?
Index schema design mapped to your platform's metadata, folder and library structure, controlled value lists, permission mapping, the load mechanism itself, and verification afterwards. It also covers duplicate handling and unique keys. The aim is that a document arrives findable, correctly restricted and consistent with everything already in the system.
Should we organise documents in folders or by metadata?
Mostly metadata, with a shallow folder structure for orientation. Folder trees encode the organisation chart of the day they were drawn and break at the next restructure. Metadata lets one document be found down several routes and survives reorganisation without files moving. Avoid deep nesting that bakes department names into saved links.
How are scanning index fields mapped to our system's metadata?
Field by field, agreed in writing before capture starts. Each target field is checked for type, whether it is mandatory, its length limit and whether it uses a fixed value list. The mapping is then proved on a pilot batch, because type mismatches and rejected lookup values only surface against a real system.
Is an API integration better than a bulk import?
They solve different problems. A bulk import tool is efficient for a one-off back-file load. An API integration is what you want for day-forward capture that has to keep running after the project ends. Where the API is limited or undocumented, a staged file share with an agreed manifest is the dependable fallback.
How do we know every document actually loaded?
Reconcile item counts between delivery and what the platform reports as stored, then open a sample inside the live system rather than trusting a results file. Spot-check index values against the source documents, and test permissions by signing in as a restricted user. Agree a rollback point before production volume runs.
Can you deliver into a system you have not worked with before?
Usually yes. We work to whatever documented import interface the platform exposes, and where none exists we deliver a staged file set with an agreed manifest that your administrator or vendor can ingest. We prove the route on a pilot batch before production volume, so any platform quirk surfaces early.
What if we have not selected a document management platform yet?
Keep the output portable. Open image formats, index data delivered as a separate delimited file, no proprietary container and no naming convention that only one product understands. Capture can then proceed on its own timeline, and the same output loads into whichever platform you eventually choose.
Related reading
- SharePoint and Microsoft 365 migrationThe platform-specific detail if SharePoint is your destination.
- Paper to digital conversionWhere the content being integrated comes from in the first place.
- Secure records migrationCustody and reconciliation when content moves out of a legacy system.
- Platforms and formats we deliver intoOutput formats, index deliverables and integration options.
- Athena Global Technologies enterprise servicesThe parent company's wider EDMS, workflow and integration practice.