Microsoft · Data archiving
Archiving D365FO data to Dataverse long term retention: what the docs say, and what I would ask in Portugal
- Who it affects
- Finance and Operations customers with large, old data volumes.
- What changes
- Old records move to history tables and a read-only copy in Dataverse long term retention.
- D365FO impact
- Test SAF-T, the invoice signature chain and retention rules on archived years before switching it on.
Microsoft has a feature called Archive with Dataverse long term retention (LTR) for Finance and Operations. I read the public documentation to understand what it does. I also wanted to know what it means in a country where records must be kept for years and shown to the tax authority.
This is a reading of the docs. It is not a how-to from a project.
What it does
It moves old, inactive data out of the live database. The data goes through a few steps:
- A copy of the data is replicated to Dataverse long term retention.
- Records that match your criteria are marked as ready to archive.
- A reconciliation step checks that every marked record exists in LTR.
- The records move to history tables in the F&O database and are deleted from the live tables.
Today it covers general ledger, tax transactions, inventory transactions, inventory journals, sales orders and Commerce transactions.
Where the data ends up
After a job, the same record can be in two places that are not live. History tables stay in the F&O database. You can open them in inquiry pages, and you can restore them to live tables. The LTR copy in Dataverse is read-only, and Microsoft says you cannot move it back to the live tables.
On storage, Microsoft says a gigabyte moved to LTR uses about 50% less capacity on average. History tables without indexes use about 30% less than live tables. To get the full saving you need to purge the history tables. Once data is purged from history tables, it can no longer be viewed from the inquiry pages, only from the LTR copy, through Microsoft Fabric.
Limits worth knowing before you start
- A job can take around 7 to 14 days, depending on data volume. Stages run in the background and one job runs at a time.
- You need a Tier-2 or higher sandbox to test. It does not work on a cloud-hosted dev box.
- Related tables are not archived for you. Archiving inventory transactions does not archive sales orders.
- X++ delete is not honored when the archive moves data out of the live tables.
- Attachments are not supported yet.
- Custom fields and tables need preparation. Microsoft provides a schema validation script, and an index named ArchiveSysVersionIdx on custom tables. Without it, reconciliation can become very slow and jobs can time out.
- After a database refresh, restore the F&O database and copy the Dataverse environment together, with no archive job running. Do not relink to a different Dataverse environment, because Microsoft says it causes data loss.
Questions I would ask for Portugal
I do not have full answers to these. They are what I would test in a sandbox before suggesting the feature to anyone.
- Retention. Portuguese tax rules require keeping records for ten years (article 52 of the VAT Code, article 123(4) of the Corporate Income Tax Code). Microsoft says auditors can view archived data through Microsoft Fabric or the history table inquiry pages. What I would test is whether that still works ten years later, after the history tables have been purged, and who keeps the access.
- SAF-T. If a SAF-T export reads from live tables, what does a full-year file look like after that year has been archived? The tax authority can ask for older periods.
- Certified invoicing. Invoices in a series are chained by signature. After archiving, can you still read and verify the chain?
- Deletion. When the retention period ends, how do you remove data from both history tables and LTR? The FAQ covers purging history tables. I did not find how the LTR copy is deleted. A GDPR erasure request does not override a legal duty to keep records (GDPR article 17(3)(b)), so this is mostly a question for the end of the retention period.
For developers
If you have custom tables or fields in a supported scenario, run the schema validation script before the first job, not after the first failure. And plan the test around the refresh rule above, because sandboxes get refreshed often.
Sources: Archive data with Dataverse, Set up and manage archive data, Archive FAQ, Archive customization.