Delete before you migrate: the cheapest gain in a data project

When an IT department decides to migrate its BI estate, the reflex is to count the objects to transpose and multiply by a unit cost. That calculation assumes everything deserves migrating. In practice it never does.

Why dead flows accumulate

Nobody ever deletes anything, for a rational reason: the cost of a mistaken deletion is visible and immediate — a report breaks, a director calls — whereas the cost of keeping is invisible and spread out. The individual trade-off is therefore always to keep.

The result accumulates over ten or fifteen years. Flows feed tables that feed reports nobody opens. Extracts were created for a project that ended long ago. Duplicates exist because two departments asked for the same thing six months apart.

How to identify what is dead

Do not ask the teams. When asked, out of caution they answer that everything is useful, and they are right to: they have no way to verify.

The reliable method is access log analysis over a rolling twelve months. Three sources are enough:

1. The reporting tool's logs. Which reports were opened, by how many distinct people, how often.
2. The database logs. Which tables were queried outside the loading jobs.
3. The scheduler. Which jobs are still scheduled, and which have been failing silently for months with nobody reporting it — an excellent indicator of no consumer.

A report opened zero times in twelve months, fed by a flow whose tables nobody else queries, is dead. That is not an opinion, it is an observation.

The deletion protocol that reassures

The difficulty is not technical, it is political. Nobody wants to sign off a deletion.

A three-stage protocol removes the objection:

Stage 1 — dormancy. The flow is disabled but retained, with an alert that fires if anyone tries to access the tables concerned. Duration: two to three months, including one accounting close if the domain is financial.

Stage 2 — notification. A named list of dormant flows is sent to the domain owners, with a written objection period. No reply counts as agreement — that point is what unblocks the process.

Stage 3 — archive then delete. The flow's code is archived in a dedicated repository, the tables are exported one last time, then everything is removed. The archive keeps the decision reversible for a year, which is enough to make the approach acceptable.

What it changes for the project

On an estate of a few hundred jobs, setting aside a third is not unusual. The effect is threefold: the migration cost drops in the same proportion, the nightly processing window frees up, and the new platform starts without inheriting debt it would carry for ten years.

It is also, incidentally, the only sales argument that consists of deliberately reducing the size of a project. A provider who proposes this step is telling you something about how they work.