Forty Enhancements and an Upgrade, With the Migration Scripts Rewritten to Run in Parallel

The upgrade from 2017x to 2021x carried a large set of data migration scripts, and nobody knew how long they would take to run. System integration testing answered the question unhelpfully: more than four weeks. That was not a performance annoyance — it set the length of every subsequent test cycle, and with forty enhancements also in flight on 2017x, it put the whole timeline at risk.

The scripts were treated as the problem to solve rather than a fixed cost. Redundant checks inside them were identified and removed. Each script was then analysed for whether it was needed at all, which cut the number materially. The remaining work was made parallel: data was partitioned into manageable chunks and the scripts modified to execute concurrently across those partitions, with code changes to make the parallel execution safe. The result was an upgrade whose test cycles were short enough to run more than once.