“Can you run that migration test again?” … said 5,000 times by the QA team
I wish I had thought of this when designing our IBM i AS/400 data migration process.
The number one thing I didn’t give enough thought to was the many repeated test runs of the data migration that would be necessary. Smaller shops may not have so many test runs, but large shops almost certainly will. In our last project we had hundreds and hundreds of test runs.
Sometimes we reran the test migration in whole, sometimes in part. (remember that!)
But here’s the thing: many, many times we were asked, what were the input and output data in test run X, which may have taken place two or more weeks ago, and what was the logic that was used in that run?
“Why did we get this? Why didn’t we get that? Are you sure you got all the data? Did you convert things that weren’t supposed to be converted? What was the logic to select records? …to extract this field?…”
Unless you’ve really organized your test data libraries, and your migration source code in a very careful and consistent way, it can be time consuming or even impossible to answer those questions.
In our case we got this about 80% right. But that 20% gap cost us a lot of time and injected uncertainty.
Lessons learned: think through how you will store and track repeated test migrations. You will need some kind of clear versioning and naming methodology, probably mostly of libraries (with those dinky ten byte names.) It’s worth getting this right from the beginning.
Just as important, which we did get right, is how you break down the process. I will write about that in an upcoming post.
Anyone have experience with a good, comprehensive naming system for migration data versioning libraries they’d like to share?