Owen Pettiford’s 8-point guide to a successful data migration
Remember the mantra drilled into you by teachers, lectures and headmistresses?
“By failing to prepare, you are preparing to fail”.
I remember a letter sent home with this in the run-up to exams. It’s enough to make anyone’s heart beat a little faster.
This mantra also acts as a word of warning for CIOs about to undertake a data migration project, which are often underestimated. So, as students go back to school this week, what lessons can CIOs learn from them to ensure a fuss-free data-migration project?
Students use the summer months to refresh and replace school uniforms, IT teams should follow suit and take the time to run data quality checks before any migrations.
For example, checking that all the mandatory forms are filled with data that makes sense, like numbers where letters should be or vice versa. Once you’ve got the rules set in place you can assess how your data ranks against this for instance if your data quality is 61 percent then it has plenty of room for improvement.
Having poor data quality, whereby you go into the database and the data isn’t there, can have serious knock-on effects on the business costing it time and money alongside leaving employees frustrated.
On the whole, we find people are most surprised at how poor quality their data really is. For example, we’ve had examples where a significant number of products don’t have the full instructions of how they can be made in the system. This might be manageable if the team is small, but it isn’t sustainable as the business grows.
During this back to school summer clean, you might also discover the data you’ve been hoarding over the years. For example, we’ve seen instances where firms haven’t deleted any previous data and of the 20,000 vendors in their databases, in the last two years, they have only used 2,000 of them. This is a perfect time to remove the data you no longer need or use.
Like teachers encourage you to plan before jumping into a big project, the same applies for data migration projects. IT teams shouldn’t underestimate how mighty data migration projects are. And the costly ramifications if it goes askew – just ask TSB.
The key things to consider at this point are, what are you going to migrate? When? In which order? How closely does the system fit your current and future business model? And how happy are the users?
It’s this first but crucial step that is often neglected and the reason why data migrations fail.
New Term, New School?
Students and parents will weigh up the pros and cons of shifting schools or staying put. Similarly, this is an important choice for CIOs to make. Do you stick with a system you know and upgrade it or take the plunge and start from scratch?
Upgrading a current system might sound easier, but it does mean you have to carry your previous decisions with you.
Take CRM systems, they usually have a way of coding the customers, and once you’ve done this in that system it’s hard to change.
Whereas going for greenfield, might seem scarier, much like having to make new friends at a new school, but it allows you to do whatever you want and the end result could be transformative.
Taking the plunge with a new system can also be de-risked with the support of technology specialists in this field – much like a buddy assigned when you start a new school. And much like an entrance exam, you might find that your system fails the upgrade test and you have to have a plan B in place.
Assigning a Data Leader
You can have the best facilities in the world, but what makes a school great is a visionary headteacher steering the ship. Similarly, for a successful data migration project, a data leader is needed. This leader will ensure someone is accountable for its success and failure and long-term vision of where the future of the company is heading is crucial.
A good data leader must have respect from the business and be a good internal negotiator. Often, it’s wrongly handed over to someone more junior, but for a successful implementation, this should be run by the CDO or equivalent to ensure the entire organisation and other senior stakeholders are on board.
Picking the Right Subjects and Strategies
Selecting the right subjects is key to paving the way to a chosen career path.
Choose unwisely and this can have big knock-on effects. Similarly, CIOs need to ensure the right data migration strategy is chosen. For instance, is it going to be a big bang migration or a trickle migration?
Big bang migrations involve completing the entire migration in small, defined chunks. It does involve system downtime while the data is extracted from the source system, processed, and loaded into the new system.
On the other hand, trickle migrations take a gradual approach to migrating data. Rather than aim to complete the whole event in a short period, a trickle migration involves running the old and new systems in tandem and migrating the data in phases. This method means no downtime that certain businesses with applications requiring 24/7 operation need. Which strategy you opt for will, of course, rely on your business needs.
Following this, you’ll also have to decide the order in which the migration will be done in – by county, by function area or a combination of both?
How to Keep Those Start of Term Good Intentions
The start of term for students means a fresh perspective and good habits. Much like after having done all the hard work for a successful migration your data will be in the best possible condition. The next step is keeping data in a good condition.
By and large, people go back to their old ways and the data decays over time.
The good news is there are many technology platforms to help with data stewardship. After the project finishes, reflect on the lessons learnt and turn it into an ongoing department. Key considerations for this stage are finding out who owns the data, whose KPIs are reflected by keeping it clean, what rules from the migration can be used for ongoing data upkeep? For instance, you might be very diligent in your CRM system and fill in all your fields, but your colleague might not be as diligent. Therefore, having a data guardian helps to enforce the rules and monitor the data’s condition.
Our best practice at this point is to divide the accountability across the organisation by whoever has the most skin in the game, from customer to product teams, and make them data advocates.
Unfortunately, few companies understand the critical relationship between a successful data migration and a successful go-live. It’s not just about data and a database. All too often data migration is misunderstood, underfunded, inappropriately resourced and poorly scoped. And for the 38% of businesses who reported they were planning to undergo a data migration in the next year, its success will be determined on the senior stakeholders taking it seriously, after all, it’s one of the top three reasons system implementations fail.
So, as we enter September, remember the words of warning from your previous teachers and don’t make the same mistake many others do by failing to prepare. The eight key takeaway steps to follow are:
1: Prepare – assign a team, set specific roles, systems and schedules
2: Extract – collect and stage source data
3: Profile – analyse and clean source data
4: Design – design and stage target system schema
5: Map – map source to the new target fields
6: Construct – manually enrich the data
7: Transform – simulate and validate loading
8: Load – load your data into production systems