Data Migration Strategy – Planning & Executing

Data migration is never been easier, unless it’s planned well and the right migration strategy is used. Data migration typically accounts for more than 60% of the total hours in a database development project and requires a lot of engagement and input. Data migration is different from database migration and involves translating data itself from one platform or device to another. A good Data migration strategy should take into account the tasks that needs to be performed in Pre-migration, Migration and Post-migration

Pre-migration

The pre-migration phase includes all the tasks which are to be carried out before the actual data migration takes place

Understanding the Data

Selecting the right data migration strategy starts with understanding the data. Before migrating data, data models needs to be analyzed.  If the database vendor are same like Oracle to Oracle, then copying total database schema would suffice the requirement. If it is from one DB to the other like Oracle to MySQL then we need to take care of data type and database specific issues.

Setting the expectation with Business:

Data migration is often done to introduce a new system or upgrade the current system. This may require the business to adapt their operational process or create additional processes to support the system.  It is important to set the expectation with the business about the changes, downtime and the new process that needs to be followed after data migration. The business managers who knows the data of the system should be well informed about the data migration process. 

Business planning should also include task owner chart where each task assigned to the owner, who will be present on the day of migration and takes predefined actions in case something goes wrong.

Phasing

Phasing is a good idea if the data involved in migration is huge or affects multiple systems. Understanding how the data is put together can help in migrating data in phases. Phasing will also help in data cleansing and integration issues, where the Data expert can focus on particular area and resolve issues quickly.

Data Validation

Data validation can be viewed as an extension of “Understanding Data” stage. Source data validation may be required in cases where the target system has specific mandatory attributes. Some cases may also require one to one detailed mapping or user defined source data manipulation.

Data Cleansing

Poor data quality in a legacy system can affect the data migration in numerous ways. Inaccurate data can be both costly and time consuming. Data cleansing would ensure that the source data is in fit to be sent to the target system.

Testing

Perhaps the most important task of the pre-migration process is testing. Everything and anything that might go wrong should be tested and necessary back up plan should be made. Testing should be carried out with different data sets repeatedly until the process achieves the target system quality check criteria consistently.

Migration

The migration process should ensure that the data is transferred from source to target system seamlessly. As mentioned earlier, the task owners should be present during the migration. Migration downtime should allow unexpected delays and source systems should be frozen before the migration begins. Production run and communication of status are two important phases of migration.

Post Migration

It’s the final and most important phase of data migration. Depending on the quality checks at the Target system, the source system can be on standby or run parallel. Cleanup of the temporary system process and ensuring the final data is complete is important aspect of the post migration process. Fallout assessment must be done and any manual intervention should be carried out while keeping the business users in the information loop.