A company recently wanted to switch databases. A new vendor had already been found and implementation was just about to start when it was discovered that the vendor had assumed that the company would be responsible for managing the whole process of data conversion.

The company, on its part, had assumed that the implementation fee included data conversion and was surprised to discover that $20,000 more had to be paid to the vendor for data conversion to be done.

Data conversion is one of the most critical phases of moving to a new database. It can also be one of the costliest and may lead to unexpected expenses. Below are the key principles to bear in mind when you plan for the data conversion.

Data conversion is both labor intensive and expensive. It is thus important to convert as little as possible. Invest in expert opinions for technical things like converting string to int.

It is a great time to do some spring cleaning. For instance, demographic data that was regarded as important 3 years ago but is no longer up to date or important, this is the perfect chance to clean up your database by not converting that data to your new system.

You typically want to convert names, critical participation data, and key demographic data, but beyond that, you need to carefully consider other data before conversion.

Data should always be cleaned before conversion. Unless you have been vigilant about keeping data clean in your legacy system, chances are there are lots of duplicate data in the database.

The process of data conversion is an excellent opportunity to de-dupe data prior to converting it into your new database. It is also important to update bad physical addresses, bad email addresses, or other data first and then convert to the new system. Avoid starting with bad data in a new database or users will become suspicious of the new system immediately.

Convert open invoices only when it comes to financial records. For many systems, financial records are the hardest part of the data conversion process since there are numerous “set up” details needed.

For instance, if you have a product sale in your legacy database you wish to convert to the new database you need to set up the product, then pricing, and probably even inventory.

Unless it is an open invoice, invoice information should be converted as “static” data. For instance, instead of capturing even registration data for last year’s annual meeting in the financial module, create a demographic field or note indicating the people that were in attendance.

For small data sets, you should use manual data entry. Follow the rule of 100 or 1,000. If you have less than 100 records to convert for a specific subset of data, it is usually easier to rekey this data after setting up the new database, instead of using a script to convert the data into a new system.

Conversely, if you have over 1,000 records of a particular subset of data, it is often easier to write a conversion script as opposed to entering the data manually.

Legacy data should always be kept available. Regardless of the data you convert, it is always important to keep a copy of the legacy system available for reference, should you ever need it. However, access to the legacy system should be limited to just one or a few staff; otherwise, staff will tend to use the legacy system as a shadow database.

Data conversion is a complicated process for most database conversions. If you follow the guidelines provided here, the process won’t be so messy but rather manageable.