Importance of rollback testing in migration

Standard

Hi guys, have been busy with a project, hence the lack of posting.

I have been tasked to migrate a current system that relies on Windows server (yeah everyone hates windows) to one based on Linux. The servers received signals from a particular car device in TCP format.

The challenge here is to make sure when you point the car device from windows server to linux server, the user can’t feel it, as it is already in production. So we do have a few prototypes in place. We divert the signal from the windows to linux and modify the api to be able to read from 2 different data sources. Then we start to redirect some customers car devices from sending signal to the windows server to sending it to linux server. And lo and behold, customer complained that one of their feature is not working. So we decide to switch back. Then I realised that the way how the engineers do the reverse pointing is to send the signal back to the device via the server using TCP, identified by the ID. Oops, didn’t implement this into my new system. Lucky there is an alternative, which is to use an SMS to the device SIM card.

Hence a lesson learned here. Don’t just test the new system and migrate. If the current system is working but unable to scale and collapse after a certain load, and you have set up a new system that is tested to be able to handle that load, but not battle harden, it is a must to do rollback tests so that when something went wrong, you know you fall back to the old battle harden system before customers start ringing. Don’t just migrate. Do a tango and roll back as well to test if the system work as normal, then you start to go all out to shift one system to the other.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s