Last weekend our hosting company moved the physical location of their servers — from San Antonio to Chicago. There were a number of very serious issues that appeared post-migration caused by the hosting company. I will address the issues and the responsiveness from the hosting company in another post once all of the post-mortem details have been provided to me. But what the downtime and issues made me quickly realize is I need to have a current, well-documented disaster plan for both the blogs and for my startup.
With a WordPress blog, there is the website (css, html, images, video, etc.) and then there’s the database. I have backups of both. Luckily there was no downtime passed the migration period for the blogs. As long as we keep current backups of both content and database, it would be relatively easy to create a new server installation, either at the same host or a different host and copy everything to the new location. Then it should be as simple as switching the DNS and waiting for the internet pipes to switch the tracks to the new location.
Obviously every startup and every company will have a different disaster plan. We backup our database several times a day so we can always revert back to a backup should the need arise. And we test the backups on a regular basis using a mirrored database. When my startup was down due to the hosting issues, I knew I needed to quickly be able to communicate to both current customers who would want to login to the system and to potential customers looking to place an order. I updated the affected web pages with a message explaining the situation and offering a way for customers to contact us, both by phone and by email. I also prepared an email blast but since the server was up and down, I decided not to send the email. I did monitor the social networks and responded to any queries or questions from customers.
Now that the site is back, I have created a duplicate of our public website as a “disaster version”. This will allow for quick deployment in the event that another round of downtime appears in the future. With a click I can move the disaster version into production and be assured that all of the right updates have been made. Once the issue is resolved, I can revert back to the pre-disaster website and continue business as usual. It’s important to remember that both the regular and disaster versions must be kept in sync.
I will also be looking into additional options around creating a complete mirror of both the website and database just in case further downtime arises. Once this “live” mirror is setup, should we experience any significant downtime, I can first activate the above disaster version and then just switch the DNS to the live mirror and there should very little impact.
I’d love feedback here about your disaster plans for your startups along with general tips and tricks. Leave a comment and I will add the feedback to the post.