Why being prepared for disasters means more than just routine backup

Zach Schuler, founder and CEO, Cal Net Technology Group
Zach Schuler, founder and CEO, Cal Net Technology Group

If your business isn’t completely dependent on technology, then you are in the minority these days. Given this dependence, protecting your business from an IT failure should top your priority list.
“Having been in the IT business now for 16 years, I’ve seen my fair share of close calls and, unfortunately, my fair share of outright disasters when it comes to IT,” says Zack Schuler, founder and CEO of Cal Net Technology Group. “There are three particular disasters that stick out in my mind. In each of these three cases the companies were taking nightly backups of their data, and they thought this was enough.”
Smart Business spoke to Schuler about how businesses can avoid these kinds of mistakes.
What are some of the worst disasters you’ve encountered?
The first case was a company that had a sprinkler break right above its servers. While it was taking a daily backup, the company left the tapes on top of its servers. The tapes were drenched and basically unusable after the downpour. The server hard drives were sent to data recovery, and after several days the company was up and running again. Had the tapes been taken off site, the downtime would have been significantly less.
The second case was a company that had its building burn down. Its current tapes were stored on site; however, the company had an older set that was taken off site. After a painful data reconstruction process, and several months later, the company was able to get back on its feet.
The last case was a company that experienced an Internet outage for a week when a major telephone company had its T1 down. This was the company’s only connection to the Internet, and its business was highly dependent on email, so this outage had a significant impact on its business. The company lost a percentage of its revenue as a result of the outage.
Needless to say, none of the above companies were prepared for the type of disaster that they suffered, yet all of them were backing up their data.
How can businesses avoid costly downtime?
Here are three important questions that you can pose to whoever manages your IT, and some tips that will get you one step closer to being truly prepared in case of emergency.
1. What is your plan in case of a lengthy Internet failure? The smart thing to do is to make sure that you have multiple connections to the Internet, over different mediums. Having a connection via a T1 and a DSL line is not a smart move, as they both traverse over the strands of wire. An Internet connection through a telephone company and another through a cable provider is the way to go.
2. What is your plan in case of a physical site failure, such as a fire, earthquake, etc.? Something as simple as a long-term power outage in your building can be a lot more common than one would think. On more than one occasion we’ve seen a building lose power for several days, and companies basically send their employees home. We had a client that was prepared in this scenario. It sent its employees to work from home, as it had a hot-site set up that employees were able to connect to from home.
3. What is your plan in the event of a major hardware failure? Even if your equipment is under warranty, if a particular part fails on a server, and the vendor is out of stock on that part, you could see some downtime. In this scenario, you should have a transition plan documented whereby you can easily move the data from one server’s backup over to another server, perhaps in a virtualized environment, to keep running.
What is the most common issue you’ve encountered with companies’ backup plans?
Perhaps the biggest overall error that I’ve seen companies make is that they don’t have any documented plan in place to recover from any of the above scenarios. Most companies simply don’t test their backups by going through a simulated failure. They assume that the backup is running as they’ve been told. The smartest action that you can take is to go through a simulated failure. Pretend that any of the above scenarios has happened, and try to recover from them. We assist IT departments with this type of work frequently, and we’ve never walked into a disaster recovery test whereby we didn’t make a tweak of some sort to make the plan better, thus more recoverable.
Zack Schuler is founder and CEO of Cal Net Technology Group. Reach him at [email protected].
Insights Technology is brought to you by Cal Net Technology Group