Winds of change Featured

8:00pm EDT July 26, 2009

Once you get your company’s network up and running, you’re not done with the process. Instead, you’re just getting started. It’s not enough just to have a network in place; you have to monitor it, update it and measure changes to make sure your customers are having the best possible experience.

“You cannot deliver your services without a network that’s very available and that’s running at high quality,” says Ishmel Goodrum, vice president of network operations at Insight Communications. “You can’t keep your network running without touching it; it’s inevitable. You have to have a solid plan for making changes to it, because if you don’t, you’re going to have customers who are very unhappy with you. Your services could be great, but if they’re not available on your network due to human error or a lack of planning, it can hurt your business.”

Smart Business spoke with Goodrum about how to make changes to your network for maximum impact on your company and minimum impact on your customers.

Why does a business need to make changes to its network?

Expansion of the network and making changes to deliver more services or better services is one reason. Another reason is to make sure that the services you already offer continue to be offered with increasing quality through software and hardware upgrades, capacity increases and routine maintenance that reduces failure. You change it to keep it operating at its optimum level. The third reason is to repair network impairments.

How do you make changes to the network without disrupting your customers?

It begins with meticulous planning. You must know exactly what you’re going to change, make very detailed, documented plans for change, test those changes before you make them on a live network and brainstorm on every single potential failure scenario to plan for everything that could possibly go wrong.

Then you have to put plans in place for when things do go wrong and be able to repair or recover fast or have backout plans to put the network back to its previous state very quickly. It is also essential to determine the best time to make a change. This is most often the time of least customer use. Lastly, when deploying new products, expanding the network or doing scheduled maintenance that’s going to impact customers, you should always notify customers ahead of time not just of a disruption but also of possible disruptions. In some cases, it’s notification that something is happening that’s going to deny access to a Web site that customers rely on, for example.

How can failing to have a detailed plan hurt your business?

Mistakes made during network changes can be very costly. You can make mistakes that cost you a lot of money, and sometimes it’s hard to capture the true financial impact.

One way is to calculate downtime and the cost of customers calling your customer care center when the network is down. What’s tough to capture are the multiple occurrences of improper maintenance before one mistake is the straw that broke the camel’s back. And then that customer disconnects or transfers service over to a competitor.

Sometimes it can be arduous to capture that what really caused the customer to leave you was small outages that resulted from improper network changes and disruptions due to irresponsible maintenance. To help pinpoint that, analyze your downtime against customer complaints and see where those line up. You can draw a definite correlation between when and how you changed your network and when a customer complained or disconnect activity began to increase.

How do you evaluate whether change to your network was successful?

It’s very important to set a goal around continuous improvement and measure yourself against what you intend to do. If you intend to make X change and you expect it to have Y impact on customers and take Z amount of time, then success would mean having it go as planned.

You can also measure it in a different way. If you say, ‘I plan to interrupt services for 10,000 customers,’ and then you interrupt service for 10,000 customers plus 5 percent, you’ve failed. Or if you plan to interrupt services for an hour, and it goes an hour and 15 minutes, you didn’t meet your plan. The ability to back out or recover is also critical. If you’re in the middle of maintenance and you come across a problem, if you back out successfully, that is not considered unsuccessful. However, at that point review of the original plan is required to understand the problem and correct the process before reattempting the change.

How does a test plan come into play?

The test plan involves understanding what the customer’s experience should be and testing it from the customer’s point of view after the change.

If you’re making a change to a piece of equipment and you know that when you make that change, you’re going to see a change in the amount of traffic on the network, it’s not good enough just to verify that you have traffic back once that equipment is back online. You have to understand if the traffic you’re seeing after the change is the right type of traffic, and sometimes you can only do that by testing from the customer’s point of view.

If you don’t, you can make changes and give yourself a false sense of comfort by looking at the network that everything is OK, but in fact, you broke something you didn’t think you were going to break. You don’t see it through your network testing, but the customer sees it, so you have to test from their perspective as well.

Ishmel Goodrum is vice president of network operations at Insight Communications. Reach him at or (502) 410-7629.