Network Migrations – Or Plan B

May 6th, 2010

Thanks for visiting the AMK blog. This is where we like to talk about best practices in the I.T. industry. We are an I.T. Company based in Ottawa, Ontario, Canada. AMK designs and builds next generation secure, scalable, survivable networks: providing our customers with the building blocks necessary to ensure secure access to their information.

The last few months we seen a trend, and been busy, assisting clients with their network migrations; or Plan B as we have come to call the phenomenon. With the continued decline of Nortel, now Avaya, and HP acquiring 3COM and H3C clients are faced with the decision of what to do next. Add to that an aging IT Infrastructure that may have issues regarding ongoing support due to end of life; clients sometimes find themselves struggling with how or with whom they should migrate their infrastructure. Finally, whatever vendor is selected there are the issues of  in-house skill sets, technology differences, and interoperability of features. So how do you get from A to Plan B? Well, here are six suggestions for you to consider.

  1. PLAN-PLAN-PLAN – A page taken from the real estate mantra. We often tell our clients that one hour of planning saves 4 hours of work. When you are migrating from one technology vendor to another this is critical to success. This includes how to connect the new vendor to the old, what services are affected and how they are affected, and then a back out plan if the migration has unforeseen issues.
  2. Identify the Features – this may seem trivial, after all aren’t all features the same? Please do not say “Yes” to that.  Everything from VLANs, IP Addressing, Routing,VRRP, DHCP, the use of Spanning Tree, Link aggregation,  to even port mapping. Each vendor has a slightly different approach to each of these features. We have had clients that had Nortel and Split Multi-Link Trunking (SMLT) in the core of their network and need to migrate to HP ProCurve. Well SMLT is proprietary and so is HP’s Meshing feature. Furthermore, Meshing and routing cannot exist on the same switch ( at the time of writing this blog).
  3. Test everything – Even on a small scale it is critical to build a small lab and test the features and configurations for interoperability. Pretty much every vendor’s trunking feature is interoperable with each other. However, it’s when there is a link failure that anomalies occur. A seamless fail over ends up being seconds; even up to 30 seconds. If your network has Voice over IP what happens? Your calls are dropped.
  4. Documentation is King – Attempting a migration without thorough documentation is like the Toronto Maple Leafs’ chances of winning the Ryder Cup; it’s not going to happen. Think about that one for a minute.
  5. Involve the other groups – Working with the other groups in your organization to further identify issues will increase the success of your migration. From servers to printers, to the secure perimeter. With servers being able to be physical or virtual, or having server side network teaming, it is important to identify these issues and incorporate them into the design.
  6. Do not rush to migrate – Missing any of these steps or shortening them increases the risk of failure and ultimately delays in the migration. We had a client that made assumptions that all features were the same and built the configuration during the migration only to find out that the new vendor disabled Spanning Tree by default and VRRP did not work exactly the same way as the old vendor. Then the way VLANs were configured was different causing devices to be on the wrong network. Finally, add that the syntax was similar yet different; learning on the fly is never a good idea.

We discovered that clients were having serious issues with their migrations that we decided to develop a specialized service to wrap around migrations to help our clients. We had one client that spent a year on their migration, with our Migration Service we were able to completely convert the network within 3 months. How is that possible? Well, this is what we do for a living. We have had the opportunity to see how clients have done it wrong and how they have done it right. We have over 100 years of telecommunications/IT Infrastructure experience; and we have documented a process that works every time. Can you do this without us? Sure you can, we recommend that you consider the above, take your time, be thorough, and be prepared. If you need our help, then we are more than happy to give it to you.

Thank you for your time. If you have comments or care to share your experiences with the AMK community feel free. Next time we will discuss the importance of grounding in your network. Something I think we take for granted until it causes a problem.

If you have suggestions, questions, comments, or challenges please send a note at blog AT amknetworking DOT com. If you think we can assist your company or organization, please Request A Consultation with us.

Tags: , , , , , , ,
Posted in Best Practices in I.T., Thoughts | 2 Comments »

Avoiding VoIP Disasters

November 20th, 2009

Thanks for visiting the AMK blog. This is where we like to talk about best practices in the I.T. industry. We are an I.T. Company based in Ottawa, Ontario, Canada. AMK designs and builds next generation secure, scalable, survivable networks: providing our customers with the building blocks necessary to ensure secure access to their information.

This time we are going to discuss some steps to avoid some very serious Voice over IP(voIP) disasters. When clients engage us after their VoIP deployment ended with poor results we hear some similar complaints. “Everything was fine before we put VoIP on the network”; “We have Gigabit in the core and 100 megabits to the desktop so we know the network is fine”; “We were under pressure to deliver VoIP so we just put it in the network”; “We had the network certified for VoIP 2 years ago so we thought it was fine for VoIP now”. This is almost always followed with, “It must be the VoIP equipment because everything was fine before we put it in the network!!” In every instance the root cause was not the VoIP equipment, rather it was the IT infrastructure. The problem was always there, it was just not visible until a real time application was put on the network. So what were the real problems? Here is a list of our top 5:

  1. Spanning Tree – This is one of the least favorite protocols on a network, but it does serve a purpose. It is not so much Spanning Tree itself; it is how it is implemented. Spanning Tree ensures that there is one active path between any two devices in a network – which is great. The problem occurs when there is a topology change that affects the network infrastructure. The convergence of the network can take from 30 seconds to 2 minutes. It is worse when Spanning Tree is disabled and loops are created. Now there are multiple active paths which create broadcast storms which will consume all the bandwidth in the network. Another problem occurs when multiple spanning tree groups exist and there is a connection between groups. Again a loop is created and the network experiences various issues. One client had experienced over 100,000 topology changes in a month!! When they added VoIP on the network failed! It wasn’t Spanning Tree itself but how it was employed.
  2. Quality of Service – No matter what, Quality of Service (QoS) must always be used in a VoIP network must be designed well; especially in a multi-vendor environment. This is so critical for success. Note that each vendor may apply different values and queues for Voice traffic. We had a client that had two different vendors and accepted the default values for QoS and queuing. The result was one vendor’s VoIP traffic QoS values were the other vendor’s routing traffic QoS values. The users’ experiences intermittent call drops.
  3. VLAN contamination – This is a term that we coined to describe when one VLAN is directly connected to another VLAN through VLAN unaware ports. Imagine a VLAN that has a high level of broadcasts or multicast connected to a VoIP VLAN. Two things happen. One is the bandwidth is consumed and the second is that each phone or telephony device must process the broadcast or multicast to then discard the packet. The users’ experiences jitter and call drops. Iif QoS is employed now the broadcast/multicast traffic is sent through the network with a higher level of service.
  4. Port mismatching – Autonegotiation is a great feature, it has the ability to identify the data rate (I do not like to use the word speed as that is a misnomer) on the receive pair and negotiate the best data rate to use; whether it is 10/100 Megabits or 1 Gigabit Ethernet. Autonegotiation cannot determine the duplex of the connection. If both ends of a connection are not configured for the same operation – hard coded or autonegotiation – the outcome is a low performance connection. Some devices require a hard coded configuration. One client connected a signaling server which was hard coded to a switch port configured for autonegotiation. The mismatch caused calls to be dropped or call setup failure. In other cases the users’ experienced jitter and significant delay.
  5. Grounding – Yes Grounding. A client using a multivendor solution in which both vendors were adamant that the other was at fault. What we found was that there was a grounding issue on the core switch. There were 3 volts on the ground. The users experienced a wide variety of issues from static on the call; one end was loud while the other was quiet, to loud screeches then the call drops. I also include static shocks in this category as proper grounding and static wrist bands prevent static shocks. Why is this important? 50 percent of all static shocks result in immediate failure. The other 50 percent will fail over the next 5 years. This is the really important part. MOST manufacturers will null and void your warranty if failure is due to improper grounding or static shock – and they will find out.

So how can you avoid a VoIP disaster? Well, first of all do not make assumptions about anything on your network.  Second, if you have well documented processes then follow them diligently. If you do not, here are some suggestions from the process that we follow. As with everything we do this list is a living thing and should be reviewed and adjusted as needed.

  • Develop, maintain, and follow a project plan. This is the only way to measure success of your project.
  • Perform a thorough gap analysis of your IT infrastructure. It does not matter if a vendor or reseller gave it the gold stamp 2 years ago. Your network has most likely changed in that time.
  • Review QoS and the architecture
  • Design a VoIP infrastructure based on the results of the gap analysis.
  • Build and use a lab to test both the recommendations of the gap analysis and the design.
  • Make the necessary changes in the network
  • Pilot and review the VoIP technology.
  • Document everything. This includes test plans, implementation plans, configuration files, topology diagrams, everything!! This step trumps all other steps.

Well that is it for this time. I hope you have enjoyed our post. If you have suggestions, questions, comments, or challenges please send a note at blog AT amknetworking DOT com. If you think we can assist your company or organization, please Request A Consultation with us.

Tags: , , , , , , ,
Posted in Thoughts | 14 Comments »

Ottawa I.T. Best Practices

August 26th, 2009

IT Professional

Thanks for visiting the AMK blog. This is where we like to talk about best practices in the I.T. industry. We are an I.T. Company based in Ottawa, Ontario, Canada. AMK designs and builds next generation secure, scalable, survivable networks: providing our customers with the building blocks necessary to ensure secure access to their information.

The need for best practices in any group, department, or organization is mandatory to ensure consistent operation of the group, department, or organization. There are several lines of thought on the means to achieve the best practices; some common approaches are Six Sigma, International Organization for Standarization (ISO), Information Technology Infrastructure Library (ITIL), Information Technology Service Management (ITSM), Control Objectives for Information and related Technology (COBIT), and Microsoft Operations Framework (MOF). Each of these have common set of ideals and requires, in some cases, a significant financial and resource investment by an organization to achieve certification or accreditation. Unless mandated by a regulatory body it may not be necessary to achieve certification or accreditation; howbeit it is still very beneficial for an organization to develop a framework for best practices.

The key to best practices is two things: consistency and control. If tasks are executed in a consistent manner then the outcomes can be predicted and the numbers of unknowns are reduced; in other words – control. The results are a continuously operational organization; information down time or loss is reduced, risks are managed, and efficiencies are realized.

When we discuss this with clients we sometimes hear, “we don’t have time for Best Practices”, ” We cannot afford Best Practices”, or “Best Practices are too complicated for us”. Yes, we are all busy and if you review some of the practices from above it can be expensive and a level of complexity that does not match the outcome for a business. BUT, you know what? If your business is operating today, you do follow a practice of some sort. What may be missing is the documentation and training aspect; and of course the most important part is the commitment to the practice. The alternative to this is what we call Social Practice, or Practice by Hanging Around. So a new employee starts and learns from the person already doing the job. What are they really learning? All the bad habits of the person already doing the job. The veteran may know how to do their job, but they may not know how to teach someone how to do the job. Documentation, training, and dedication to the practice are the first steps to a well developed IT best practice. We have one client that when asked about the success rate of their IT projects the response was, “We are at about 25% successful”. Read that again and let that sink in! Why were they only 25% successful? They hired expensive Project Managers, and top rate consultants. They had not defined any processes nor could they measure success upon completion of an IT project. They did not want to invest in an expensive process model. So what did they do? We helped them document how they did things and trained people of the entire process. Projects are better run and they have a higher success rate. Now it is up to them to continue the commitment to ensure continued success.

So do you need to invest in an industry standard practice? Maybe if it is right for you. If not then develop your own that makes sense for your business. Then when the time is right, investing in one of those models will be easier and not as costly.

I hope you have enjoyed our first post. If you have suggestions, questions, comments, or challenges please send a note at blog AT amknetworking DOT com. If you think we can assist your company or organization, please Request A Consultation with us.

Next month we plan to discuss how to avoid VoIP disasters.

Ottawa I.T. Services, I.T. Services Ottawa, Ottawa I.T. Network Specialists, Ottawa Information Technology, Ottawa I.T. Solutions, Ottawa Network Technology

Tags: , , , , ,
Posted in Best Practices in I.T. | 1 Comment »

We Are The I.T. Infrastructure Specialists!

July 31st, 2009

Welcome to AMK Networking’s blog!

Here, we get to create dialogue about the best practices and current updates within the I.T. industry, as well as share some of our approaches to solutions and challenges associated with managing and building I.T. technology. Stay tuned for our first post!

Tags: , , , , ,
Posted in Thoughts | No Comments »