Port-Channel Loops

So I’m one of those ‘why’ guys.  If you tell me I can’t do something I have to know why.  So when someone was telling me about a recent loop they encountered because of a port-channel configuration issue, I had to know the how, what, why and where.

As we all know, we should always configuring switch to switch port-channels with ‘mode active’.  It’s better to allow the switch to determine what links should be in the bundle rather than force it.  However, there are still many ‘mode on’ configurations left out there and I run across them once in awhile. 

I’m not going to dive into the details here on how LACP (802.3ad) works.  However, you should know that there are only 3 valid combinations of settings that work to form a port-channel.  They are…

Active – Active

Active – Passive

On – On

Active – The switch will openly try and negotiate a LACP channel
Passive – The switch will participate in a LACP channel, but won’t start the conversation
On – On.  Always on.
 
So the risk here is that one side of a port-channel could be set to on, and the other side could be set to active.  If that’s the case, the active side won’t ever negotiate the link.  So is that a problem?  Yep, it sure is.  Let’s look at an example and see what happens…
 
image

So in this case, we have switch 1 configured to start LACP negotiations.  Unfortunately, switch 2 doesn’t care.  He’s in ‘mode on’ and is going to form a port-channel regardless of what happens.  So let’s configure each side and see what happens.  Switch 2’s port-channel loads right away and shows up.  Switch 1’s port channel attempts to negotiate but fails since switch 2 doesn’t care.  At this point, the ports on Switch 1 become ‘independent’ which effectively means that they are just physical ports connected to switch 2. 

Now, let’s go back and talk about spanning-tree.  Let’s run through the two possibilities here…

Switch 1 is the root
Let’s say that switch 1 happens to be the root of our spanning tree.  After we configure the port-channels, we notice the following errors on each switch.

image

image 

So it looks like both switches shut down their interfaces that were supposed to be part of the port-channel.  You can see that switch 2 detected a channel-mis-configuration (STP) right before the ports went down.  If switch 1 is the root, it will send hello’s out all of it’s ports.  In this case spanning-tree doesn’t see a single port-channel, it sees two physical ports.  So spanning-tree sends the hello out both ports facing switch 2.  image

Switch 2 thinks that it should have received the hello over only one of the links in what it perceives to be a port-channel interface.  Since it received it over more than one, it knows that there is an issue and it put’s it’s ports in errDisable mode. At this point, the link between the switches it completely down.  

Switch 2 is the root
So now let’s see what happens when switch 2 starts as the root switch.  In this case, switch 2 forwards it’s hellos out all of it’s ports.  However, spanning-tree only see’s one port (po1) going to switch 1.  So switch 2 is forwarding hellos out of port-channel interface as well as the interface towards the workstation.  See below…

image

image

So at this point, spanning-tree doesn’t see a problem.  Switch 1 is receiving the hellos and forwarding them out all of it’s interfaces with the exception of the one it received it on.  So it would look something like this…
 image

Now that doesn’t look so bad does it?  Seems like fairly normal operation.  But that only shows the initial hellos the root bridge sends.  Take a look at what happens when host 192.168.127.2 starts sending broadcast traffic…

image

In step 1, the host sends a packet with a local broadcast destination (192.168.127.255).  Switch 2 receives the broadcast and forwards it out all of it’s remaining interfaces (PO1) in step 2.  At this point, switch 1 receives the broadcast on one of the two links that switch 2 believes to be part of it’s port-channel interface.  In step 3, switch 1 forwards it out all of the interfaces except the one that it heard the broadcast on.  This would include the link to the host 192.168.127.1 as well as the second link that switch 2 thinks is in it’s port-channel.  The end result?

image

Switch 2 is suffering from CAM table stability issues.  It’s hearing a MAC address from 192.168.127.2 out of the port facing the host (Gi1/0/1) as well as the port-channel (Po1).  Now imagine this on a large scale with more than one host.  That switch could easily become overwhelmed and tip over in such a scenario. 

Conclusion
So as you can see, it’s best to have both sides negotiate.  If you don’t you can run into some pretty serious issues. 

Tags:

  1. Dambo’s avatar

    Thanks. Helped me a lot understand our issues.

    Reply

  2. Kyllan’s avatar

    Thank you, it’s probably biting my ass at this moment…

    Reply

  3. Viktor’s avatar

    We might have had this issue a few days ago, but didn’t prevent it and entire campus network went down. All Equipment had to be rebooted manualy. Feel sorry for a guy who had to run through IDFs. My question is could have access points disassociate from WLC when port channel was being reconfigured wrong and caused access pint go back to discovery mode and start broadcast storm?

    Reply

Reply

Your email address will not be published. Required fields are marked *