CCIE

You are currently browsing articles tagged CCIE.

So I’ve been doing a lot of CCIE study labbing lately and it’s become rather apparent that the only way to verify IPv4 unicast connectivity is to use TCL scripts on the routers.  Its really pretty easy.  I generally crank the scripts out in notepad and then paste them into the router.  One might look like this…

foreach address {
172.90.134.1
172.90.134.3
172.90.107.1 } {ping $address}

Then all you need to do is enter the TCL shell by typing ‘tclsh’ at the enable prompt.  Then just paste your script in…

image

Exit drops your back out of the TCL shell and back to the normal enable prompt.  I initially found that I was having issues with the scripts because of my lack of spaces.  You NEED a space after the first line variable (address) and one in-between the two brackets separating the last IP and the ping command (…107.7} {ping…). 

While this appears to work well on the routers, the switches (at least the ones for the v4 lab) don’t appear to have the TCL shell.  Must find another solution for those… Edit – Apparently you use macros on the switch, see example below…

image

Tags:

BGP Route Reflectors

One of the main points to remember about iBGP relationships is that iBGP assumes that all of the routers within a single AS are fully meshed.  That is, each router is peered to every other router within the same AS.  The purpose of this is namely to fix any issues around loop prevention.  You’ll recall the BGP uses the AS-Path attribute to prevent against loops.  A BGP speaker will drop any update it receives that has it’s own AS listed in the AS-Path attribute.  Since iBGP updates are sent within a single AS, the AS-path attribute isn’t updated and if iBGP speakers continued to forward updates, we would have routing loops.  Bottom line is that iBGP speakers will not forward iBGP learned routes to other iBGP peers. 

As you can imagine, in some designs, peering every router to every other router in the same AS could be quite the task.  One of the ways to get around this restriction is to use route reflectors.  I think the concept is best explained with an example so let’s just jump right into a topology…

image

Given this topology, and assuming that all of the routers are in AS 100, let’s peer each of the directly connected routers and see what we get…

image

image

image

image

image

As expected, each router did NOT forward any of the iBGP updates it heard to any of the other routers.  At this point, router2 has the most visibility of the network with 4 out of the 5 router loopbacks in it’s BGP table (it’s own and one for each BGP peer that it has). 

Peering each of the routers together would mean that I’d have to configure 12 more BGP sessions.  That would be a serious pain. 

This is where we can use route reflectors to save ourselves some pain and agony.  Let’s configure router2 as a route reflector to start with…

image

Once the configuration goes in you should see the sessions bounce and reset…

image

Now let’s see what we have on router3 which previously has 3 out of the 5 loopbacks…

image

We see that router3 now has all of the loopbacks.  So let’s talk about what a route reflector actually does.  You saw above the configuration isn’t all that involved.  We made router2 a route reflector just by telling it that it’s peers were route reflector clients.  By doing that, we’re telling router2 to sort of change the traditional rules of iBGP advertisements.  Essentially we tell router2 to act by the following rules…

-Advertise routes learned from route reflector clients to other route reflector clients AND non-route reflector clients.
-Routes learned from non-route reflector clients should ONLY be advertised to route reflector clients.

In the case above, we told router2 that all of it’s peers were route reflector clients.  When we did that, it used rule 1 above and advertised the loopbacks from routers 1, 3, and 5 to all of it’s peers (which happen to be routers 1, 3, and 5).  Lets’ modify the configuration a little to show rule 1 working with non-route reflector clients…

image

Looking at router5 we see that it has the same prefixes as before in it’s BGP table…

image

AKA – Things are still working the way they should since the other prefixes are being advertised by route reflector clients.  If a prefix was advertised to the route reflector (router2) from a non-route reflector client, router5 would no longer receive it since the normal rules of iBGP would apply. 

Looking at the iBGP table on router3, we can see that the route reflector is acting on rule 2 listed above and advertising routes learned from non-route reflector clients to other route-reflector clients…

image

We can tell this rule is working since router3 has the loopback for router5 (a non-route reflector client) in it’s BGP table.  Route reflectors sound hard, but if you remember the two rules listed above, they are much easier to work with and understand.

Now let’s take a look at a couple of the prefixes in the BGP table…

image

Looking at the router1 and the router2 loopbacks we note that there’s a difference in the output we are receiving.  Namely we notice that the first entry for 172.64.32.1 includes an additional line at the bottom of the output listing the ‘originator ID’ as well as the ‘Cluster list’.  These are two BGP attributes that are used by route reflectors.  Note that router3 believes that the originator for the 172.64.32.1 prefix is 10.0.0.14 and that the cluster list is 10.0.0.13.  Let’s check the diagram to make sure this makes sense…image

Any guesses on what these fields mean or why these IPs are listed?  Its rather straight forward actually.  The cluster list includes a list of all route reflectors that reflect the route on it’s way to router3.  In this case, that’s just router2.  The originator ID is the router that originated the router which in this case is router1.  Taking a look at router1 and router2 we note that the IPs that are used for the cluster list and the originator ID are the BGP router IDs…

image

image

The BGP cluster ID can be changed with the ‘bgp cluster-id’ command under the BGP process.  The Originator ID will always be the BGP router ID. 

So now you might be wondering what the cluster list and the originator ID are used for.  Much like how eBGP uses AS-Path for loop prevention, iBGP uses these two attributes to fill a similar role.  BGP routers work off the following rules when it comes to the cluster list and the originator ID…

-If a BGP router receives a BGP update from an iBGP peer and detects it’s own router-ID in the originator ID field it will reject the update
-If a BGP router acting as a router reflector receives a BGP update from an iBGP peer and detects is own cluster ID in the cluster list it will reject the update.

Pretty simple right?  Let’s configure router3 to be a route reflector so we can see the cluster list in action…

image

At this point, all of our routers will have all of the loopbacks in their BGP table.  Let’s take a look at the 172.64.32.1 prefix on router 4 and see what we have…

image

Notice how the prefix is still originated by router1, but the cluster list now shows the IP addresses for router2 and router3.  These are the two route reflectors that reflected the update on it’s way to router4. 

We can see the cluster list loop prevention in action if we reset the BGP session between rotuer2 and router5.  Before we do that, let’s take a quick look at router2 and router3 to determine how they are sending BGP updates.  A handy command to look at where a BGP router will send updates is the ‘show ip bgp update-group’ command.  Let’s look at that on routers 2 and 3…

image

image

Here we can see that router2 has two separate update groups.  This means that there are two distinct sets of routers that will receive two separate sets of updates.  If all the routers were going to receive the same updates, there would only be one update group.  Here we can see that update group 1 consists of routers 3 and 1.  Update group 2 consists of router5.  This makes sense since only routers 1 and 3 are route reflector clients to router2.  The route reflector clients will always get the same set of updates.   

Router 3 has only one update group since all of it’s peers are route reflector clients.  Notice that both router2 and router3 list each other as members of the update group.  If we turn on BGP update debugging and clear the session on router5, we’ll eventually see this debug message on router2…

*Mar  3 10:19:08.011: BGP(0): 10.0.0.10 rcv UPDATE w/ attr: nexthop 10.0.0.6, origin i, localpref 100, metric 0, originator 172.64.32.5, clusterlist 10.0.0.17 0.0.0.0, path , community , extended community , SSA attribute
*Mar  3 10:19:08.011: BGP(0): 10.0.0.10 rcv UPDATE about 172.64.32.5/32 — DENIED due to: CLUSTERLIST contains our own cluster ID;

This is because router2 is reflecting the route to router3.  Router3 is then sending the route right back to router2. 

We can also see an example of the originator ID loop prevention mechanism by clearing the BGP session between router2 and router3…

*Mar  3 11:18:02.123: BGP(0): 10.0.0.10 rcv UPDATE w/ attr: nexthop 10.0.0.10, origin i, localpref 100, metric 0, originator 172.64.32.2, clusterlist 172.64.32.3, path , community , extended community , SSA attribute
*Mar  3 11:18:02.123: BGP(0): 10.0.0.10 rcv UPDATE about 172.64.32.2/32 — DENIED due to: ORIGINATOR is us;

Here we see that router3 is reflecting the router originated by router2 (172.64.32.2/32) back to router2.  Router2 sees it’s own originator ID in the update and drops it. 

There you have it.  A quick look at route reflectors in action!

Tags: ,

It’s a well known fact that eBGP peers need to be (by default) directly connected.  That is, the BGP packets generated by a BGP speaker have a TTL of one.  When a BGP peer receives the packet, it decrements the TTL on ingress and process the packet normally.  If the BGP peer is more than one layer 3 hop away, the first router to receive the packet from the BGP speaker will decrement the TTL of the packet.  So in the case of a BGP peer multiple hops away, the eBGP packet sent by the BGP speaker will have a TTL of one.  The next router in the path to the peer will decrement the TTL to 0, realize it can’t forward the packet, drop it, and generate an ICMP TTL expired in transit back to router1.  Let’s examine this behavior by using this lab topology…

image

Let’s assume here that we want to setup a eBGP peering between router1 and router4.  To make matters a little more interesting, let’s also use the loopback interfaces on each router as the update-source for the BGP session.  I’ll be using OSPF as the IGP to get reachability between all of the networks.  The BGP config on router1 and router4 will then look like this…

Router1
image

Router4
image

Taking a look at the BGP summary, we see that both routers are in the IDLE mode…

image

image

Let’s take a quick look at the BGP packets being sent from router1 and examine their TTL…

image

A capture on the router1’s interface facing router2 curiously doesn’t show any BGP traffic at all.  This is rather strange.  I would have expected to see BGP packets being sent towards 172.64.32.2 with the default TTL of 1.  However, it appears that this isn’t occurring.  A quick look at the ‘show ip bgp neighbors’ commands yields this output…

image

Interesting.  So the router knows that the neighbor isn’t directly connected.  Apparently this means that the router knows it’s hop count is 1 and that since it isn’t directly connected it doesn’t even try to establish a connection.  We can tell the router to still attempt the the connection by issuing this command under the BGP configuration…

image

Now if we take a look at our capture, we should see some BGP packets…

image

If you look at the above output closely we can see that router1 is attempting to establish a TCP connection on port 179 with router4.  We can also see that router2 (10.0.0.13) is replying back to router1 informing it that the TTL has expired in transit.  An up close examination of the TCP packet shows the TTL of the packet to be 1 which is what we’d expect to see…

image

Let’s go ahead and re-enable the directly connected neighbor check on router1 and instead change the eBGP multihop setting…

image

A second look at the capture shows the TCP packets being generated with a TTL of a 2 and the ICMP TTL expired in transit packets being generated by router3 (10.0.0.10)…

image

image

Let’s take a look at the diagram again to make sure we understand where we’re at…

image

We are now telling router1 to attempt it’s connection to router4 so long as it is less than or equal to 2 hops away.  We saw the TCP SYN leave router1 destined for router4 with a TTL of 2.  Router2 receives the packet, processes it, decrements the TTL by 1, and sends it on its way to router3 with a TTL of 1.  Router 3 process the packet and realizes that it cant send a packet with a TTL of 0 (after it decrements it), and replies with a ICMP TTL expired in transit. 

Side Note:  This is probably a good time to discuss how Cisco routers handle TTL in general.  From my experience, Cisco network devices process TTL in this manner.  If the packet is destined for the router (control plane etc.) a packet coming across the wire to the device with a TTL of 1 is fine.  The router will decrement the TTL of the packet on interface ingress, and then pass the packet up to the control plane.  This implies that a TTL of 0 is fine on a local device EVEN if you are headed to a loopback interface on the router.  However, if the packet is not destined for the router the router will decrement the TTL, do a forwarding lookup, realize the packet is not destined for itself, and will then drop the packet and generate a ICMP TTL expired message back to the sender.

Now let’s change the eBGP multihop setting to 3 on router1 and see what happens…

image 

We can see that router4 is sending a TCP reset back to router1…

image

Much like how router1 didn’t want to play ball when it knew it’s neighbor was directly connected, router4 is in the same boat.  It’s completely refusing the TCP session.  If we were to enable the ‘disable-connected-check’ on router4 we’d see that router4’s replies to router1 would be dropped on router3 since they only have a TTL of 1.  Let’s confirm that theory…

image

Just as we thought, lets look at the packets in more detail…

Here’s router1 reaching router4.  By this time the TTL of the packet is 1…
image

Here’s the reply from router4 back to router1.  Notice that the TTL of this reply is also 1…
image

And finally, here’s router3 telling router4 that the TTL expired in transit…
image

So now let’s turn the eBGP multihop on router4 to three and see what happens…

image

image

And the session is now up!  Now let’s look at some quick verification commands to determine how eBGP multihop is configured…

image

The allowed hop count is recorded in the ‘show ip bgp neighbor’ command.  Note that when you are using the default hop count of 1, this command won’t display anything.  The full output of the ‘show ip bgp neighbor’ command will show whether or not the neighbor is directly connected.

So now that we beat the snot out of eBGP multihop, let’s talk about the other option which is TTL-Security.  When I first read about TTL-Security, I was rather confused.  Not by the concept of it, but why I would want to use it rather than eBGP multihop.  It should be noted here that the two features are mutually exclusive.  AKA, you can only use one of them at a time.  So let’s start with the definition of TTL-Security from Cisco…

This feature protects the eBGP peering session by comparing the value in the TTL field of received IP packets against a hop count that is configured locally for each eBGP peering session. If the value in the TTL field of the incoming IP packet is greater than or equal to the locally configured value, the IP packet
is accepted and processed normally. If the TTL value in the IP packet is less than the locally configured value, the packet is silently discarded and no ICMP message is generated. This is designed behavior; a response to a forged packet is unnecessary

Sounds quite a bit like eBGP multihop in some ways doesn’t it?  Well, the key to understanding TTL-Security (at least for me) was these two sentences…

eBGP multihop configures the maximum number of hops in which a eBGP speaker can use to reach a eBGP peer.  TTL-Security assumes the default TTL of 255 is being used and ensures that the TTL of the received packet is greater than or equal to the minimum TLL (255 minus configured hop count).

Confused?  Let’s run through a quick example so you can see what I mean.  I’m going to remove the eBGP multihop command from router1 and router4 and reconfigure them with the TTL-Security command…

image

image

So let’s check out what we see on the wire…

Here we can see the packet from router4 headed to router1.  Check out the TTL of the packet…

image

Now check out the TTL of the packet coming from router1 headed to router4 as it is seen on the wire between router3 and router4…

image

As you can see, the initial packets being generated by each router are starting at 255.  The packet sent by router1 reaches a TTL of 253 by the time it hits the wire between router3 and router4.  So what’s going on here?  The TTL is more than sufficient for the TCP packets to arrive at each router.  Why isn’t the session coming up?

TTL-Security takes the number you supply in the ‘ttl-security hop’ command and uses it to determine a minimum incoming TTL.  The maximum TTL is assumed on Cisco devices to be 255.  That being said, specifying a TTL-security value of 2 means my TTL minimum is 253 and the TTL maximum is 255.  When router 1 sends the packet to router 4 we saw the packet leaving router3 headed to router4 with a TTL of 253. 

So if the value of the incoming packet was 253 why doesn’t the connection work when the minimum TTL is 253?  The bottom line is that Cisco router decrement the TTL of a packet on ingress to an interface (see side note above).  The instant that packet is received by the router, the TTL is decremented before it can reach the control plane. 

Making sense?  Let’s have some picture to clarify the point…

image

As we can see above, the TTL starts off with 255 and then decrements.  The red lines show the TTL ‘on the wire’.  That is, the TTL generated by the router that pushes the frame onto that segment.  So while the TTL of the packet between router3 and router4 is 253, router4 still has to do it’s job and decrement the TTL to 252.  The general concept is the same between eBGP multihop and TTL-Security, but the purpose of each is rather different.  Take these two examples for instance…

image

Consider we have a rogue router that’s many hops away.  The rogue router and router1 both appear to be within the correct distance for router4 to accept the session.  Granted, router4’s reply to the rogue router won’t work since he likely has en eBGP multihop setting of 3 (like router1 in this example) but router4 will still accept the initial TCP SYN for the BGP session.  If we change this to TTL-Security we get something that looks like this…

image

Considering we had TTL-Security configured for 3 hops on both router1 and router4, the rogue router wouldn’t be able to create a session to router4 because it’s arriving TTL is not in between the minimum(252) and maximum(255) range.  However, the connection from router1 would work just fine since it’s arriving TTL at router4 is 253 (252 once router4 processes it).  Once configured we’d see something like this on router4…

image

So as you can see, there is a distinct use case for TTL-Security.  I hope this helps clear things up a bit. 

Tags: ,

« Older entries