In our last post, we talked about one of the major differences between LDP and RSVP – the ability to define EROs or explicit route objects. We demonstrated how we could configure LSP paths through our network by providing a set of loose or strict next hops for the LSP to take. This was a rather huge paradigm shift because it meant we could define paths that didn’t align with what the IGP thought to be the best path through the network. What we didn’t talk about was how the ingress router determined if these paths were feasible. In this post, we’ll deep dive on the traffic engineering database (TED) and how it works in conjunction with the constrained shortest path first (CSPF) algorithm to build RSVP LSPs through a network.
It’s important to remember that the ingress label switching router (LSR) is really the thing doing most of the work in regards to setting up RSVP LSPs. Well – to be fair – the egress LSR is the one that actally sends the RESV message back toward the ingress LSR with the label information which is what’s required for the LSP to work. However – the ingress LSR is the one that has to do the initial calculation to figure out the path to egress. Doing so generates a set of ERO which it then includes in it’s path message toward the egress LSR. To be clear – generating ERO is something that happens whether you define it or not. For instance, if we were to remove our defined paths from the lab and revert back to simple LSP definitions using the to statement, we would still see the same process happening…
The above is from a capture taken when we simply have the following LSP configuration on vMX1…
set protocols mpls label-switched-path vmx1-vmx4 to 4.4.4.4
What we did in the last post was put more constraints on what we expected the outcome of the CSPF calculations to be by providing our own defined ERO. This is why this calculation is referred to as “constrained” shortest path first because it’s not entirely unlike any other SPF algorithm (like OSPF). The only difference is that it’s taking into consideration the constraints you define as it makes it’s SPF calculations.
If we talk about OSPF we know that it’s a link state routing protocol meaning that the routers exchange information with their neighbors. That process continues on all other neighbors until the information has been flooded everywhere. The result is you end up with every router having a rather complete picture of the entire topology. In the case of OSPF – this information is used to make decisions about routing. However – this information can also be extended to include information about other things like bandwidth reservations or metrics that are unrelated to the underlying IGP. This is actually what we did in the first post of RSVP where we used the set protocols ospf traffic-engineering command to enable OSPF to send this additional information. This other additional information can also be used for a variety of applications which we’ll cover later on. But in the end – the CSPF calculation is done in a very similar manner to how SPF algorithms run – it just so happens that you have extra information at your disposal to influence (or constrain) the calculation to give you the results you want.
SO – enough jabbering – let’s get into the weeds. I’d like to pick things up from where we left them at the end of the last post. Specifically in regards to our lab having LSPs that looked like this…
Let’s validate that’s the same path we’re still using by using our handy dandy traceroute mpls commands…
On vMX1…
[email protected]> traceroute mpls rsvp vmx1-vmx4
Probe options: retries 3, exp 7
ttl Label Protocol Address Previous Hop Probe Status
1 300336 RSVP-TE 10.1.1.1 (null) Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
2 300208 RSVP-TE 10.1.1.3 10.1.1.1 Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
3 3 RSVP-TE 10.1.1.5 10.1.1.3 Egress
FEC-Stack-Sent: RSVP
Path 1 via ge-0/0/1.0 destination 127.0.0.64
[email protected]>
Looks good – and now on vMX4…
[email protected]> traceroute mpls rsvp vmx4-vmx1
Probe options: retries 3, exp 7
ttl Label Protocol Address Previous Hop Probe Status
1 300320 RSVP-TE 10.1.1.6 (null) Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
2 300192 RSVP-TE 10.1.1.3 10.1.1.6 Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
3 3 RSVP-TE 10.1.1.8 10.1.1.3 Egress
FEC-Stack-Sent: RSVP
Path 1 via ge-0/0/2.0 destination 127.0.0.64
[email protected]>
So as expected – things still seem to be obeying the rules we defined in our static EROs. So lets now dig into the TED and the CSPF calculation. To se this in action, Im going to configure some trace options on vMX1…
set protocols mpls traceoptions file cspf
set protocols mpls traceoptions flag cspf
set protocols mpls traceoptions flag cspf-link
set protocols mpls traceoptions flag cspf-node
Now let’s start watching that log file with the monitor start cspf command and then clear the LSPs with the clear mpls lsp all command. You should get a ton of output which I wont bother pasting in as a huge chunk. Let’s instead review it piece by piece.
Chunk 1…
Feb 1 02:51:28.429348 RPD_MPLS_LSP_DOWN: MPLS LSP vmx1-vmx4 down on primary(loose-vmx1-vmx4)
Feb 1 02:51:28.430726 RPD_MPLS_PATH_DOWN: MPLS path loose-vmx1-vmx4 down on LSP vmx1-vmx4
Feb 1 02:51:28.430916 Change in TED since last CSPF run, new CSPF needed for path vmx1-vmx4(primary loose-vmx1-vmx4) 0 seq 10
Feb 1 02:51:28.430929 CSPF adding path vmx1-vmx4(primary loose-vmx1-vmx4) to CSPF queue 0
Feb 1 02:51:28.430938 CSPF creating CSPF job
Feb 1 02:51:28.431091
Feb 1 02:51:28.431116 CSPF for path vmx1-vmx4(primary loose-vmx1-vmx4), begin at 0000.0000.0000.00 , starting
Feb 1 02:51:28.431159 bandwidth: CT0=0bps ; setup priority: 0; random
Nothing terribly exciting here – we see the LSP go down, we see that CSPF is going to run, and then we see it begin processing.
Chunk 2…
Feb 1 02:51:28.431193 CSPF ERO 1 loose next hop to 2.2.2.2
Feb 1 02:51:28.431220 CSPF starting from 0000.0000.0000.00 (1.1.1.1) to 2.2.2.2, hoplimit 254
Feb 1 02:51:28.431233 Node 0000.0000.0000.00 (1.1.1.1) metric 0, hops 0, avail 0 32000 32000 32000
Feb 1 02:51:28.431254 Link 10.1.1.0->10.1.1.1(0000.0000.0000.00/2.2.2.2, Link IDs 334->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431269 Reverse Link for 10.1.1.0(1.1.1.1:334)->10.1.1.1(2.2.2.2:0) is 10.1.1.1(2.2.2.2:333)->10.1.1.0(1.1.1.1:0)
Feb 1 02:51:28.431293 link's interface switch capability descriptor #1
Feb 1 02:51:28.431301 encoding: Packet, switching: Packet
Feb 1 02:51:28.431306 link passes constraints
Feb 1 02:51:28.431319 Link 10.1.1.8->10.1.1.9(0000.0000.0000.00/3.3.3.3, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431330 Reverse Link for 10.1.1.8(1.1.1.1:335)->10.1.1.9(3.3.3.3:0) is 10.1.1.9(3.3.3.3:335)->10.1.1.8(1.1.1.1:0)
Feb 1 02:51:28.431334 link's interface switch capability descriptor #1
Feb 1 02:51:28.431337 encoding: Packet, switching: Packet
Feb 1 02:51:28.431340 link passes constraints
Feb 1 02:51:28.431346 Node 0000.0000.0000.00 (2.2.2.2) metric 1, hops 0, avail 1 32000 32000 32000
Feb 1 02:51:28.431350 CSPF Reached target
Feb 1 02:51:28.431357 CSPF completed in 0.000113s
Feb 1 02:51:28.431373 CSPF ERO for vmx1-vmx4(primary loose-vmx1-vmx4) (1 hops)
Feb 1 02:51:28.431379 node 10.1.1.1/32
Ok – now we’re cooking! In the first line we see that it’s beginning its work to evaluate the path to reach the loose hop of 2.2.2.2 to do this it’s going to look at both possible paths to reach the hop. This would include the path toward vMX2 and the path to vMX3. Lines 4-8 show it evaluating the path directly to vMX2 while lines 9-13 show it evaluating the path toward vMX3. In both cases, it decides that the link passes the constraints defined. Now it’s really just down to a matter of the least number of hops to reach the defined ERO. In our case – the path directly to vMX2 would be the better path so we see that as being selected in lines 17 and 18.
Chunk 3…
Feb 1 02:51:28.431384 CSPF ERO 2 loose next hop to 3.3.3.3
Feb 1 02:51:28.431390 CSPF starting from 0000.0000.0000.00 (2.2.2.2) to 3.3.3.3, hoplimit 253
Feb 1 02:51:28.431395 Node 0000.0000.0000.00 (2.2.2.2) metric 0, hops 0, avail 0 32000 32000 32000
Feb 1 02:51:28.431403 Link 10.1.1.1->10.1.1.0(0000.0000.0000.00/1.1.1.1, Link IDs 333->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431407 skipped: end point already visited in prio CSPF run
Feb 1 02:51:28.431413 Link 10.1.1.2->10.1.1.3(0000.0000.0000.00/3.3.3.3, Link IDs 334->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431419 Reverse Link for 10.1.1.2(2.2.2.2:334)->10.1.1.3(3.3.3.3:0) is 10.1.1.3(3.3.3.3:333)->10.1.1.2(2.2.2.2:0)
Feb 1 02:51:28.431423 link's interface switch capability descriptor #1
Feb 1 02:51:28.431427 encoding: Packet, switching: Packet
Feb 1 02:51:28.431435 Exact Reverse Link for 10.1.1.2->10.1.1.3 is 10.1.1.3->10.1.1.2
Feb 1 02:51:28.431442 link passes constraints
Feb 1 02:51:28.431449 Link 10.1.1.6->10.1.1.7(0000.0000.0000.00/4.4.4.4, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431456 Reverse Link for 10.1.1.6(2.2.2.2:335)->10.1.1.7(4.4.4.4:0) is 10.1.1.7(4.4.4.4:335)->10.1.1.6(2.2.2.2:0)
Feb 1 02:51:28.431459 link's interface switch capability descriptor #1
Feb 1 02:51:28.431463 encoding: Packet, switching: Packet
Feb 1 02:51:28.431491 Exact Reverse Link for 10.1.1.6->10.1.1.7 is 10.1.1.7->10.1.1.6
Feb 1 02:51:28.431495 link passes constraints
Feb 1 02:51:28.431500 Node 0000.0000.0000.00 (3.3.3.3) metric 1, hops 0, avail 1 32000 32000 32000
Feb 1 02:51:28.431503 CSPF Reached target
Feb 1 02:51:28.431509 CSPF completed in 0.000089s
Feb 1 02:51:28.431516 CSPF ERO for vmx1-vmx4(primary loose-vmx1-vmx4) (2 hops)
Feb 1 02:51:28.431520 node 10.1.1.1/32
Feb 1 02:51:28.431524 node 10.1.1.3/32
So this is much of what we saw in the last chunk – but now the router evaluates paths from the perspective of vMX2. Again – vMX1 is using the information in the TED to make this assessment. The first link it evaluates is the link between vMX1 and vMX2. Notice however on line 5 that it indicates that it’s already used this endpoint in a previous run so it skips it entirely. It then moves onto the next link, the one between vMX2 and vMX3. This link passes the constraints (because we don’t have any link (BW) constraints yet) and it then moves onto the third link which is the path between vMX2 and vMX4. This last link also passes the constraints so the router now has to pick from two paths, the one to vMX3 and the one to vMX4. Again – from a hop count perspective vMX4 is closer so we see that being selected in line 23 and being added onto the existing ERO.
Chunk 4…
Feb 1 02:51:28.431534 CSPF credibility 0
Feb 1 02:51:28.431538 CSPF final destination 4.4.4.4
Feb 1 02:51:28.431545 CSPF starting from 0000.0000.0000.00 (3.3.3.3) to 4.4.4.4, hoplimit 252
Feb 1 02:51:28.431550 Node 0000.0000.0000.00 (3.3.3.3) metric 0, hops 0, avail 0 32000 32000 32000
Feb 1 02:51:28.431557 Link 10.1.1.9->10.1.1.8(0000.0000.0000.00/1.1.1.1, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431561 skipped: end point already visited in prio CSPF run
Feb 1 02:51:28.431567 Link 10.1.1.3->10.1.1.2(0000.0000.0000.00/2.2.2.2, Link IDs 333->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431570 skipped: end point already visited in prio CSPF run
Feb 1 02:51:28.431576 Link 10.1.1.4->10.1.1.5(0000.0000.0000.00/4.4.4.4, Link IDs 334->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 1 02:51:28.431582 Reverse Link for 10.1.1.4(3.3.3.3:334)->10.1.1.5(4.4.4.4:0) is 10.1.1.5(4.4.4.4:333)->10.1.1.4(3.3.3.3:0)
Feb 1 02:51:28.431586 link's interface switch capability descriptor #1
Feb 1 02:51:28.431590 encoding: Packet, switching: Packet
Feb 1 02:51:28.431594 Exact Reverse Link for 10.1.1.4->10.1.1.5 is 10.1.1.5->10.1.1.4
Feb 1 02:51:28.431597 link passes constraints
Feb 1 02:51:28.431602 Node 0000.0000.0000.00 (4.4.4.4) metric 1, hops 0, avail 1 32000 32000 32000
Feb 1 02:51:28.431606 CSPF Reached target
Feb 1 02:51:28.431611 CSPF completed in 0.000050s
Feb 1 02:51:28.431617 CSPF ERO for vmx1-vmx4(primary loose-vmx1-vmx4) (3 hops)
Feb 1 02:51:28.431621 node 10.1.1.1/32
Feb 1 02:51:28.431625 node 10.1.1.3/32
Feb 1 02:51:28.431629 node 10.1.1.5/32
Feb 1 02:51:28.431991 CSPF for path vmx1-vmx4(primary loose-vmx1-vmx4) done!
Feb 1 02:51:28.469827 RPD_MPLS_PATH_UP: MPLS path loose-vmx1-vmx4 up on LSP vmx1-vmx4 path bandwidth 0 bps
Feb 1 02:51:28.470973 RPD_MPLS_LSP_UP: MPLS LSP vmx1-vmx4 up on primary(loose-vmx1-vmx4) Route 10.1.1.1 10.1.1.3 10.1.1.5 lsp bandwidth 0 bps
The above output represents the final run. The router has done it’s job and matched on both of our loosely defined EROs. Now it has one last destination to reach which is the LSP endpoint we defined with the to specification in the LSP definition. Again – the router makes these computations as if it was vMX3. Notice that it immediately discounts the links to vMX1 and vMX2 as they’ve already been visited as part of the previous CSPF runs. But wait a second, the link between vMX1 and vMX3 was evaluated, but not selected, so why can’t that be used here? We saw similar behavior in the previous chunk. If you think about it – this process of excluding the links makes pretty good sense. You want to keep moving closer to the egress router, not back towards the ingress router on a link that was evaluated by a router closer to the ingress. So at this point – the only link left is the one facing vMX4 which passes our constraints and is used. In line 18-21 we can see the completed ERO list and after that the router reports that the CSPF has completed and that the LSP is up!
Note: It’s interesting to note that if you did the same exercise on vMX4 that has an LSP defined with strict hops – you’d see the same logic occurring. That is – even though we tell it exactly where to go – it goes through the links one at a time and ensures that they pass the defined constraints.
So pretty cool huh? But I feel like this was perhaps more of an introduction to SPF – not CSPF. So let’s spice things up a bit and start looking at the most basic of reservations we can make – bandwidth.
So the first thing we’re going to do – is limit the bandwidth on a link that RSVP thinks is available to use. To see the impact of this on the TED, let’s look at what vMX4 thinks of vMX1 before we make any changes…
[email protected]> show ted database extensive 1.1.1.1
TED database: 0 ISIS nodes 4 INET nodes
NodeID: 1.1.1.1
Type: Rtr, Age: 2806 secs, LinkIn: 2, LinkOut: 2
Protocol: OSPF(0.0.0.0)
To: 2.2.2.2, Local: 10.1.1.0, Remote: 10.1.1.1
Local interface index: 334, Remote interface index: 0
Color: 0 <none>
Metric: 1
Static BW: 1000Mbps
Reservable BW: 1000Mbps
Available BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
Interface Switching Capability Descriptor(1):
Switching type: Packet
Encoding type: Packet
Maximum LSP BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
To: 3.3.3.3, Local: 10.1.1.8, Remote: 10.1.1.9
Local interface index: 335, Remote interface index: 0
Color: 0 <none>
Metric: 1
Static BW: 1000Mbps
Reservable BW: 1000Mbps
Available BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
Interface Switching Capability Descriptor(1):
Switching type: Packet
Encoding type: Packet
Maximum LSP BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
[email protected]>
Now let’s go ahead and limit the bandwidth on vMX1 with this command…
set protocols rsvp interface ge-0/0/1.0 subscription 50
Once committed, let’s go back and run that same command on vMX4…
[email protected]> show ted database extensive 1.1.1.1
TED database: 0 ISIS nodes 4 INET nodes
NodeID: 1.1.1.1
Type: Rtr, Age: 3 secs, LinkIn: 2, LinkOut: 2
Protocol: OSPF(0.0.0.0)
To: 2.2.2.2, Local: 10.1.1.0, Remote: 10.1.1.1
Local interface index: 334, Remote interface index: 0
Color: 0 <none>
Metric: 1
Static BW: 1000Mbps
Reservable BW: 500Mbps
Available BW [priority] bps:
[0] 500Mbps [1] 500Mbps [2] 500Mbps [3] 500Mbps
[4] 500Mbps [5] 500Mbps [6] 500Mbps [7] 500Mbps
Interface Switching Capability Descriptor(1):
Switching type: Packet
Encoding type: Packet
Maximum LSP BW [priority] bps:
[0] 500Mbps [1] 500Mbps [2] 500Mbps [3] 500Mbps
[4] 500Mbps [5] 500Mbps [6] 500Mbps [7] 500Mbps
To: 3.3.3.3, Local: 10.1.1.8, Remote: 10.1.1.9
Local interface index: 335, Remote interface index: 0
Color: 0 <none>
Metric: 1
Static BW: 1000Mbps
Reservable BW: 1000Mbps
Available BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
Interface Switching Capability Descriptor(1):
Switching type: Packet
Encoding type: Packet
Maximum LSP BW [priority] bps:
[0] 1000Mbps [1] 1000Mbps [2] 1000Mbps [3] 1000Mbps
[4] 1000Mbps [5] 1000Mbps [6] 1000Mbps [7] 1000Mbps
[email protected]>
Ah hah! So on the link to vMX2 (2.2.2.2) ,from the perspective of vMX1, we now see that the available bandwidth has been cut in half. So now let’s try defining a bandwidth reservation as part of our ingress LSP on vMX1…
set protocols mpls label-switched-path vmx1-vmx4 bandwidth 600m
Before you commit this change, it would be wise to run run monitor start cspf so you can see the output of the CSPF calculations once this constraint is in. Let’s walk through what happens – but from a more visual point of view…
ERO 2.2.2.2 First hop calculation…
Feb 2 04:01:25.753034 CSPF ERO 1 loose next hop to 2.2.2.2
Feb 2 04:01:25.753052 CSPF starting from 0000.0000.0000.00 (1.1.1.1) to 2.2.2.2, hoplimit 254
Feb 2 04:01:25.753066 constraint bandwidth: CT0=600Mbps
Feb 2 04:01:25.753075 Node 0000.0000.0000.00 (1.1.1.1) metric 0, hops 0, avail 0 32000 32000 32000
Feb 2 04:01:25.753085 Link 10.1.1.0->10.1.1.1(0000.0000.0000.00/2.2.2.2, Link IDs 334->0) metric 1 color 0x00000000 bw 500Mbps
Feb 2 04:01:25.753098 Reverse Link for 10.1.1.0(1.1.1.1:334)->10.1.1.1(2.2.2.2:0) is 10.1.1.1(2.2.2.2:333)->10.1.1.0(1.1.1.1:0)
Feb 2 04:01:25.753133 CT0 avail: 500Mbps, adaptive: 0bps local: 0bps (0s), net: -100Mbps
Feb 2 04:01:25.753138 link hasn't enough BW
Feb 2 04:01:25.753148 Link 10.1.1.8->10.1.1.9(0000.0000.0000.00/3.3.3.3, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753155 Reverse Link for 10.1.1.8(1.1.1.1:335)->10.1.1.9(3.3.3.3:0) is 10.1.1.9(3.3.3.3:335)->10.1.1.8(1.1.1.1:0)
Feb 2 04:01:25.753163 CT0 avail: 1000Mbps, adaptive: 0bps local: 0bps (0s), net: 400Mbps
Feb 2 04:01:25.753182 link's interface switch capability descriptor #1
Feb 2 04:01:25.753191 encoding: Packet, switching: Packet
Feb 2 04:01:25.753195 link passes constraints
Above is the output for the first ERO calculation that vMX1 will run. It is important to remember that this is done hop by hop. Notice that the fourth line indicates that this is being run from the perspective of node 1.1.1.1. In the above output, CSPF will evaluate the links connected to vMX1 to determine its first hop toward the ERO of 2.2.2.2. As we saw previously – it will evaluate the following links…
However – as you can see on line 4 of the above output – the router immediately catches onto the fact that there isn’t enough bandwidth available on the link to vMX2 (highlighted in orange) and discards it. This leaves us with the only available path being the link to vMX3 which we see in line 10 passes the constraints.
ERO 2.2.2.2 Second hop calculation…
Feb 2 04:01:25.753207 Node 0000.0000.0000.00 (3.3.3.3) metric 1, hops 0, avail 1 12800 32000 32000
Feb 2 04:01:25.753217 Link 10.1.1.9->10.1.1.8(0000.0000.0000.00/1.1.1.1, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753221 skipped: end point already visited
Feb 2 04:01:25.753227 Link 10.1.1.3->10.1.1.2(0000.0000.0000.00/2.2.2.2, Link IDs 333->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753234 Reverse Link for 10.1.1.3(3.3.3.3:333)->10.1.1.2(2.2.2.2:0) is 10.1.1.2(2.2.2.2:334)->10.1.1.3(3.3.3.3:0)
Feb 2 04:01:25.753244 CT0 avail: 1000Mbps, adaptive: 0bps local: 0bps (0s), net: 400Mbps
Feb 2 04:01:25.753248 link's interface switch capability descriptor #1
Feb 2 04:01:25.753252 encoding: Packet, switching: Packet
Feb 2 04:01:25.753260 Exact Reverse Link for 10.1.1.3->10.1.1.2 is 10.1.1.2->10.1.1.3
Feb 2 04:01:25.753268 link passes constraints
Feb 2 04:01:25.753275 Link 10.1.1.4->10.1.1.5(0000.0000.0000.00/4.4.4.4, Link IDs 334->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753282 Reverse Link for 10.1.1.4(3.3.3.3:334)->10.1.1.5(4.4.4.4:0) is 10.1.1.5(4.4.4.4:333)->10.1.1.4(3.3.3.3:0)
Feb 2 04:01:25.753289 CT0 avail: 1000Mbps, adaptive: 0bps local: 0bps (0s), net: 400Mbps
Feb 2 04:01:25.753292 link's interface switch capability descriptor #1
Feb 2 04:01:25.753296 encoding: Packet, switching: Packet
Feb 2 04:01:25.753300 Exact Reverse Link for 10.1.1.4->10.1.1.5 is 10.1.1.5->10.1.1.4
Feb 2 04:01:25.753304 link passes constraints
Feb 2 04:01:25.753309 Node 0000.0000.0000.00 (2.2.2.2) metric 2, hops 0, avail 2 12800 12800 32000
Feb 2 04:01:25.753313 CSPF Reached target
Feb 2 04:01:25.753320 CSPF completed in 0.000204s
Feb 2 04:01:25.753347 CSPF ERO for vmx1-vmx4(primary loose-vmx1-vmx4) (2 hops)
Feb 2 04:01:25.753353 node 10.1.1.9/32
Feb 2 04:01:25.753357 node 10.1.1.2/32
Again – notice that we’re now running the CSPF calculation from the perspective on vMX3 (line 1 above). After reviewing the possible links – the router determines that the path back to vMX1 isn’t good since it was already looked at – and sees that the links to vMX4 and vMX2 pass the constraints. Clearly the path to vMX2 that’s direct is the best next hop so that one is chosen…
Success! We’ve now met the constraint of getting to the loose hop of 2.2.2.2. However – if you’ve been paying attention – you are likely starting to see a problem developing. Let’s keep going through the logs though and see how this plays out…
ERO 3.3.3.3 First hop calculation…
Feb 2 04:01:25.753361 CSPF ERO 2 loose next hop to 3.3.3.3
Feb 2 04:01:25.753369 CSPF starting from 0000.0000.0000.00 (2.2.2.2) to 3.3.3.3, hoplimit 252
Feb 2 04:01:25.753374 constraint bandwidth: CT0=600Mbps
Feb 2 04:01:25.753379 Node 0000.0000.0000.00 (2.2.2.2) metric 0, hops 0, avail 0 32000 32000 32000
Feb 2 04:01:25.753394 Link 10.1.1.1->10.1.1.0(0000.0000.0000.00/1.1.1.1, Link IDs 333->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753398 skipped: end point already visited in prio CSPF run
Feb 2 04:01:25.753404 Link 10.1.1.2->10.1.1.3(0000.0000.0000.00/3.3.3.3, Link IDs 334->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753408 skipped: end point already visited in prio CSPF run
Feb 2 04:01:25.753414 Link 10.1.1.6->10.1.1.7(0000.0000.0000.00/4.4.4.4, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753420 Reverse Link for 10.1.1.6(2.2.2.2:335)->10.1.1.7(4.4.4.4:0) is 10.1.1.7(4.4.4.4:335)->10.1.1.6(2.2.2.2:0)
Feb 2 04:01:25.753428 CT0 avail: 1000Mbps, adaptive: 0bps local: 0bps (0s), net: 400Mbps
Feb 2 04:01:25.753431 link's interface switch capability descriptor #1
Feb 2 04:01:25.753435 encoding: Packet, switching: Packet
Feb 2 04:01:25.753439 Exact Reverse Link for 10.1.1.6->10.1.1.7 is 10.1.1.7->10.1.1.6
Feb 2 04:01:25.753443 link passes constraints
As we look to reach the second loose hop we defined (3.3.3.3) we now need to evaluate the paths from vMX2. As we do that, we see that we quickly discount the paths to both vMX1 and vMX3 as they’ve already been visited as part of this run. This only leaves the path to vMX4…
Now we need to try and reach 3.3.3.3 from the perspective of vMX4…
ERO 3.3.3.3 Second hop calculation…
Feb 2 04:01:25.753449 Node 0000.0000.0000.00 (4.4.4.4) metric 1, hops 0, avail 1 12800 32000 32000
Feb 2 04:01:25.753455 Link 10.1.1.7->10.1.1.6(0000.0000.0000.00/2.2.2.2, Link IDs 335->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753459 skipped: end point already visited
Feb 2 04:01:25.753465 Link 10.1.1.5->10.1.1.4(0000.0000.0000.00/3.3.3.3, Link IDs 333->0) metric 1 color 0x00000000 bw 1000Mbps
Feb 2 04:01:25.753468 skipped: end point already visited in prio CSPF run
Feb 2 04:01:25.753474 CSPF completed in 0.000080s
Feb 2 04:01:25.753478 CSPF couldn't find a route to 3.3.3.3
Feb 2 04:01:25.753499 CSPF for path vmx1-vmx4(primary loose-vmx1-vmx4) done!
Above we can clearly see the problem – with the given bandwidth constraints – we’ve forced the traffic into a path that doesn’t end well. Once the traffic reaches vMX4 – we’ve exhausted all possible paths to reach 3.3.3.3 and we see in line 7 above that we weren’t able to build an LSP that meets this criteria…
As you can see – by manually defining EROs you can rather quickly get yourself into trouble.
As a last step in this lab, let’s clear the bandwidth constraint and one of the defined ERO on vMX1 and talk briefly about other metrics we can use to influence the LSP path…
delete protocols mpls label-switched-path vmx1-vmx4 bandwidth 600m
delete protocols rsvp interface ge-0/0/1.0 subscription 50
delete protocols mpls path loose-vmx1-vmx4 2.2.2.2
After deleting the configuration above – if we do a quick trace – we should see vMX1 taking this path for the LSP…
[email protected]> traceroute mpls rsvp vmx1-vmx4
Probe options: retries 3, exp 7
ttl Label Protocol Address Previous Hop Probe Status
1 300336 RSVP-TE 10.1.1.9 (null) Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
2 3 RSVP-TE 10.1.1.5 10.1.1.9 Egress
FEC-Stack-Sent: RSVP
Path 1 via ge-0/0/2.0 destination 127.0.0.64
[email protected]>
Based on what we know about path selection – this makes good sense. Now let’s talk about how we can influence the path making decision with TE metrics. I can define a metric on a link that is specific only to TE. For instance – let’s look at the metric for the LSP as it stands now….
[email protected]> show mpls lsp ingress name vmx1-vmx4 extensive
Ingress LSP: 1 sessions
4.4.4.4
From: 1.1.1.1, State: Up, ActiveRoute: 0, LSPname: vmx1-vmx4
ActivePath: loose-vmx1-vmx4 (primary)
LSPtype: Static Configured, Penultimate hop popping
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
*Primary loose-vmx1-vmx4 State: Up
Priorities: 7 0
SmartOptimizeTimer: 180
Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 2)
...
<additional output removed>
So we can see that currently this CSPF metric is 2. If we looked at the IGP metric for 4.4.4.4 we’ll see that this all aligns…
[email protected]> show route table inet.0 4.4.4.4
inet.0: 18 destinations, 18 routes (18 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
4.4.4.4/32 *[OSPF/10] 2d 14:40:49, metric 2
> to 10.1.1.1 via ge-0/0/1.0
to 10.1.1.9 via ge-0/0/2.0
[email protected]>
Now let’s add a TE metric on vMX1 for the link facing vMX3…
set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 te-metric 3
Now let’s clear the LSPs (clear mpls lsp all) and check out metrics again…
[email protected]> show mpls lsp ingress name vmx1-vmx4 extensive
Ingress LSP: 1 sessions
4.4.4.4
From: 1.1.1.1, State: Up, ActiveRoute: 0, LSPname: vmx1-vmx4
ActivePath: loose-vmx1-vmx4 (primary)
LSPtype: Static Configured, Penultimate hop popping
LoadBalance: Random
Encoding type: Packet, Switching type: Packet, GPID: IPv4
*Primary loose-vmx1-vmx4 State: Up
Priorities: 7 0
SmartOptimizeTimer: 180
Computed ERO (S [L] denotes strict [loose] hops): (CSPF metric: 3)
...
<additional output removed>
Notice that out CSPF metric has changed. If we do another trace, we’ll see we’re taking a new path as well…
[email protected]> traceroute mpls rsvp vmx1-vmx4
Probe options: retries 3, exp 7
ttl Label Protocol Address Previous Hop Probe Status
1 300496 RSVP-TE 10.1.1.1 (null) Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
2 300464 RSVP-TE 10.1.1.3 10.1.1.1 Success
FEC-Stack-Sent: RSVP
ttl Label Protocol Address Previous Hop Probe Status
3 3 RSVP-TE 10.1.1.5 10.1.1.3 Egress
FEC-Stack-Sent: RSVP
Path 1 via ge-0/0/1.0 destination 127.0.0.64
[email protected]>
It’s important to call out that this change in no way impacted the underlying IGP metrics in the inet.0 table.
[email protected]> show route table inet.0 4.4.4.4
inet.0: 18 destinations, 18 routes (18 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
4.4.4.4/32 *[OSPF/10] 2d 14:52:35, metric 2
> to 10.1.1.1 via ge-0/0/1.0
to 10.1.1.9 via ge-0/0/2.0
[email protected]>
I hope this gave you some insight into how CSPF runs and how important the TED is to it’s calculations. In the next post we’ll continue to cover some of the additional features of RSVP that make it more flexible than LDP.
I’m absolutely loving the detailed explanations of RSVP and specifically how the ERO is calculated from the perspective of each router in the path even though it’s actually calculated on the ingress LSR. This is a point that always confused me but makes a lot more sense now.
I think there may be a typo on this page. On the explanation of the CSPF chunk #3, you indicate that both vMX3 and vMX4 could be used but that vMX4 was chosen, but I think you mean vMX3 was chosen. If that’s not true, then I’m really confused! 🙂
Keep up the great info!