I know we haven’t gotten to the point of actually discussing inter-as option B or option C but we did cover all of the mechanics required for them to work (at least briefly). Before we go there though – I did want to cover a different technology that can also help solve our end to end LSP problems. Carrier Supporting Carrier – or more commonly just referred to as CsC (or in Cisco parlance Carrier of Carriers or CoC (or maybe I have that backwards?)) is a means to nest MPLS VPN sessions. Think of it this way. If you’re a huge backbone provider (we’ll call you the “uber provider”) selling transport to customers (we’ll call them customer carriers) – there’s a good chance that those customer carriers will need a way to isolate their customer on their backbone. How do we do that? Well MPLS VPNs of course! However you, the uber provider, also need a means to keep the customer carriers isolated as they traverse your backbone. So what do we do? Sounds a lot like a carrier supporting a carrier huh? CsC to the rescue!
At this point you might be wondering how we got here. To be clear, I didn’t just pull CsC out of my “write a random blog on this topic” hat. We’ve spent the last few posts really trying to describe what it means to have end to end LSPs and why they are so important. We’ve talked briefly about how we can do this with label stitching and even touched briefly (admittedly by accident) on the inter-AS architectures B and C that allow us to stitch together discrete label domains. But those solutions are not the only way to crack this problem. CsC can solve this problem too! Although, for some reason, it always seems to be discounted in this space. I’m not sure why that is – perhaps it’s assumed to be complex or geared toward a specific uses case. However – for those of you that read the great book “MPLS in the SDN era” you may recall that CsC is covered at the end of chapter 9 which itself is titled “Inter-Domain MPLS Services”. And while the authors don’t discuss it in detail or specifically call out that it can be used to solve this problem – I believe it’s in that chapter for a reason. Alright – enough chit chat – let’s get to the lab.
For the sake of making this look more real life(ish) I’ve expanded the lab greatly as you can see below. The goal of doing this was to make the LSP segments longer so that we saw some real label operations happening (not just routers popping off labels since they were PHP routers).
As per usual, the blue numbering indicates interface numbering (eg. 0 = ge-0/0/0) and the black numbering indicates the IP addressing. In our case, all of the interface addressing is link local and comes out of 169.254.10.0/24. So for example the link between vMX1 and vMX2 is 169.254.10.0/31. I’ve also removed the loopbacks from the diagram this time around since those will be a little more dynamic this time.
So let’s talk about what we want this to look like…
The easiest way to think of this is that we have a carrier that is split in two which is represented by the routers in the orange blocks. We’ll call this the “orange carrier”. In the left orange block we have a carrier network which connects to a customer CE router. That customer wishes to talk to it’s other CE router on the right side but our orange carrier doesn’t have any direct connectivity to it’s network on the right. To make the carrier “whole”, we need to connect the left side of the orange carrier network to the right side. To do this, we leverage a connection from another transport provider which is represented by the routers in the green box. We’ll call this the “green carrier”. Going forward, we’ll assume that the orange carrier is a customer to the green carrier. You’ll notice that we have updated the name of the router to include its role. You’ll notice some familiar terminology here. There are PE, CE, and P routers – but we also now have a new class of those routers prefaced with “CSC”. The CSC model works much like the normal PE/CE model. In this case, the orange carrier is a customer of the green carrier. So the orange carrier node that connects to the green carrier (vMX5) is considered the CSC-CE router while the green carrier router that connects to it’s customer is considered the CSC-PE router (vMX6).
So at this point – why don’t I stop talking your ear off and we can just get into the config. Since this configuration is more in depth, Im going to start each router with a different baseline configuration. Don’t worry, I’ll talk through anything that looks crazy…
vMX1
set interfaces ge-0/0/0 unit 0 family inet address 10.10.10.1/24
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.0/31
set routing-options autonomous-system 65001
set protocols bgp group to_provider type external
set protocols bgp group to_provider export customer_routes
set protocols bgp group to_provider neighbor 169.254.10.1 peer-as 65002
set policy-options prefix-list customer_routes 10.10.10.0/24
set policy-options policy-statement customer_routes term direct from protocol direct
set policy-options policy-statement customer_routes term direct from prefix-list customer_routes
set policy-options policy-statement customer_routes term direct then accept
set policy-options policy-statement customer_routes then reject
The base configuration of vMX1 is fairly straight forward. In this lab, it’s acting as a pure CE router and our PE/CE protocol of choice is going to be BGP. vMX1 will come from AS 65001 and connect to the PE in AS 65002. For the sake of clarity, I haven’t configured a loopback address since we wont be using it in this lab and I don’t want to confuse things.
Note: I’ll just mention here that the choice of CE/PE protocol being BGP here has very little bearing on the larger architecture. AKA – Don’t read too much into the BGP config between the CE and PE routes in the orange carrier. It’s just there to get the customer prefixes into the VRF table on the PE. Don’t let it distract you from the other BGP peerings that are going on that support CsC.
vMX2
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.1/31
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.2/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 2.2.2.2/32 primary
set routing-options router-id 2.2.2.2
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/1.0
set routing-instances customer instance-type vrf
set routing-instances customer interface ge-0/0/0.0
set routing-instances customer route-distinguisher 100:100
set routing-instances customer vrf-target target:100:100
set routing-instances customer vrf-table-label
set routing-instances customer routing-options autonomous-system 65002
set routing-instances customer protocols bgp group customer1 type external
set routing-instances customer protocols bgp group customer1 neighbor 169.254.10.0 peer-as 65001
The base configuration of vMX2 get’s us a few things. First – an OSPF configuration so that we can talk to the rest of the orange provider router loopbacks. We also enable MPLS and LDP on the upstream interface. Secondly – we define a customer routing instance for our customer. Inside the routing-instance we define our BGP parameters so that we can peer with the customer.
vMX3
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.3/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.4/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 3.3.3.3/32 primary
set routing-options router-id 3.3.3.3
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
vMX3 is our first P router so the configuration here is drop dead easy. OSPF, MPLS, and LDP on all of the interfaces.
vMX4
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.5/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.6/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 4.4.4.4/32 primary
set routing-options router-id 4.4.4.4
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
Same deal here on vMX4. Another P router configuration.
vMX5
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.7/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.8/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 5.5.5.5/32 primary
set routing-options router-id 5.5.5.5
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
vMX5 is our CSC-CE router between the orange and green provider. Now this is where things “could” get interesting but surprisingly don’t. What other configuration wasn’t that interesting again? Oh yeah – the CE config on vMX1 (and vMX14). That’s because a CSC-CE should look a lot like a CE router! Since this is a CE/PE configuration, you might expect BGP here – but in this case, we’re not going to do it. I’ll talk about that more below.
vMX6
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.9/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.10/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 6.6.1.1/32
set routing-options router-id 6.6.1.1
set protocols rsvp interface ge-0/0/1.0
set protocols mpls label-switched-path to-vmx9 to 9.9.1.1
set protocols mpls interface ge-0/0/1.0
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
The base configuration of the green carrier routers aims to get things rolling for communication within the green carrier. Notice that the loopback address is different in this case (6.6.1.1) and is a theme we will see with all the routers inside of the green carrier. Since we’re going to run RSVP within the green carrier – we also define a LSP to the far end of the green carrier (vMX9). Beyond that it’s just a standard OSPF configuration.
vMX7
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.11/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.12/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 7.7.1.1/32 primary
set routing-options router-id 7.7.1.1
set protocols rsvp interface ge-0/0/1.0
set protocols rsvp interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols mpls interface ge-0/0/0.0
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
Again – a boring P router configuration. Nothing to see here – move along.
vMX8
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.13/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.14/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 8.8.1.1/32 primary
set routing-options router-id 8.8.1.1
set protocols rsvp interface ge-0/0/1.0
set protocols rsvp interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols mpls interface ge-0/0/0.0
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
Same deal – a boring P router configuration. Boring is nice right? 🙂
vMX9
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.15/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.16/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 9.9.1.1/32
set routing-options router-id 9.9.1.1
set routing-options autonomous-system 65000
set protocols rsvp interface ge-0/0/0.0
set protocols mpls label-switched-path to-vmx6 to 6.6.1.1
set protocols mpls interface ge-0/0/0.0
set protocols ospf traffic-engineering
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
Just like we saw on vMX6. This should get us connectivity within the green carrier as well as define the RSVP LSP back to vMX6.
vMX10
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.17/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.18/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 10.10.10.10/32 primary
set routing-options router-id 10.10.10.10
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
This configuration is pretty much the same that we saw on vMX5. Don’t worry about how simple this looks for now. We’ll talk more about why this is later on.
vMX11
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.19/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.20/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 11.11.11.11/32 primary
set routing-options router-id 11.11.11.11
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
Another boring P router config. OSPF, LDP, and MPLS.
vMX12
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.21/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.22/31
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces fxp0 unit 0 family inet address 192.168.127.12/24
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 12.12.12.12/32 primary
set routing-options static route 10.171.200.0/22 next-hop 192.168.127.100
set routing-options router-id 12.12.12.12
set protocols mpls interface ge-0/0/0.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set protocols ldp interface ge-0/0/1.0
More boring P router config.
vMX13
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.23/31
set interfaces ge-0/0/0 unit 0 family mpls
set interfaces ge-0/0/1 unit 0 family inet address 169.254.10.24/31
set interfaces lo0 unit 0 family inet no-redirects
set interfaces lo0 unit 0 family inet address 13.13.13.13/32 primary
set routing-options router-id 13.13.13.13
set protocols mpls interface ge-0/0/0.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set protocols ldp interface ge-0/0/0.0
set routing-instances customer instance-type vrf
set routing-instances customer interface ge-0/0/1.0
set routing-instances customer route-distinguisher 100:100
set routing-instances customer vrf-target target:100:100
set routing-instances customer vrf-table-label
set routing-instances customer routing-options autonomous-system 65004
set routing-instances customer protocols bgp group customer1 type external
set routing-instances customer protocols bgp group customer1 neighbor 169.254.10.25 peer-as 65003
We’re doing the same thing here that we did on vMX2. We configure our customer routing instance and map the interface facing the CE into it. We also configure BGP inside the routing-instance to peer with the CE in AS 65003 and we configure out local AS to 65004.
vMX14
set interfaces ge-0/0/0 unit 0 family inet address 169.254.10.25/31
set interfaces ge-0/0/1 unit 0 family inet address 192.168.10.1/24
set routing-options autonomous-system 65003
set protocols bgp group to_provider type external
set protocols bgp group to_provider export customer_routes
set protocols bgp group to_provider neighbor 169.254.10.24 peer-as 65004
set policy-options prefix-list customer_routes 192.168.10.0/24
set policy-options policy-statement customer_routes term direct from protocol direct
set policy-options policy-statement customer_routes term direct from prefix-list customer_routes
set policy-options policy-statement customer_routes term direct then accept
set policy-options policy-statement customer_routes then reject
Alright – so that was a lot of configuration – but hopefully none of it was too new for you. The goal of the baseline configuration was to get the base protocols running in each of our orange and green “boxes”. At this point, our CE’s should be configured and peered into their VRFs in the orange carrier. Let’s make sure that’s the case…
[email protected]> show bgp summary instance customer
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
customer.inet.0
1 1 0 0 0 0
customer.mdt.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
169.254.10.0 65001 2257 2256 0 0 16:53:02 Establ
customer.inet.0: 1/1/1/0
[email protected]> show route receive-protocol bgp 169.254.10.0 table customer.inet.0
customer.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.10.10.0/24 169.254.10.0 65001 I
[email protected]>
Notice the syntax we use to look at things related to the customer routing instance. In the case of seeing the BGP summary we can tack on the instance customer command. However, when looking at the routes received we can just call out the table we want to see. So it looks like the peering is up on the left side and we’re receiving the advertisement, let’s check the right side as well…
[email protected]> show bgp summary instance customer
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
customer.inet.0
1 1 0 0 0 0
customer.mdt.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
169.254.10.25 65003 2256 2254 0 0 16:52:20 Establ
customer.inet.0: 1/1/1/0
[email protected]> show route receive-protocol bgp 169.254.10.25 table customer.inet.0
customer.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 192.168.10.0/24 169.254.10.25 65003 I
[email protected]>
Again – all looks good there. Now let’s take a look at what’s going on within the orange carrier left and right boxes. Based on the baseline configuration I’d expect that each router within each box can see each other router in the same box both with OSPF as well as LDP. Let’s check…
[email protected]> show route table inet.3
inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
3.3.3.3/32 *[LDP/9] 16:58:21, metric 1
> to 169.254.10.3 via ge-0/0/1.0
4.4.4.4/32 *[LDP/9] 16:58:19, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299792
5.5.5.5/32 *[LDP/9] 16:58:08, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299808
[email protected]>
Ok – so that looks just about like I’d expect it to look. vMX2 has inet.3 entries for all the other routers within it’s currently reachable network (orange box). Confirming this is enough to prove to me that OSPF is also working since LDP relies on it. Let’s validate the right side orange box as well…
[email protected]> show route table inet.3
inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.10.10.10/32 *[LDP/9] 16:59:04, metric 1
> to 169.254.10.22 via ge-0/0/0.0, Push 299792
11.11.11.11/32 *[LDP/9] 16:59:04, metric 1
> to 169.254.10.22 via ge-0/0/0.0, Push 299776
12.12.12.12/32 *[LDP/9] 16:59:04, metric 1
> to 169.254.10.22 via ge-0/0/0.0
[email protected]>
Perfect. And last but not least, let’s confirm that the green carrier also has a baseline configuration. I’m hoping that we see an RSVP LSP between vMX6 and vMX9 fully formed…
[email protected]> show rsvp session
Ingress RSVP: 1 sessions
To From State Rt Style Labelin Labelout LSPname
9.9.1.1 6.6.1.1 Up 0 1 FF - 299792 to-vmx9
Total 1 displayed, Up 1, Down 0
Egress RSVP: 1 sessions
To From State Rt Style Labelin Labelout LSPname
6.6.1.1 9.9.1.1 Up 0 1 FF 3 - to-vmx6
Total 1 displayed, Up 1, Down 0
Transit RSVP: 0 sessions
Total 0 displayed, Up 0, Down 0
[email protected]>
Perfect! So where do we go from here? At this point, we once again appear to have 3 distinct label domains. Each orange carrier is running LDP within it’s respective box (left and right side) and the backbone or green carrier is running RSVP. So let’s remember what our goal is here…
If it was all the same to the orange carrier, they’d prefer that this looked like one big happy provider network. In fact, they require that. They don’t care about what the green provider is doing to make that happen – their only requirement is connectivity between their two discrete networks that supports them running MPLS VPNs end to end (which dictates the use of end to end LSPs).
Now – when we talk about CsC – there are two primary ways to tackle this solution. One way is to run LDP and an IGP (like OSPF) inside the CSC-PE (vMX6 and vMX9) orange carrier VRF. This solution is often frowned upon because of the complexities that can come along with using OSPF as a PE/CE protocol as well as the fact that BGP really is the defacto standard for PE/CE peering. You’ll hear no disagreement from me on BGP being the clear choice for CE/PE protocols. It’s is the defacto standard for good reason. BGP allows a much greater level of policy control than OSPF does and quite simply is better designed for this kind of situation. HOWEVER – one does not always need this level of policy control. Remember that these types of deployments are not always company A and company B situations where A is trying to protect itself from B and vice-versa. That is – there’s no reason a single entity can’t use CsC to scale its MPLS network and help stitch together end to end LSPs.
In my research to write this post, I found out that configuring CsC using LDP and OSPF as I’ve described above wasn’t talked about at all. Cisco had some documentation on it and it’s mentioned periodically but I couldn’t find any examples of doing it on JunOS. That being said, I think it’s a reasonable solution and so that will be the way we configure CsC in this blog. We will also try to cover the alternative solution of doing CsC with BGP (BGP-LU specifically) but that will be in another blog.
So – now that we have our baseline configuration – where do we start? Let’s start by trying to get the connectivity between the two orange carrier segments working through the green carrier. As I’ve hinted at above, to do that we’ll be configuring LDP and OSPF inside of the VRF on each CSC-PE. One of the things I’d like to try to reiterate about CsC is that it is not unlike a standard MPLS VPN setup. The PE/CE configuration looks pretty darn similar as the CsC layer as it does at the lower customer layer. So let’s start by configuring a routing instance for the orange carrier on each CSC-PE….
vMX6
set interfaces lo0 unit 1 family inet address 6.6.6.6/32
set routing-instances orange_carrier instance-type vrf
set routing-instances orange_carrier interface ge-0/0/0.0
set routing-instances orange_carrier interface lo0.1
set routing-instances orange_carrier route-distinguisher 1:1
set routing-instances orange_carrier vrf-target target:1:1
set routing-instances orange_carrier vrf-table-label
set routing-instances orange_carrier routing-options router-id 6.6.6.6
set routing-instances orange_carrier protocols mpls interface ge-0/0/0.0
set routing-instances orange_carrier protocols mpls interface lo0.1
set routing-instances orange_carrier protocols ospf area 0.0.0.0 interface lo0.1
set routing-instances orange_carrier protocols ospf area 0.0.0.0 interface ge-0/0/0.0 interface-type p2p
set routing-instances orange_carrier protocols ldp interface ge-0/0/0.0
The above configuration accomplishes a number of things. First – note that we added another loopback interface (lo0.1) which we’ll use inside the VRF. Second – we defined a typical routing-instance configuration that included configuring MPLS, OSPF, and LDP inside of the VRF. Beyond them being there in the first place, their configuration should look pretty straight forward. Alright – now let’s do the same on vMX9 – the other CSC-PE…
vMX9
set interfaces lo0 unit 1 family inet address 9.9.9.9/32
set routing-instances orange_carrier instance-type vrf
set routing-instances orange_carrier interface ge-0/0/1.0
set routing-instances orange_carrier interface lo0.1
set routing-instances orange_carrier route-distinguisher 1:1
set routing-instances orange_carrier vrf-target target:1:1
set routing-instances orange_carrier vrf-table-label
set routing-instances orange_carrier routing-options router-id 9.9.9.9
set routing-instances orange_carrier protocols mpls interface ge-0/0/1.0
set routing-instances orange_carrier protocols mpls interface lo0.1
set routing-instances orange_carrier protocols ospf area 0.0.0.0 interface ge-0/0/1.0 interface-type p2p
set routing-instances orange_carrier protocols ospf area 0.0.0.0 interface lo0.1
set routing-instances orange_carrier protocols ldp interface ge-0/0/1.0
Once that’s in place. We should now see that the orange carrier routers view the CSC-PE routers as part of their LDP domain…
[email protected]> show route table inet.3
inet.3: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
9.9.9.9/32 *[LDP/9] 00:17:55, metric 1
> to 169.254.10.22 via ge-0/0/0.0, Push 299824
10.10.10.10/32 *[LDP/9] 18:54:21, metric 1
> to 169.254.10.22 via ge-0/0/0.0, Push 299792
11.11.11.11/32 *[LDP/9] 18:54:21, metric 1
> to 169.254.10.22 via ge-0/0/0.0, Push 299776
12.12.12.12/32 *[LDP/9] 18:54:21, metric 1
> to 169.254.10.22 via ge-0/0/0.0
[email protected]>
Notice that we now see a path to vMX9 in inet.3. We’ll see that on the left side we now also see the CSC-PE loopback as well…
[email protected]> show route table inet.3
inet.3: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
3.3.3.3/32 *[LDP/9] 18:57:15, metric 1
> to 169.254.10.3 via ge-0/0/1.0
4.4.4.4/32 *[LDP/9] 18:57:13, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299792
5.5.5.5/32 *[LDP/9] 18:57:02, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299808
6.6.6.6/32 *[LDP/9] 00:00:04, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299824
[email protected]>
Perfect. So now we’ve begun the integration of the orange carrier into the green carrier network, However, beyond providing connectivity for LDP and OSPF – we haven’t really done anything. If the green carrier is going to provide VPN connectivity for the orange carrier – then we need to provide a way for both of the green carrier CSC-PEs to exchange route information. This will come in the form a MP-BGP peering in the inet-vpn address family. So let’s get that configured next…
vMX6
set routing-options autonomous-system 65000
set protocols bgp group green_carrier_vpn type internal
set protocols bgp group green_carrier_vpn family inet-vpn unicast
set protocols bgp group green_carrier_vpn neighbor 9.9.1.1 local-address 6.6.1.1
set protocols bgp group green_carrier_vpn neighbor 9.9.1.1 peer-as 65000
vMX9
set routing-options autonomous-system 65000
set protocols bgp group green_carrier_vpn type internal
set protocols bgp group green_carrier_vpn family inet-vpn unicast
set protocols bgp group green_carrier_vpn neighbor 6.6.1.1 local-address 9.9.1.1
set protocols bgp group green_carrier_vpn neighbor 6.6.1.1 peer-as 65000
Once that iBGP peering is up – we should see the session come up and become established. As we’ve seen before – a BGP peering in the inet-vpn family will automatically export all of the VRF or routing-instance routes as VPNv4 routes to the peer…
[email protected]> show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
bgp.l3vpn.0
9 9 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
9.9.1.1 65000 11 10 0 0 1:45 Establ
bgp.l3vpn.0: 9/9/9/0
orange_carrier.inet.0: 9/9/9/0
[email protected]> show route advertising-protocol bgp 9.9.1.1
orange_carrier.inet.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 2.2.2.2/32 Self 4 100 I
* 3.3.3.3/32 Self 3 100 I
* 4.4.4.4/32 Self 2 100 I
* 5.5.5.5/32 Self 1 100 I
* 6.6.6.6/32 Self 100 I
* 169.254.10.2/31 Self 4 100 I
* 169.254.10.4/31 Self 3 100 I
* 169.254.10.6/31 Self 2 100 I
* 169.254.10.8/31 Self 100 I
[email protected]>
As you can see the prefixes we’re interested in are the left side orange carrier loopbacks. If we look on vMX9 we should see them in the bgp.l3vpn.0 table…
[email protected]> show route table bgp.l3vpn.0
bgp.l3vpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
1:1:2.2.2.2/32
*[BGP/170] 00:02:45, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:3.3.3.3/32
*[BGP/170] 00:02:45, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:4.4.4.4/32
*[BGP/170] 00:02:45, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:5.5.5.5/32
*[BGP/170] 00:02:45, MED 1, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:6.6.6.6/32
*[BGP/170] 00:02:45, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:169.254.10.2/31
*[BGP/170] 00:02:45, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:169.254.10.4/31
*[BGP/170] 00:02:45, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:169.254.10.6/31
*[BGP/170] 00:02:45, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
1:1:169.254.10.8/31
*[BGP/170] 00:02:45, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
[email protected]>
Perfect. But now what do we do with these? Our job as the green carrier is to help exchange these routes between the left and the right side orange carrier networks. And if we look at the routing instance table, we’ll see that this has happened…
[email protected]> show route table customer_carrier.inet.0 protocol bgp
customer_carrier.inet.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2.2.2.2/32 *[BGP/170] 00:09:09, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
3.3.3.3/32 *[BGP/170] 00:09:09, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
4.4.4.4/32 *[BGP/170] 00:09:09, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
5.5.5.5/32 *[BGP/170] 00:09:09, MED 1, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
6.6.6.6/32 *[BGP/170] 00:09:09, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.2/31 *[BGP/170] 00:09:09, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.4/31 *[BGP/170] 00:09:09, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.6/31 *[BGP/170] 00:09:09, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.8/31 *[BGP/170] 00:09:09, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
[email protected]>
So while that’s awesome that this worked as expected – the orange carrier routers in the left box won’t get these routes? Why not? Because we’re only running OSPF to those routers. In order for them to get the routes, we have to redistribute them into OSPF. Since I don’t really care about the link local addresses, let’s write a policy to only import the remote loopbacks on both vMX6 and vMX9…
vMX6
set policy-options prefix-list BGP-INTO-OSPF 9.9.9.9/32
set policy-options prefix-list BGP-INTO-OSPF 10.10.10.10/32
set policy-options prefix-list BGP-INTO-OSPF 11.11.11.11/32
set policy-options prefix-list BGP-INTO-OSPF 12.12.12.12/32
set policy-options prefix-list BGP-INTO-OSPF 13.13.13.13/32
set policy-options policy-statement BGP-INTO-OSPF term BGP from protocol bgp
set policy-options policy-statement BGP-INTO-OSPF term BGP from prefix-list BGP-INTO-OSPF
set policy-options policy-statement BGP-INTO-OSPF term BGP then accept
set policy-options policy-statement BGP-INTO-OSPF then reject
set routing-instances orange_carrier protocols ospf export BGP-INTO-OSPF
vMX9
set policy-options prefix-list BGP-INTO-OSPF 2.2.2.2/32
set policy-options prefix-list BGP-INTO-OSPF 3.3.3.3/32
set policy-options prefix-list BGP-INTO-OSPF 4.4.4.4/32
set policy-options prefix-list BGP-INTO-OSPF 5.5.5.5/32
set policy-options prefix-list BGP-INTO-OSPF 6.6.6.6/32
set policy-options policy-statement BGP-INTO-OSPF term BGP from protocol bgp
set policy-options policy-statement BGP-INTO-OSPF term BGP from prefix-list BGP-INTO-OSPF
set policy-options policy-statement BGP-INTO-OSPF term BGP then accept
set policy-options policy-statement BGP-INTO-OSPF then reject
set routing-instances orange_carrier protocols ospf export BGP-INTO-OSPF
So with this config our remote PEs should see all of the loopbacks of the orange carrier routers from the remote side…
[email protected]> show route table inet.0
inet.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2.2.2.2/32 *[Direct/0] 19:55:39
> via lo0.0
3.3.3.3/32 *[OSPF/10] 19:20:18, metric 1
> to 169.254.10.3 via ge-0/0/1.0
4.4.4.4/32 *[OSPF/10] 19:20:18, metric 2
> to 169.254.10.3 via ge-0/0/1.0
5.5.5.5/32 *[OSPF/10] 19:20:04, metric 3
> to 169.254.10.3 via ge-0/0/1.0
6.6.6.6/32 *[OSPF/10] 00:23:07, metric 4
> to 169.254.10.3 via ge-0/0/1.0
9.9.9.9/32 *[OSPF/150] 00:01:07, metric 0, tag 3489725928
> to 169.254.10.3 via ge-0/0/1.0
10.10.10.10/32 *[OSPF/10] 00:00:16, metric 5
> to 169.254.10.3 via ge-0/0/1.0
10.171.200.0/22 *[Static/5] 19:55:39
> to 192.168.127.100 via fxp0.0
11.11.11.11/32 *[OSPF/10] 00:00:16, metric 6
> to 169.254.10.3 via ge-0/0/1.0
12.12.12.12/32 *[OSPF/10] 00:00:16, metric 7
> to 169.254.10.3 via ge-0/0/1.0
13.13.13.13/32 *[OSPF/10] 00:00:16, metric 8
> to 169.254.10.3 via ge-0/0/1.0
169.254.10.2/31 *[Direct/0] 19:53:21
> via ge-0/0/1.0
169.254.10.2/32 *[Local/0] 19:53:21
Local via ge-0/0/1.0
169.254.10.4/31 *[OSPF/10] 19:20:18, metric 2
> to 169.254.10.3 via ge-0/0/1.0
169.254.10.6/31 *[OSPF/10] 19:20:18, metric 3
> to 169.254.10.3 via ge-0/0/1.0
169.254.10.8/31 *[OSPF/10] 19:20:04, metric 4
> to 169.254.10.3 via ge-0/0/1.0
192.168.127.0/24 *[Direct/0] 19:55:39
> via fxp0.0
192.168.127.2/32 *[Local/0] 19:55:39
Local via fxp0.0
224.0.0.2/32 *[LDP/9] 19:22:11, metric 1
MultiRecv
224.0.0.5/32 *[OSPF/10] 19:20:38, metric 1
MultiRecv
[email protected]>
Perfect! But now we have another problem. Just because we have a route to them – doesn’t mean we can build an LSP to them…
[email protected]> show route table inet.3
inet.3: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
3.3.3.3/32 *[LDP/9] 19:26:30, metric 1
> to 169.254.10.3 via ge-0/0/1.0
4.4.4.4/32 *[LDP/9] 19:26:28, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299792
5.5.5.5/32 *[LDP/9] 19:26:17, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299808
6.6.6.6/32 *[LDP/9] 00:29:19, metric 1
> to 169.254.10.3 via ge-0/0/1.0, Push 299824
[email protected]>
They’re still missing from our inet.3 table. If our goal is for this to look rather seamless to the orange carrier, then they are going to need inet.3 entries in order to build their LSPs. The fix for this is to modify the default LDP policy. In Juniper – by default – only loopback addresses are exported into LDP (which I think makes things awfully clean). And when I say exported – I mean put into the inet.3 table. We need to change this now to import other prefixes, in this case the ones we learned from BGP, into LDP as well. To do this, we modify the LDP egress policy inside of the VRF as follows…
vMX6 and vMX9
set policy-options policy-statement ldp-export term from_bgp from protocol bgp
set policy-options policy-statement ldp-export term from_bgp from prefix-list BGP-INTO-OSPF
set policy-options policy-statement ldp-export term from_bgp then accept
set policy-options policy-statement ldp-export term from_l0 from interface lo0.1
set policy-options policy-statement ldp-export term from_l0 then accept
set policy-options policy-statement ldp-export then reject
set routing-instances orange_carrier protocols ldp egress-policy ldp-export
In addition to validating that they came from BGP we can leverage the existing prefix-list we created BGP-INTO-OSPF to only capture the remote loopbacks we’re interested in. We also want to make sure that we include the local loopback interface as part of the export. Once this is applied on both CSC-PE nodes (vMX6 and vMX9) we should see all the entries we’d expect in the inet.3 table of all of the orange carrier routers. Let’s pick one at random to validate that assumption…
[email protected]> show route table inet.3
inet.3: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2.2.2.2/32 *[LDP/9] 00:04:23, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299856
3.3.3.3/32 *[LDP/9] 00:04:23, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299872
4.4.4.4/32 *[LDP/9] 00:04:23, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299888
5.5.5.5/32 *[LDP/9] 00:04:23, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299904
6.6.6.6/32 *[LDP/9] 00:04:23, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299920
9.9.9.9/32 *[LDP/9] 00:00:14, metric 1
> to 169.254.10.18 via ge-0/0/0.0, Push 299936
10.10.10.10/32 *[LDP/9] 21:54:56, metric 1
> to 169.254.10.18 via ge-0/0/0.0
12.12.12.12/32 *[LDP/9] 21:54:51, metric 1
> to 169.254.10.21 via ge-0/0/1.0
13.13.13.13/32 *[LDP/9] 21:54:46, metric 1
> to 169.254.10.21 via ge-0/0/1.0, Push 299808
[email protected]>
Great! So at this point, we have inet.3 resolution for LDP for all of the orange carrier routers as well as their LDP endpoints int he CSC-PE routing-instances!
So at this point – there’s really only one thing left to do and that’s configure a inet-vpn peering for our orange carrier. To do this, we’ll peer vMX2 directly to vMX13 since we now have reachability between the two loopbacks through the green carrier.
vMX2
set routing-options autonomous-system 65100
set protocols bgp group vpn type internal
set protocols bgp group vpn family inet-vpn unicast
set protocols bgp group vpn neighbor 13.13.13.13 local-address 2.2.2.2
set protocols bgp group vpn neighbor 13.13.13.13 peer-as 65100
vMX13
set routing-options autonomous-system 65100
set protocols bgp group vpn type internal
set protocols bgp group vpn family inet-vpn unicast
set protocols bgp group vpn neighbor 2.2.2.2 local-address 13.13.13.13
set protocols bgp group vpn neighbor 2.2.2.2 peer-as 65100
Once that’s configured – we should see the peering come up and the peers start automatically advertising any VPNv4 prefixes to each other…
[email protected]> show bgp summary
Groups: 2 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
bgp.l3vpn.0
2 2 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
13.13.13.13 65100 6 7 0 0 31 Establ
customer.inet.0: 2/2/2/0
bgp.l3vpn.0: 2/2/2/0
169.254.10.0 65001 2946 2947 0 0 22:02:20 Establ
customer.inet.0: 1/1/1/0
[email protected]> show route advertising-protocol bgp 13.13.13.13
customer.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 10.10.10.0/24 Self 100 65002 65001 I
* 169.254.10.0/31 Self 100 I
[email protected]>
Perfect! To help level set where we are with BGP configuration the image below describes the various peerings we have configured at this point. Note with the exception of the PE/CE eBGP peerings (which are largely of no consequence to the larger design) the two main BGP peerings we have are for a VPNv4 BGP peering between both the CSC-PE nodes of the green carrier and the PE nodes of the orange carrier…
So will our ping work at this point? Let’s go look and see…
left_client:~# ping 192.168.10.100 -c 5
PING 192.168.10.100 (192.168.10.100) 56(84) bytes of data.
--- 192.168.10.100 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 3999ms
left_client:~#
Darn. Ok – so what’s going on now? Let’s trace the label path and see if we can sort this out…
[email protected]> show route table customer.inet.0 192.168.10.100
customer.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.10.0/24 *[BGP/170] 00:03:42, localpref 100, from 13.13.13.13
AS path: 65004 65003 I, validation-state: unverified
> to 169.254.10.3 via ge-0/0/1.0, Push 16, Push 299904(top)
[email protected]>
Ok – so vMX2 thinks it should be pushing a VPN label and then what Im guessing is an LDP label…
[email protected]> show ldp database | grep 299904
299904 13.13.13.13/32
299904 13.13.13.13/32
[email protected]>
Yes, a LDP label onto the frame. So vMX3 will get a frame with a top label of 299904….
[email protected]> show route table mpls.0 label 299904
mpls.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299904 *[LDP/9] 00:17:28, metric 1
> to 169.254.10.5 via ge-0/0/1.0, Swap 299904
[email protected]>
Which it then swaps for the same label. So vMX4 will get 299904…
[email protected]> show route table mpls.0 label 299904
mpls.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299904 *[LDP/9] 00:18:05, metric 1
> to 169.254.10.7 via ge-0/0/1.0, Swap 299904
[email protected]>
Im sensing a theme here…
[email protected]> show route table mpls.0 label 299904
mpls.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299904 *[LDP/9] 00:18:35, metric 1
> to 169.254.10.9 via ge-0/0/1.0, Swap 299904
[email protected]>
Now what will vMX6 do with it?
[email protected]> show route table orange_carrier.mpls.0 label 299904
orange_carrier.mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299904 *[LDP/9] 00:19:03, metric 1, metric2 3
> to 169.254.10.11 via ge-0/0/1.0, label-switched-path to-vmx9
[email protected]>
Aha! It’s putting it into the RSVP tunnel to vMX9. Let’s see what label operation it’s doing though…
Note: Im doing fancy greps here to see the label operations just since it’s the easiest way to see the RSVP label operations.
[email protected]> show route table orange_carrier.mpls.0 label 299904 extensive | grep "Label operation"
Label operation: Swap 16, Push 299792(top)
Label operation: Swap 16
Label operation: Swap 16
[email protected]>
Ok – so vMX7 will get a label of 299792…
[email protected]> show route table mpls.0 label 299792 extensive | grep "Label operation"
Label operation: Swap 299792
[email protected]>
Which is then swaps for 299792 before it sends it to vMX8…
[email protected]> show route table mpls.0 label 299792 extensive | grep "Label operation"
Label operation: Pop
Label operation: Pop
[email protected]>
vMX8 would be the PHP router for the RSVP label domain so popping the label might make some sense here. So that would mean that vMX9 would get a label of 16…
[email protected]> show route table mpls.0 label 16
mpls.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
16 *[VPN/0] 02:49:55
> via lsi.1 (orange_carrier), Pop
[email protected]>
Huh. So it’s popping the label off and dumping it into the VRF. So what we have looks a lot like this…
The labels in orange are imposed by the orange carrier while the ones in green are by the green carrier. If this were a normal MPLS VPN scenario I’d expect the PHP router for the green carrier, which in our case is vMX8, to pop off the transport label and send a frame with a VPN label to vMX9. Now remember – this VPN we’re talking about here is in the green carrier – so it’s the VPN label that was imposed by vMX6 (label 16 in the above example). vMX9 would use that VPN label to sort out what VRF the traffic should land in. But the problem is that when it does that, there’s still yet another VPN label on the frame which was imposed by our orange carrier. We can see this in a packet capture between vMX8 and vMX9…
We’d expect (actually need) vMX9 to leave the bottom layer VPN label alone since that’s only relevant to the orange carrier. In this case, it’s popping off the green carriers VPN label and dumping the labelled frame into the orange_carrier VRF which is doing us no good at all. So why isn’t it stitching our LSPs together?
The problem as it turns out is related to the vrf-table-label command. The router in this case is largely doing what we asked it to do. We’ve talked about the vrf-table-label command in previous posts – and you might recall that it has two rather interesting impacts on how the router handles the VRF. First, when enabled, it assigns the same label for every prefix from that given VRF. We can see that if we look at the routes being advertised over to vMX6….
[email protected]> show route advertising-protocol bgp 6.6.1.1 extensive
orange_carrier.inet.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
* 9.9.9.9/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
Localpref: 100
AS path: [65000] I
Communities: target:1:1
* 10.10.10.10/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 1
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 11.11.11.11/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 2
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 12.12.12.12/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 3
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 13.13.13.13/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.16/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
Localpref: 100
AS path: [65000] I
Communities: target:1:1
* 169.254.10.18/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 2
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.20/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 3
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.22/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
[email protected]>
Second – it forces a local VRF lookup through an automatically created LSI interface. This allows us to overcome some of the problems that come along with having to do things like ARP lookups inside the VRF. However, in this case, it’s actually causing us some problems. If we force a lookup in the VRF – we can’t really precompute an end to end LSP path. So vMX9 in this case is going dump the traffic in the VRF through the LSI interface and try to resolve things there. That’s what we saw above. Unfortunately for us, in this case, that just won’t work since we have another VPN label on the frame for the orange carrier.
To fix this, lets start by removing vrf-table-label on both CSC-PE routers…
vMX6 and vMX9
delete routing-instances orange_carrier vrf-table-label
After making that change, let’s look at the routes being advertised to vMX6 again…
[email protected]> show route advertising-protocol bgp 6.6.1.1 extensive
orange_carrier.inet.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
* 9.9.9.9/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301216
Nexthop: Self
Flags: Nexthop Change
Localpref: 100
AS path: [65000] I
Communities: target:1:1
* 10.10.10.10/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 1
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 11.11.11.11/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 2
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 12.12.12.12/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 3
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 13.13.13.13/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.16/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
Localpref: 100
AS path: [65000] I
Communities: target:1:1
* 169.254.10.18/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 2
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.20/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 3
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
* 169.254.10.22/31 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301232
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
[email protected]>
Alright – do we have a unique label for each possible prefix? No – we dont. The thing is – locally allocated labels are a means for a router to say “if I get a label of X I know I need to do Y”. So in some cases, the router regardless of destination prefix will do the same thing. If we look at the destinations for the majority of the prefixes we’ll see that they are all using the label of 301232. So what does that label do?..
[email protected]> show route table mpls.0 label 301232
mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
301232 *[VPN/170] 00:04:15
> to 169.254.10.17 via ge-0/0/1.0, Pop
301232(S=0) *[VPN/170] 00:04:15
> to 169.254.10.17 via ge-0/0/1.0, Pop
[email protected]>
Pop again… Darn. So as you guessed – our ping still isn’t working. So let’s look at the orange carriers VRF table again…
[email protected]> show route table orange_carrier.inet.0
orange_carrier.inet.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both
2.2.2.2/32 *[BGP/170] 00:14:25, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
3.3.3.3/32 *[BGP/170] 00:14:25, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
4.4.4.4/32 *[BGP/170] 00:14:25, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
5.5.5.5/32 *[BGP/170] 00:14:25, MED 1, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
6.6.6.6/32 *[BGP/170] 00:14:26, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
9.9.9.9/32 *[Direct/0] 3d 02:39:56
> via lo0.1
10.10.10.10/32 *[OSPF/10] 00:22:44, metric 1
> to 169.254.10.17 via ge-0/0/1.0
11.11.11.11/32 *[OSPF/10] 00:22:44, metric 2
> to 169.254.10.17 via ge-0/0/1.0
12.12.12.12/32 *[OSPF/10] 00:22:44, metric 3
> to 169.254.10.17 via ge-0/0/1.0
13.13.13.13/32 *[OSPF/10] 00:22:44, metric 4
> to 169.254.10.17 via ge-0/0/1.0
169.254.10.2/31 *[BGP/170] 00:14:25, MED 4, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.4/31 *[BGP/170] 00:14:25, MED 3, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.6/31 *[BGP/170] 00:14:25, MED 2, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.8/31 *[BGP/170] 00:14:26, localpref 100, from 6.6.1.1
AS path: I, validation-state: unverified
> to 169.254.10.14 via ge-0/0/0.0, label-switched-path to-vmx6
169.254.10.16/31 *[Direct/0] 3d 02:39:56
> via ge-0/0/1.0
169.254.10.16/32 *[Local/0] 3d 02:39:56
Local via ge-0/0/1.0
169.254.10.18/31 *[OSPF/10] 00:22:44, metric 2
> to 169.254.10.17 via ge-0/0/1.0
169.254.10.20/31 *[OSPF/10] 00:22:44, metric 3
> to 169.254.10.17 via ge-0/0/1.0
169.254.10.22/31 *[OSPF/10] 00:22:44, metric 4
> to 169.254.10.17 via ge-0/0/1.0
224.0.0.2/32 *[LDP/9] 3d 02:39:56, metric 1
MultiRecv
224.0.0.5/32 *[OSPF/10] 3d 02:39:56, metric 1
MultiRecv
[email protected]>
Can anyone spot the issue? The routes that vMX9 knows about to reach the destination we need to get to (in this case we really want to get to the loopback of vMX13 (13.13.13.13)) are all know locally through OSPF. That wont work if we want to ride an LSP all the way there! We’ve run into this in a previous post – but the fix for this is to copy the routes from inet.3 into inet.0 since the router itself can’t resolve the destinations in inet.3 (only BGP can). To do this, we’ll use our mpls-forwarding command for MPLS traffic engineering…
vMX6 and vMX9
set routing-instances orange_carrier protocols mpls traffic-engineering mpls-forwarding
Once that’s in – our ping should start working…
root@left_client:~# ping 192.168.10.100 -c 5
64 bytes from 192.168.10.100: icmp_seq=1 ttl=57 time=3.86 ms
64 bytes from 192.168.10.100: icmp_seq=2 ttl=57 time=4.63 ms
64 bytes from 192.168.10.100: icmp_seq=3 ttl=57 time=4.12 ms
64 bytes from 192.168.10.100: icmp_seq=4 ttl=57 time=6.33 ms
64 bytes from 192.168.10.100: icmp_seq=5 ttl=57 time=4.68 ms
root@left_client:~#
And if we look at our routing table again – we’ll see that the destinations we’re interested in now have LDP routes for forwarding…
[email protected]> show route table orange_carrier.inet.0 protocol ldp
orange_carrier.inet.0: 21 destinations, 25 routes (21 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both
10.10.10.10/32 #[LDP/9] 00:01:31, metric 1
> to 169.254.10.17 via ge-0/0/1.0
11.11.11.11/32 #[LDP/9] 00:01:31, metric 1
> to 169.254.10.17 via ge-0/0/1.0, Push 299776
12.12.12.12/32 #[LDP/9] 00:01:31, metric 1
> to 169.254.10.17 via ge-0/0/1.0, Push 299792
13.13.13.13/32 #[LDP/9] 00:01:31, metric 1
> to 169.254.10.17 via ge-0/0/1.0, Push 299808
224.0.0.2/32 *[LDP/9] 3d 02:44:45, metric 1
MultiRecv
[email protected]>
This is one case where not having vrf-table-label on could be a good idea. Not having it on allows us to remove the constraint of one label per VRF which means that the router can allocate as many labels as it needs based on how many different actions it needs to perform. However, this also means that we’re chewing through labels. Look at this excerpt of the routes being advertised to vMX6 now…
* 9.9.9.9/32 (1 entry, 1 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301216
Nexthop: Self
Flags: Nexthop Change
Localpref: 100
AS path: [65000] I
Communities: target:1:1
@ 10.10.10.10/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301392
Nexthop: Self
Flags: Nexthop Change
MED: 1
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
@ 11.11.11.11/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301408
Nexthop: Self
Flags: Nexthop Change
MED: 2
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
@ 12.12.12.12/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301424
Nexthop: Self
Flags: Nexthop Change
MED: 3
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
@ 13.13.13.13/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301440
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
We’re now allocating a lot more unique labels than we were before. So is there a happy middle ground here? As it turns out – we can also get this to work with vrf-table-label enabled as well. So let’s turn that back on…
vMX6 and vMX9
set routing-instances orange_carrier vrf-table-label
Our ping should break once more – but let’s talk about why this isn’t working in greater detail. Let’s make sure we’re seeing the same behavior again…
[email protected]> show route advertising-protocol bgp 6.6.1.1 13.13.13.13/32 extensive
orange_carrier.inet.0: 21 destinations, 25 routes (21 active, 0 holddown, 0 hidden)
@ 13.13.13.13/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 16
Nexthop: Self
Flags: Nexthop Change
MED: 4
Localpref: 100
AS path: [65000] I
Communities: target:1:1 rte-type:0.0.0.0:1:0
[email protected]> show route table mpls.0 label 16
mpls.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
16 *[VPN/0] 00:00:49
> via lsi.768 (orange_carrier), Pop
[email protected]>
Yep – ok. Same problem. But let’s think about this for a moment. We talked earlier that routes from the orange carrier would be turned into VPNv4 routes on the CSC-PEs and then exported back into OSPF. Since we turned on our traffic engineering flag for mpls-forwarding we copied the routes we care about (the router loopbacks) from inet.3 into inet.0 right?..
[email protected]> show route table orange_carrier.inet.0 13.13.13.13/32
orange_carrier.inet.0: 21 destinations, 25 routes (21 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both
13.13.13.13/32 @[OSPF/10] 00:09:29, metric 4
> to 169.254.10.17 via ge-0/0/1.0
#[LDP/9] 00:09:29, metric 1
> to 169.254.10.17 via ge-0/0/1.0, Push 299808
[email protected]>
Remember that the mpls-forwarding option is the safest option to use since it only copies the routes for forwarding use. But in this case, that’s actually biting us. See – when the router converts those routes to VPNv4 routes it’s doing so with the OSPF route since that’s the one thats winning. We can tell this is the case by looking at the communities attached to the route. Notice above how the community list included rte-type:0.0.0.0:1:0. That’s actually a community that BGP uses so it knows what kind of OSPF route it’s dealing with. This is actually the “route type” field as described in RFC 4577. The layout looks like this…
AREA Number (4 bytes) : Route Type (1 byte) : Options (1 byte)
So we’re saying this came from area 0 (0.0.0.0) and is a route type of intra-area. In either case, this is an OSPF route that’s being translated and sent to vMX6 as a VPNv4 route. Since vMX9 believes that the OSPF route is the winning route – it still think that it should pop off what it believes to be the last label (16 (the green carrier VPN label)) and send it into the VRF toward the destination which it knows about through OSPF. To fix this, we need to tell the router to prefer the LDP route. We can do this by changing our traffic engineering flag to bgp-igp-both-ribs which will whole sale copy the routes from inet.3 into inet.0. Let’s try that out…
vMX6 and vMX9
set routing-instances orange_carrier protocols mpls traffic-engineering bgp-igp-both-ribs
Now if we look at the routing table, we should see the LDP routes in there and notice how they are no longer just for forwarding, they’re winning for both forwarding and routing use…
[email protected]> show route table orange_carrier.inet.0 13.13.13.13/32
orange_carrier.inet.0: 21 destinations, 25 routes (21 active, 0 holddown, 0 hidden)
@ = Routing Use Only, # = Forwarding Use Only
+ = Active Route, - = Last Active, * = Both
13.13.13.13/32 *[LDP/9] 00:00:28, metric 1
> to 169.254.10.17 via ge-0/0/1.0, Push 299808
[OSPF/10] 00:47:39, metric 4
> to 169.254.10.17 via ge-0/0/1.0
[email protected]>
This means that the route being sent to vMX6 is no longer the OSPF route…
[email protected]> show route advertising-protocol bgp 6.6.1.1 13.13.13.13/32 extensive
orange_carrier.inet.0: 21 destinations, 25 routes (21 active, 0 holddown, 0 hidden)
* 13.13.13.13/32 (2 entries, 2 announced)
BGP group green_carrier_vpn type Internal
Route Distinguisher: 1:1
VPN Label: 301568
Nexthop: Self
Flags: Nexthop Change
MED: 1
Localpref: 100
AS path: [65000] I
Communities: target:1:1
[email protected]>
Notice the lack of the rte-type community above. And if we look at what the router will do with that label…
[email protected]> show route table mpls.0 label 301568
mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
301568 *[VPN/170] 00:02:08
> to 169.254.10.17 via ge-0/0/1.0, Swap 299808
[email protected]> show ldp database instance orange_carrier
Input label database, 9.9.9.9:0--10.10.10.10:0
Labels received: 10
Label Prefix
300240 2.2.2.2/32
300256 3.3.3.3/32
300272 4.4.4.4/32
300288 5.5.5.5/32
299920 6.6.6.6/32
299936 9.9.9.9/32
3 10.10.10.10/32
299776 11.11.11.11/32
299792 12.12.12.12/32
299808 13.13.13.13/32
Output label database, 9.9.9.9:0--10.10.10.10:0
Labels advertised: 10
Label Prefix
301504 2.2.2.2/32
301472 3.3.3.3/32
301456 4.4.4.4/32
301488 5.5.5.5/32
299968 6.6.6.6/32
3 9.9.9.9/32
301328 10.10.10.10/32
301344 11.11.11.11/32
301360 12.12.12.12/32
301376 13.13.13.13/32
[email protected]>
It will swap that label with one the one it learned through LDP to get the traffic to vMX13. And our test ping should once again be working…
root@left_client:~# ping 192.168.10.100 -c 5
64 bytes from 192.168.10.100: icmp_seq=1 ttl=57 time=3.86 ms
64 bytes from 192.168.10.100: icmp_seq=2 ttl=57 time=4.63 ms
64 bytes from 192.168.10.100: icmp_seq=3 ttl=57 time=4.12 ms
64 bytes from 192.168.10.100: icmp_seq=4 ttl=57 time=6.33 ms
64 bytes from 192.168.10.100: icmp_seq=5 ttl=57 time=4.68 ms
root@left_client:~#
If we were to visualize what an end to end flow would look like it might look like this…
Note: Im only posting the graphic – we already posted an example of the commands used to get this information earlier.
You can see that the green labels imposed by the green carrier are simply for isolating the orange carrier traffic while it traverses the green carrier.
So there you have it. We’ve used CsC to connect discrete label domains. In the next post we’ll talk about how we can do CsC using BGP instead of OSPF and LDP in the CSC-PE VRF. Stay tuned!