MPLS 101 – Label Distribution Protocol (LDP)

In our last post, we saw a glimpse of what MPLS was capable of.  We demonstrated how routers could forward traffic to IP end points without looking at the IP header.  Rather, the routers performed label operations by adding (pushing), swapping, or removing (popping) the labels on and off the packet.  This worked well and meant that the core routers didn’t need to have IP reachability information for all destinations.  However – setting this up was time consuming.  We had to configure static paths and operations on each MPLS enabled router.  Even in our small example, that was time consuming and tedious.  So in this post we’ll look at leveraging the Label Distribution Protocol (LDP) to do some of the work for us.  For the sake of clarity, we’re going to once again start with a blank slate.  So back to our base lab that looked like this…

Note: I refer to the devices as routers 1-4 but you’ll notice in the CLI output that their names are vMX1-4.

Each device had the following base configuration…

interfaces {
    ge-0/0/0 {
        enable;
        unit 0 {
            family inet {
                address 10.2.2.0/31;
            }
        }
    }
    ge-0/0/1 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.0/31;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 1.1.1.1/32;
            }
        }
    }
}

interfaces {
    ge-0/0/0 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.1/31;
            }
        }
    }
    ge-0/0/1 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.2/31;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 2.2.2.2/32;
            }
        }
    }
}

interfaces {
    ge-0/0/0 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.3/31;
            }
        }
    }
    ge-0/0/1 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.4/31;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 3.3.3.3/32;
            }
        }
    }
}

interfaces {
    ge-0/0/0 {
        enable;
        unit 0 {
            family inet {
                address 10.1.1.5/31;
            }
        }
    }
    ge-0/0/1 {
        enable;
        unit 0 {
            family inet {
                address 10.2.2.2/31;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 4.4.4.4/32;
            }
        }
    }
}

Again – nothing exciting here. The only difference between this base config and the one in the previous post was that I’ve given all of the routers loopback addresses rather than just routers 1 and 4. But while we’re at it – we know from the previous post that to use MPLS we need to turn MPLS on each interface which will pass labelled traffic.  So as we did in our first post, let’s once again enable MPLS on all of the interfaces that we expect to pass labelled traffic…

interfaces {
    ge-0/0/1 {
        unit 0 {
            family mpls;
        }
    }
}
protocols {
    mpls {
        interface ge-0/0/1.0;
    }
}

interfaces {
    ge-0/0/0 {
        unit 0 {
            family mpls;
        }
    }
    ge-0/0/1 {
        unit 0 {
            family mpls;
        }
    }
}
protocols {
    mpls {
        interface ge-0/0/0.0;
        interface ge-0/0/1.0;
    }
}

interfaces {
    ge-0/0/0 {
        unit 0 {
            family mpls;
        }
    }
    ge-0/0/1 {
        unit 0 {
            family mpls;
        }
    }
}
protocols {
    mpls {
        interface ge-0/0/0.0;
        interface ge-0/0/1.0;
    }
}

interfaces {
    ge-0/0/0 {
        unit 0 {
            family mpls;
        }
    }
    ge-0/0/1 {
        unit 0 {
        }
    }
}
protocols {
    mpls {
        interface ge-0/0/0.0;
    }
}

So now we’re back to pretty much where we were in the last post before we started programming our static LSP.  As you’ll recall – that was tedious and we had to manually generate and track what labels we were using.  To overcome this – a group of protocols exist called “Label distribution Protocols” which take care of much of this tedious work for us.  The two primary label distribution protocols are LDP (Label Distribution Protocol) and RSVP (Resource Reservation Protocol).  In this post, we’ll be covering LDP since its the easiest to get up and running with and we’ll tackle RSVP in an upcoming post.

Enabling LDP is just as easy as enabling MPLS was.  We simply turn it on for any interface which we expect to handle labelled packets…

protocols {
    ldp {
        interface ge-0/0/1.0;
    }
}

protocols {
    ldp {
        interface ge-0/0/0.0;
        interface ge-0/0/1.0;
    }
}

protocols {
    ldp {
        interface ge-0/0/0.0;
        interface ge-0/0/1.0;
    }
}

protocols {
    ldp {
        interface ge-0/0/0.0;
    }
}

Now we can verify that LDP is on with various show commands…

root@vmx1> show ldp interface
Interface          Address                          Label space ID   Nbr   Next
                                                                    count  hello
ge-0/0/1.0         10.1.1.0                         1.1.1.1:0         1      1

root@vmx1>

Notice that router 1 believes it has a single LDP interface which is accurate. Also notice that LDP has identified the IP address of this interface. We’ll also note that the neighbor count (Nbr count) is listed as 1 so if we run the same command on router 2 we should see that it’s interfaces are now also enabled for LDP…

root@vmx2> show ldp interface
Interface          Address                          Label space ID   Nbr   Next
                                                                    count  hello
ge-0/0/0.0         10.1.1.1                         2.2.2.2:0         1      1
ge-0/0/1.0         10.1.1.2                         2.2.2.2:0         1      0

root@vmx2>

Indeed it is – so the neighbor count being 1 implies that there is an LDP relationship between router 1 and router 2 on their directly connected interfaces. If we performed the same command on router 3 and router 4 we would see similar output. You might be wondering what the Label space ID field is.  This is often referred to simply as the LDP ID and consists of two components separated by a colon.  The first piece is the neighbors transport address.  In the above output, you can see that each router is using it’s own local loopback address (2.2.2.2 in the case of router 2) for each of it’s LDP enabled interfaces. If we look at the LDP neighbors for router 2 we’ll see the LDP ID for it’s adjacent neighbors…

root@vmx2> show ldp neighbor
Address                             Interface       Label space ID     Hold time
10.1.1.0                            ge-0/0/0.0      1.1.1.1:0            14
10.1.1.3                            ge-0/0/1.0      3.3.3.3:0            12

root@vmx2>

As you can see – router 2 has two neighbors, 10.1.1.0 (ge-0/0/1 on router 1) and 10.1.1.3 (ge-0/0/0 on router 3).  Notice that each neighbors LDP ID starts with the loopback address of the neighboring router.  1.1.1.1 for router router 1 and 3.3.3.3 for router router 3.  With MPLS (especially on Juniper) it’s common for a nodes transport address to be the loopback address given that the transport address is expected to be unique.  The second part of the LDP ID in all cases appears to be 0.  So what does that mean?

The number after the colon indicates the means in which the node is allocating MPLS labels.  0 means that the node is using what is called a per-router (sometimes also called per-platform) label allocation method which is the default on Juniper devices.  This means that regardless of interface – all labels advertised for FECs need to be unique.  The other option is per-interface label space and means that in addition to labels you also track the interface they are associated with.  That is – with per-interface label space you could advertise the same label, out of two different interfaces, for two different FECs.  With per-router label space you would need to advertise two unique labels one for each FEC.  For now we’ll be dealing only with per-router label space.

Note: We talked a little bit about what FECs were in our last post.  In the majority of cases, a FEC is represented by a prefix.  That being said, it’s safe in most cases to use the two terms interchangeably.  However – FECs get labels assigned to them and in JunOS only a router’s loopback address, by default, is considered a FEC.  So when I refer to FECs throughout this post, Im referring to the routers /32 loopback address which is (naturally) a prefix.  I’ll try to only use the term prefix when referring to a prefix that is routed across an MPLS network and FEC when talking about a prefix that MPLS is aware of (has a label)

So we’ve verified our interfaces are enabled for LDP and we’ve even seen that we have some neighbors at this point.  The catch here is that there is a distinction between finding neighbors and building sessions.  Just because we have a neighbor doesn’t mean we have a valid LDP session.  LDP has merely discovered that there are other possible peers reachable through it’s interfaces by using LDP hellos.  The real magic happens once an LDP session is established.  If we look at router 2 we’ll see that we don’t currently have any active LDP sessions…

root@vmx2> show ldp session
  Address                           State       Connection  Hold time  Adv. Mode
1.1.1.1                             Nonexistent Closed        0          DU
3.3.3.3                             Nonexistent Closed        0          DU

root@vmx2>

So why is this? The routers are obviously talking otherwise we wouldn’t see any neighbors. One of the requirements for LDP is that a given router can reach the transport address of any LDP peers. In this case, router 2 doesn’t know about any of the other routers loopbacks…

root@vmx2> show route table inet.0

inet.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2.2.2.2/32         *[Direct/0] 3w0d 23:51:04
                    > via lo0.0
10.1.1.0/31        *[Direct/0] 3w1d 00:15:22
                    > via ge-0/0/0.0
10.1.1.1/32        *[Local/0] 3w1d 00:15:22
                      Local via ge-0/0/0.0
10.1.1.2/31        *[Direct/0] 3w1d 00:15:22
                    > via ge-0/0/1.0
10.1.1.2/32        *[Local/0] 3w1d 00:15:22
                      Local via ge-0/0/1.0
10.20.30.0/24      *[Direct/0] 3w6d 07:43:03
                    > via fxp0.0
10.20.30.136/32    *[Local/0] 3w6d 07:43:03
                      Local via fxp0.0
224.0.0.2/32       *[LDP/9] 3w0d 23:37:17, metric 1
                      MultiRecv

root@vmx2>

The simple fix here is to setup an IGP like OSPF and advertise the loopbacks into it. Let’s do that on all 4 routers now…

set protocols ospf area 0 interface ge-0/0/1
set protocols ospf area 0 interface lo0.0

set protocols ospf area 0 interface ge-0/0/0
set protocols ospf area 0 interface ge-0/0/1
set protocols ospf area 0 interface lo0.0

set protocols ospf area 0 interface ge-0/0/0
set protocols ospf area 0 interface ge-0/0/1
set protocols ospf area 0 interface lo0.0

set protocols ospf area 0 interface ge-0/0/0
set protocols ospf area 0 interface lo0.0

Notice that on the edge routers (routers 1 and 4) I dont put the user facing interfaces in OSPF. If I did, we wouldn’t need MPLS 🙂

We can quickly validate on one of our hosts that OSPF adjacencies have formed…

root@vmx2> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
10.1.1.0         ge-0/0/0.0             Full      1.1.1.1          128    36
10.1.1.3         ge-0/0/1.0             Full      3.3.3.3          128    36

root@vmx2>

And if we look at the LDP sessions we should now see they are operational…

root@vmx2> show ldp session
  Address                           State       Connection  Hold time  Adv. Mode
1.1.1.1                             Operational Open          24         DU
3.3.3.3                             Operational Open          23         DU

root@vmx2>

If you check all of your routers, you should see a similar story.  So what do we get out of an LDP session?  Let’s tack on the term extensive to our show ldp session command and see…

root@vmx2> show ldp session extensive
Address: 1.1.1.1, State: Operational, Connection: Open, Hold time: 25
  Session ID: 2.2.2.2:0--1.1.1.1:0
  Next keepalive in 9 seconds
  Active, Maximum PDU: 4096, Hold time: 30, Neighbor count: 1
  Neighbor types: discovered
  Keepalive interval: 10, Connect retry interval: 1
  Local address: 2.2.2.2, Remote address: 1.1.1.1
  Up for 2d 23:09:55
  Last down 2d 23:09:56 ago; Reason: connection error
  Capabilities advertised: none
  Capabilities received: none
  Protection: disabled
  Session flags: none
  Local - Restart: disabled, Helper mode: enabled
  Remote - Restart: disabled, Helper mode: enabled
  Local maximum neighbor reconnect time: 120000 msec
  Local maximum neighbor recovery time: 240000 msec
  Local Label Advertisement mode: Downstream unsolicited
  Remote Label Advertisement mode: Downstream unsolicited
  Negotiated Label Advertisement mode: Downstream unsolicited
  MTU discovery: disabled
  Nonstop routing state: Not in sync
  Next-hop addresses received:
    10.1.1.0
  Queue depth: 0
Message type               Total                     Last 5 seconds
                       Sent      Received          Sent      Received
Initialization            1             1             0             0
Keepalive             25611         25758             1             1
Notification              0             0             0             0
Address                   1             1             0             0
Address withdraw          0             0             0             0
Label mapping             4             4             0             0
Label request             0             0             0             0
Label withdraw            0             0             0             0
Label release             0             0             0             0
Label abort               0             0             0             0

Address: 3.3.3.3, State: Operational, Connection: Open, Hold time: 21
  Session ID: 2.2.2.2:0--3.3.3.3:0
  Next keepalive in 9 seconds
  Passive, Maximum PDU: 4096, Hold time: 30, Neighbor count: 1
  Neighbor types: discovered
  Keepalive interval: 10, Connect retry interval: 1
  Local address: 2.2.2.2, Remote address: 3.3.3.3
  Up for 2d 23:10:01
  Capabilities advertised: none
  Capabilities received: none
  Protection: disabled
  Session flags: none
  Local - Restart: disabled, Helper mode: enabled
  Remote - Restart: disabled, Helper mode: enabled
  Local maximum neighbor reconnect time: 120000 msec
  Local maximum neighbor recovery time: 240000 msec
  Local Label Advertisement mode: Downstream unsolicited
  Remote Label Advertisement mode: Downstream unsolicited
  Negotiated Label Advertisement mode: Downstream unsolicited
  MTU discovery: disabled
  Nonstop routing state: Not in sync
  Next-hop addresses received:
    10.1.1.3
    10.1.1.4
  Queue depth: 0
Message type               Total                     Last 5 seconds
                       Sent      Received          Sent      Received
Initialization            1             1             0             0
Keepalive             25611         25680             0             0
Notification              0             0             0             0
Address                   1             1             0             0
Address withdraw          0             0             0             0
Label mapping             4             4             0             0
Label request             0             0             0             0
Label withdraw            0             0             0             0
Label release             0             0             0             0
Label abort               0             0             0             0

root@vmx2>

As you can see there’s quite a lot of info here but lets focus on one item for now – the received next-hop addresses for each peer.  These are learned as part of the LDP session establishment. If we looked at the output of show ldp neighbor we’d see that a couple of these addresses (for directly connected interfaces) appeared in that output as well.  However, this output gives us the complete list of possible IP addresses for each peer.  We’ll get to why these are important shortly.

So now that our sessions are established the next thing LDP will start doing for us is sending LDP label mappings.  LDP will automatically assign a label to each FEC that a router knows about and advertise that label and FEC (typically a prefix) to all of it’s peers.  LDP does this the easiest way possible – by flooding the advertisements out of every LDP enabled interface.   You might recall from the last post that Juniper devices by default advertise their own /32 loopback address as a FEC.  So in our case, each router already knows about a FEC to advertise since each router has a local loopback address.  In order to advertise the FEC with a LDP label mapping message the router needs to assign a label to the loopback.  To see what the router assigned, we can take a look at the LDP database…

root@vmx1> show ldp database
Input label database, 1.1.1.1:0--2.2.2.2:0
Labels received: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Output label database, 1.1.1.1:0--2.2.2.2:0
Labels advertised: 4
  Label     Prefix
      3      1.1.1.1/32
 299856      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

root@vmx1>

The top table Input label database shows the labels the vMX1 has received.  These are the labels that router 1 will push onto a frame’s label stack when sending traffic to these destination FECs through a given node.  The “given node” is an important distinction to make here.  Since labels are unique per router we need to track which labels we learn through which LDP peering session.  Notice in the output about that the router is showing the input and output label database for the peering session between 1.1.1.1:0 and 2.2.2.2:0.  As we’ll see later on – since LDP floods you’ll receive label mappings on any LDP enabled interface but that doesn’t necessarily mean that the router will use them in forwarding.  Let’s look at the output of the LDP database on the other 3 nodes as well…

root@vmx2> show ldp database
Input label database, 2.2.2.2:0--1.1.1.1:0
Labels received: 4
  Label     Prefix
      3      1.1.1.1/32
 299856      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Output label database, 2.2.2.2:0--1.1.1.1:0
Labels advertised: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Input label database, 2.2.2.2:0--3.3.3.3:0
Labels received: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

Output label database, 2.2.2.2:0--3.3.3.3:0
Labels advertised: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

root@vmx2>

root@vmx3> show ldp database
Input label database, 3.3.3.3:0--2.2.2.2:0
Labels received: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Output label database, 3.3.3.3:0--2.2.2.2:0
Labels advertised: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

Input label database, 3.3.3.3:0--4.4.4.4:0
Labels received: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
 299776      3.3.3.3/32
      3      4.4.4.4/32

Output label database, 3.3.3.3:0--4.4.4.4:0
Labels advertised: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

root@vmx3>

root@vmx4> show ldp database
Input label database, 4.4.4.4:0--3.3.3.3:0
Labels received: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

Output label database, 4.4.4.4:0--3.3.3.3:0
Labels advertised: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
 299776      3.3.3.3/32
      3      4.4.4.4/32

root@vmx4>

So let’s see what this looks like visually just in terms of the set of labels that router 1 is advertising…

This is where things can get a little confusing if you aren’t used to working with MPLS.  Let’s just talk about how to reach the 1.1.1.1/32 prefix to start with.  In this case router 1 is advertising a label of 3 to router 2 for the FEC 1.1.1.1/32.  3 is a special label which signals the receiving router to perform PHP (Penultimate Hop Pop) so that the sending router receives a native packet and can directly process that rather than dealing with an MPLS header.

Note: Label 3 falls into the reserved label range and is actually called the “implicitnull” label.  If a upstream router receives this label it automatically knows to pop the top label off of the label stack before forwarding the frame downstream (I talk about upstream/downstream below).

Router 2 receives the label mapping advertisement for the 1.1.1.1/32 FEC from router 1 and then assigns a label to the FEC locally.  In this case, it assigns a label of 299904 to that FEC and then advertises that label to all of its LDP peers.

Note: While not pictured, you might have caught from the output that router 2 advertises the label not only to router 3, but also back to router 1.  This is because LDP is flooding and based on the output we can see that no split-horizon rules are in play for LDP.  Also note that it advertises the same label for the FEC since we’re using a per-router(platform) label space.    

Router 3 receives the LDP label mapping from router 2 for FEC1.1.1.1/32 and assigns a label of 299856 to the FEC locally.  It then advertises the label mapping to all of its LDP peers.   This method of advertising labels is referred to as “downstream unsolicited” since each LSR (Label Switching Router (a fancy term for an MPLS enabled router)) is sending all of the label mappings to each of its peers without being asked to.  This is the default means of label distribution with LDP.  So the unsolicited piece makes sense because the LSRs are sending LDP label mapping advertisements whether you like it or not.  But what does downstream mean?

I’ll admit that the upstream and downstream terminology was initially confusing for me when I started using MPLS because it’s all relative.  The key to understanding the terms “upstream” and “downstream” in the MPLS world is that they refer to a specific FEC and how an LSR perceives that FEC.  So in our example above – router 2 would be considered to be an upstream router (or LSR, I use the terms interchangeably here) to router 1 for the FEC 1.1.1.1/32.  Likewise, router 2 would be a downstream LSR to router 3 for the same FEC.  The bottom line is that the closer you are to the originator of the FEC – the further downstream you are.  Label advertisement is downstream to upstream.  So in this case – since the downstream router is advertising label mapping upstream without being asked for it – it’s called downstream unsolicited label distribution.  So if label advertisements flow from downstream to upstream (the blue dotted line) it also makes sense that actual traffic flows from upstream to downstream (the green dotted line).

Now let’s take a minute to address the LDP flooding and lack of split-horizon rules we saw above.  For instance, let’s look at the LDP database on router 2 more closely…

root@vmx2> show ldp database
Input label database, 2.2.2.2:0--1.1.1.1:0
Labels received: 4
  Label     Prefix
      3      1.1.1.1/32
 299856      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Output label database, 2.2.2.2:0--1.1.1.1:0
Labels advertised: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Input label database, 2.2.2.2:0--3.3.3.3:0
Labels received: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

Output label database, 2.2.2.2:0--3.3.3.3:0
Labels advertised: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

root@vmx2>

This is the oddity we saw above when we looked at the LDP database on router 1.  Why is router 2 receiving a LDP mapping for 3.3.3.3/32 from both it’s peering to router 1 and router 3? Clearly, router 2 only has a single path to 3.3.3.3/32 and that’s toward router 3. So why is router 1 advertising a path to 3.3.3.3/32 as well? The reason is that LDP floods completely indiscriminately. In other words, its job is just to tell every peer about every FEC it knows about. So in this case, this is what’s happening with the 3.3.3.3/32 FEC…

So let’s walk  through this….

1. Router 3 has the FEC of 3.3.3.3/32 locally which it tells all of it’s neighbors about by advertising the prefix to all of it’s established LDP peers with a label of 3.(only showing the conversation to router 2 in the diagram).
2. Router 2 receives the label mapping, assigns a local label of 299872 to the FEC and then advertises that label mapping to all of it’s LDP peers.  All of it’s LDP peers happens to include router 1 AND router 3 which was the originator of the FEC to start with.  At this point, LDP just told router 3 that it can reach the FEC of 3.3.3.3/32 by passing router 2 a label of 299872.
3. Router 1 has also received the label mapping for 3.3.3.3/32 and happily told router 2 that it can reach the FEC by passing it a label of 299872 as well.

So this is a mess.  Moreover – this sort of advertising is looking like it’s going to cause nothing but loops.  So how does a router know which way is the correct downstream path to reach a FEC?  The answer is that each LSR relies on the underlying IGP (in this case OSPF) to determine the right label to use.   So let’s look at the inet.0 routing table on router 2 for the destination 3.3.3.3/32 to make this determination…

root@vmx2> show route table inet.0 3.3.3.3/32

inet.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

3.3.3.3/32         *[OSPF/10] 2d 23:53:57, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0

root@vmx2>

The RFC for LDP says that…

An LSR receiving a Label Mapping message from a downstream LSR for a Prefix SHOULD NOT use the label for forwarding unless its routing table contains an entry that exactly matches the FEC Element.

In other words, we need an entry in the inet.0 table for the FEC in order to even consider using the label we received to try and reach it.  As we can see above, we do have a route for 3.3.3.3/32 in inet.0 table so we’ve passed that requirement.  The problem now is how do we determine which label to use?  The problem is that the information required to make this assessment lives in a couple of different places.  Recall that when we looked at the output of show ldp session extensive on router 2 that we learned all of the MPLS enabled addresses of each LDP peer.  In this case, router 2 learned that router 1 has interface 1.1.1.0 and router 3 has interfaces 10.1.1.3 and 10.1.1.4.  If we consult the inet.0 entry for 3.3.3.3/32 we can see that it’s reachable through a physical next hop of 10.1.1.3 which the router knows is associated with the LDP peering toward router 3.  That means that router 3 is the true downstream LSR for this prefix and that router 2 should use the label it learned from router 3 to reach that FEC.  The logic looks like this…

So you now might be wondering why the inet.0 table on all of the routers doesn’t show any label operations since we just validated that the routers at least have some usable labels.  As in the first post, while the routers know what labels to use to reach a specific FEC, they still dont have a means to get traffic into an LSP to reach a particular FEC.   As we talked about in the first post, forwarding entries for MPLS are in the inet.3 routing table on Juniper equipment.  Recall that this is the routing table where Juniper stores it’s FECs and is sometimes called the FEC mapping table.  More specifically, it’s a recursive lookup table leveraged by BGP.  So let’s see what router 1 has in it’s inet.3 table…

root@vmx1> show route table inet.3

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2.2.2.2/32         *[LDP/9] 05:22:34, metric 1
                    > to 10.1.1.1 via ge-0/0/1.0
3.3.3.3/32         *[LDP/9] 05:22:34, metric 1
                    > to 10.1.1.1 via ge-0/0/1.0, Push 299872
4.4.4.4/32         *[LDP/9] 05:22:34, metric 1
                    > to 10.1.1.1 via ge-0/0/1.0, Push 299888

root@vmx1>

Ahah!  The inet.3 table does list some label operations to reach all of the FECs we currently know about.  If we compare these to the LDP input database on router 1, we’ll see that these entries line up perfectly with the entries in the LDP peering session between router 1 and router 2 that we examined earlier (output above).  Namely…

• To reach 2.2.2.2 on router 2 we send the traffic out of the ge-0/0/1.0 interface towards 10.1.1.1.  Notice that we do not push a label for this action.  This is because router 1 was told through its LDP peering session that to reach 2.2.2.2 it should use a label of 3.  3 indicates that the penultimate (the second to last) router should pop the label and deliver the traffic normally.  Since this is the second to last router it simply doesnt impose a label.
• To reach 3.3.3.3 on router 3 we send the traffic out of the ge-0/0/1.0 interface towards 10.1.1.1.  In this case, we do impose a label of 299872.  This is the label we learned from the upstream router (router 2) for the FEC/prefix 3.3.3.3/32.  You can confirm that by consulting the LDP database on the peering session between router 1 and router 2.  You’ll see that router 2 advertised a label of 299872 to reach the prefix of 3.3.3.3/32.
• To reach 4.4.4.4 on router 3 we send the traffic out of the ge-0/0/1.0 interface towards 10.1.1.1.  In this case, we do impose a label of 299888.  This is the label we learned from the upstream router (router 2) for the FEC/prefix 3.3.3.3/32.  You can confirm that by consulting the LDP database on the peering session between router 1 and router 2.  You’ll see that router 2 advertised a label of 299888 to reach the prefix of 3.3.3.3/32.

So this all seems to be lining up but let’s do the exercise one more time on router 2 so that we’re sure it makes sense.  So let’s look at the inet.3 table on router 2…

root@vmx2> show route table inet.3

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[LDP/9] 2d 22:02:52, metric 1
                    > to 10.1.1.0 via ge-0/0/0.0
3.3.3.3/32         *[LDP/9] 2d 22:02:58, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0
4.4.4.4/32         *[LDP/9] 2d 22:02:58, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0, Push 299808

root@vmx2>

Notice that in this case, only one of the FECs impose a label (4.4.4.4./32).  That’s because the other LDP peers (router 1 and 3) are directly connected so they advertise router 2 a label of 3 which you’ll recall is the implicit null label and asks the upstream router to pop the label before forwarding the frame downstream.  The only other label we learned was from router 3 and was for how to reach the 4.4.4.4/32 FEC.  In this case router 3 is saying “Sending the frame out of your ge-0/0/1.0 interface and push a label of 299808 onto the frame”.  If we consult the output of show ldp database above from router 3 we’ll see that 299808 is indeed the label router 3 advertised to router 2 for that FEC/prefix.

One thing I hope you’ve noticed is that the labels here are locally significant.  If you replicate this lab on your own, there’s a chance that you’ll see each router allocate the same label for the same prefix.  That would work just as well.  Since each router keeps track of it’s own set of labels for each specific peering session there’s no reason they can’t overlap.

Looking back at the inet.0 table we’ll see that the router wants to use OSPF to reach all of these prefixes and doesn’t list a label to be pushed…

root@vmx1> show route table inet.0

inet.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[Direct/0] 3w2d 01:28:40
                    > via lo0.0
2.2.2.2/32         *[OSPF/10] 22:06:52, metric 1
                    > to 10.1.1.1 via ge-0/0/1.0
3.3.3.3/32         *[OSPF/10] 01:27:29, metric 2
                    > to 10.1.1.1 via ge-0/0/1.0
4.4.4.4/32         *[OSPF/10] 01:26:22, metric 3
                    > to 10.1.1.1 via ge-0/0/1.0
10.1.1.0/31        *[Direct/0] 3w2d 01:28:40
                    > via ge-0/0/1.0
10.1.1.0/32        *[Local/0] 3w2d 01:28:40
                      Local via ge-0/0/1.0
10.1.1.2/31        *[OSPF/10] 22:06:52, metric 2
                    > to 10.1.1.1 via ge-0/0/1.0
10.1.1.4/31        *[OSPF/10] 01:27:29, metric 3
                    > to 10.1.1.1 via ge-0/0/1.0
10.2.2.0/31        *[Direct/0] 3w2d 01:28:40
                    > via ge-0/0/0.0
10.2.2.0/32        *[Local/0] 3w2d 01:28:40
                      Local via ge-0/0/0.0
10.20.30.0/24      *[Direct/0] 4w0d 09:40:23
                    > via fxp0.0
10.20.30.135/32    *[Local/0] 4w0d 09:40:23
                      Local via fxp0.0
224.0.0.2/32       *[LDP/9] 3w2d 00:50:16, metric 1
                      MultiRecv
224.0.0.5/32       *[OSPF/10] 22:07:40, metric 1
                      MultiRecv

root@vmx1>

So despite having the entries in inet.3 they arent being used since we dont have any forwarding entries in inet.0.  The solution to this is to put recursive routes in the inet.0 table which will force the router to reference the inet.3 table.  In a real environment this would be done with BGP but ,like we did in the first post in this series, we’re going to leverage static routes to keep things simple.  Let’s use static routes to make our client subnets reachable.  To do this, we simply add the following routes on router 1 and router 4…

set routing-options static route 10.2.2.2/31 next-hop 4.4.4.4 resolve

set routing-options static route 10.2.2.0/31 next-hop 1.1.1.1 resolve

At this point if we consult the inet.0 table on router 1 and router 4 we should see that the client subnets exist and that each router is showing a label operation to reach the destination prefix…

root@vmx1> show route table inet.0 10.2.2.2/31

inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.2.2.2/31        *[Static/5] 00:06:27, metric2 1
                    > to 10.1.1.1 via ge-0/0/1.0, Push 299888

root@vmx1>

root@vmx4> show route table inet.0 10.2.2.0/31

inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.2.2.0/31        *[Static/5] 00:06:51, metric2 1
                    > to 10.1.1.4 via ge-0/0/0.0, Push 299856

root@vmx4>

If you look at the ingress LDP database on router 1 and router 4 you’ll find that the labels being pushed to reach the client subnets line up with the label the router learned for that prefix or FEC…

root@vmx1> show ldp database
Input label database, 1.1.1.1:0--2.2.2.2:0
Labels received: 4
  Label     Prefix
 299904      1.1.1.1/32
      3      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

Output label database, 1.1.1.1:0--2.2.2.2:0
Labels advertised: 4
  Label     Prefix
      3      1.1.1.1/32
 299856      2.2.2.2/32
 299872      3.3.3.3/32
 299888      4.4.4.4/32

root@vmx1>

root@vmx4> show ldp database session 3.3.3.3
Input label database, 4.4.4.4:0--3.3.3.3:0
Labels received: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
      3      3.3.3.3/32
 299808      4.4.4.4/32

Output label database, 4.4.4.4:0--3.3.3.3:0
Labels advertised: 4
  Label     Prefix
 299856      1.1.1.1/32
 299840      2.2.2.2/32
 299776      3.3.3.3/32
      3      4.4.4.4/32

root@vmx4>

And if you check connectivity on each client, you’ll see that they are now able to ping each other successfully so long as they have a route pointing to their directly connected router interface for the remote client network. To validate we’re doing MPLS, let’s start a ping on the top client to the bottom client and do a capture on the link between router 2 and router 3.  Based on the information we know from looking at the LDP database, we can determine what labels we should see in the capture as the traffic traverses the link in between routers 2 and 3.  For the ICMP echo request I’d expect to see a label value of 299808 and for the ICMP echo reply I’d expect to see a value of 299904.  Let’s see if we’re right…

If that didn’t add up to you, look at the LDP database tables on router 2 and 3 again.  The important thing to remember is upstream versus downstream.  In our case, looking between router 2 and 3 the connectivity looks like this…

The blue arrows show the label advertisements and the green arrows show the actual data path label use.  Don’t be put off if this seems overly confusing.  It’s a relatively simple abstraction but one that cant be hard to follow without walking through it step by step.  Fortunately, there’s an easier way to see some of this as well.  What we’ve been looking at so far has mostly been the pieces of control plane that make MPLS work.  The real combination of all these efforts is displayed in the mpls.0 table.  Let’s take a look at that on router 2 so you can see what I mean…

root@vmx2> show route table mpls.0

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 3w5d 14:58:19, metric 1
                      to table inet.0
0(S=0)             *[MPLS/0] 3w5d 14:58:19, metric 1
                      to table mpls.0
1                  *[MPLS/0] 3w5d 14:58:19, metric 1
                      Receive
2                  *[MPLS/0] 3w5d 14:58:19, metric 1
                      to table inet6.0
2(S=0)             *[MPLS/0] 3w5d 14:58:19, metric 1
                      to table mpls.0
13                 *[MPLS/0] 3w5d 14:58:19, metric 1
                      Receive
299872             *[LDP/9] 3d 05:46:46, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0, Pop
299872(S=0)        *[LDP/9] 3d 05:46:46, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0, Pop
299888             *[LDP/9] 3d 05:46:46, metric 1
                    > to 10.1.1.3 via ge-0/0/1.0, Swap 299808
299904             *[LDP/9] 3d 05:46:40, metric 1
                    > to 10.1.1.0 via ge-0/0/0.0, Pop
299904(S=0)        *[LDP/9] 3d 05:46:40, metric 1
                    > to 10.1.1.0 via ge-0/0/0.0, Pop

root@vmx2>

This table describes all label operations that the router will take on labeled packets.  This table confirms what we saw above.  If router 2 receives a frame with label 299888 it will swap the label with 299808 and forward it out of the routers ge-0/0/1.0 interface toward router 3.  This would be the ICMP echo request coming from the top client headed to the bottom client.  On the return, router 2 told router 3 that it needed a label of 299904 in order to reach the downstream DEC of 1.1.1.1/32.  We can see that if router 2 receives a frame with label 299904 it will pop the label and forward the native frame to router 1 (Pen-ultimate hop popping).

This wasn’t meant to be an exhaustive view of LDP but I hope it gave you an idea of what LDP is capable of.  I also hope this demonstrated how necessary a label distribution protocol is to any MPLS network.  Doing static LSPs just isn’t feasible for sort of scalable deployment so you’ll want to make sure you’re comfortable with deploying something like LDP to help out with label assignment and distribution.  In the next post, we’ll layer on BGP so we don’t need to use static routes and talk about how LDP handles failures and reroutes.  Stay tuned!