In our last post we talked about how we can programmatically talk and listen to ExaBGP. By the end of the post, our Linux server was listening for BGP updates, processing them, and creating static routes based on the information it learned. We worked through some issues to get that far – but also recognized that we had a ways to go. In this post, we’ll start tackling some of the other issues that are lingering with this implementation. So let’s dive right in and start knocking these out!
The first issues I want to talk about isn’t actually an issue anymore – but it’s worth mentioning since we sort of solved it accidentally. At this point – we’ve only processed BGP update messages that have included a single router advertisement. Said more specifically – BGP update messages that included a single NLRI. A BGP update message can (and will) contain multiple NLRI’s so long as the path attributes are the same for all the prefixes. For folks not familiar with BGP – NLRI (Network layer reachability information) are basically the routes or prefixes that are being sent to us. If we go back and look at the BGP update message we received and decoded, we can see this outlined…
"announce": {
"ipv4 unicast": {
"10.10.10.10": [
{
"nlri": "10.10.10.12/30"
}
]
}
}
Notice that the next-hop contains a list of dictionaries which are the NLRI. So what happens if we advertise two prefixes at the time time? We can simulate this pretty easily just by resetting ExaBGP because we ended the last post by advertising a second prefix…
"announce": {
"ipv4 unicast": {
"10.10.10.6": [
{
"nlri": "9.9.9.9/32"
},
{
"nlri": "10.10.10.12/30"
}
]
}
}
So clearly we’re now receiving 2 NLRI in the same update message. But as I said – we’re already handling this because of the for loop we setup to loop through all received NLRI. The first time I played around with this – that detail slipped my mind and I was suddenly only processing a single route out of the update. So that’s an easy one – already checked off.
The next problem we’re going to have is route withdrawal (despite how many times I type that it still sounds like a more serious problem than it is). So at this point, if I kill the BGP peering on vMX1 to the server bgp_peer – those static routes will stick around until I restart ExaBGP. Not very dynamic . So to fix this – we need to consider also processing BGP withdraw messages. So do this – we can simply insert some logic to look at the update message we received and see if it’s a withdraw or announce. So we can add in the following to our script…
import sys
import json
from pyroute2 import IPRoute
ip = IPRoute()
for route in ip.get_routes():
if route.get_attr('RTA_PRIORITY') == 123:
prefix = route.get_attr('RTA_DST')+"/"+str(route["dst_len"])
ip.route("delete", dst=prefix, gateway=route.get_attr('RTA_GATEWAY'), priority=route.get_attr('RTA_PRIORITY'))
while True:
line = sys.stdin.readline().strip()
json_line = json.loads(line)
neighbor_ip = json_line['neighbor']['address']['peer']
for msg_type, attributes in json_line['neighbor']['message']['update'].items():
if msg_type == "announce":
for next_hop_address, nlri in json_line['neighbor']['message']['update']['announce']['ipv4 unicast'].items():
for prefix in nlri:
sys.stderr.write("Received a BGP update from " + neighbor_ip + '\n')
sys.stderr.write("For prefix " + prefix['nlri'] + '\n')
sys.stderr.write("With a next hop address of " + next_hop_address + '\n')
sys.stderr.flush()
ip.route("add", dst=prefix['nlri'], gateway=next_hop_address, priority=123)
elif msg_type == "withdraw":
sys.stderr.write("Better do something..." + '\n')
sys.stderr.flush()
So the logic is pretty simply – we’re now just adding a check to see if the update type was announce or withdraw. I’ll note that some of this code is getting a little ugly – but stick with me for now – we’ll clean things up once we get done adding functionality. For now – I think it’s important to keep things as explicit as possible so you can see what I’m doing.
So if we update our script, restart ExaBGP, and then remove the static route we added at the end of the last post, we should see the message we’re looking for in the logs…
Feb 11 03:23:52 bgp_peer exabgp[30926]: Better do something...
So what something should we do? Well – on withdraw we should likely pull the associated static route. So to do that, we can change our script once again to look like this…
import sys
import json
from pyroute2 import IPRoute
ip = IPRoute()
for route in ip.get_routes():
if route.get_attr('RTA_PRIORITY') == 123:
prefix = route.get_attr('RTA_DST')+"/"+str(route["dst_len"])
ip.route("delete", dst=prefix, gateway=route.get_attr('RTA_GATEWAY'), priority=route.get_attr('RTA_PRIORITY'))
while True:
line = sys.stdin.readline().strip()
json_line = json.loads(line)
neighbor_ip = json_line['neighbor']['address']['peer']
for msg_type, attributes in json_line['neighbor']['message']['update'].items():
if msg_type == "announce":
for next_hop_address, nlri in json_line['neighbor']['message']['update']['announce']['ipv4 unicast'].items():
for prefix in nlri:
sys.stderr.write("Received a BGP update announce from " + neighbor_ip + '\n')
sys.stderr.write("For prefix " + prefix['nlri'] + '\n')
sys.stderr.write("With a next hop address of " + next_hop_address + '\n')
sys.stderr.flush()
ip.route("add", dst=prefix['nlri'], gateway=next_hop_address, priority=123)
elif msg_type == "withdraw":
for route in json_line['neighbor']['message']['update']['withdraw']['ipv4 unicast']:
for nlri, prefix in route.items():
message = "Received a BGP update withdraw for " + prefix + " from " + neighbor_ip
sys.stderr.write(message + '\n')
ip.route("delete", dst=prefix, priority=123)
sys.stderr.flush()
So we’re doing pretty much the opposite of what we did when we added the route. We’re just deleting it now. So let’s update our script, restart the service, and then try the following on vMX2…
delete routing-options static route 9.9.9.9/32 next-hop 10.10.10.14
Now let’s check our logs for the ExaBGP service…
Feb 11 03:32:15 bgp_peer exabgp[12149]: Received a BGP update withdraw for 9.9.9.9/32 from 10.10.10.6
Cool – now let’s put it back…
set routing-options static route 9.9.9.9/32 next-hop 10.10.10.14
And once again check the logs…
Feb 11 03:32:20 bgp_peer exabgp[12149]: With a next hop address of 10.10.10.6
Feb 11 03:32:20 bgp_peer exabgp[12149]: For prefix 9.9.9.9/32
Feb 11 03:32:20 bgp_peer exabgp[12149]: Received a BGP update announce from 10.10.10.6
So our processing looks good! It’s also worth pointing out that because the way we’ve written our logic – we’re also in a good position for the cases where a BGP update message includes both a announce and a withdraw. So now we’re cooking with gas! Our Linux server should be able to handle anything. Let’s test it out by advertising a good chunk of routes at it…
set routing-options static route 10.20.10.0/24 discard
set routing-options static route 10.20.20.0/24 discard
set routing-options static route 10.20.30.0/24 discard
set routing-options static route 10.20.40.0/24 discard
set routing-options static route 10.20.50.0/24 discard
set routing-options static route 10.20.60.0/24 discard
set routing-options static route 10.20.70.0/24 discard
set routing-options static route 10.20.80.0/24 discard
set routing-options static route 10.20.90.0/24 discard
Drop that configuration into vMX2 and commit it. I like to open a new tab on the Linux server and run a watch to keep an eye on the routes (watch -n .1 ip route). If we look at the routing table on the Linux server – we should see all of these routes present…
user@bgp_peer:~$ ip route
default via 192.168.127.100 dev ens3 proto static
9.9.9.9 via 10.10.10.6 dev ens6 proto static metric 123
10.10.10.0/30 dev ens7 proto kernel scope link src 10.10.10.1
10.10.10.4/30 dev ens6 proto kernel scope link src 10.10.10.5
10.10.10.12/30 via 10.10.10.6 dev ens6 proto static metric 123
10.20.10.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.20.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.30.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.40.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.50.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.60.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.70.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.80.0/24 via 10.10.10.6 dev ens6 proto static metric 123
10.20.90.0/24 via 10.10.10.6 dev ens6 proto static metric 123
192.168.127.0/24 dev ens3 proto kernel scope link src 192.168.127.5
user@bgp_peer:~$
Alright – now let’s delete all of them – and add one more route to test the use case of deleting routes and adding them as part of the same update…
delete routing-options static route 10.20.10.0/24 discard
delete routing-options static route 10.20.20.0/24 discard
delete routing-options static route 10.20.30.0/24 discard
delete routing-options static route 10.20.40.0/24 discard
delete routing-options static route 10.20.50.0/24 discard
delete routing-options static route 10.20.60.0/24 discard
delete routing-options static route 10.20.70.0/24 discard
delete routing-options static route 10.20.80.0/24 discard
delete routing-options static route 10.20.90.0/24 discard
set routing-options static route 10.20.100.0/24 discard
And after committing our Linux routing table should look like…
user@bgp_peer:~$ ip route
default via 192.168.127.100 dev ens3 proto static
9.9.9.9 via 10.10.10.6 dev ens6 proto static metric 123
10.10.10.0/30 dev ens7 proto kernel scope link src 10.10.10.1
10.10.10.4/30 dev ens6 proto kernel scope link src 10.10.10.5
10.10.10.12/30 via 10.10.10.6 dev ens6 proto static metric 123
10.20.100.0/24 via 10.10.10.6 dev ens6 proto static metric 123
192.168.127.0/24 dev ens3 proto kernel scope link src 192.168.127.5
user@bgp_peer:~$
Perfect! So at this point – we’ve spent a ton of time dealing with the processing of routes we’ve received. What we’ve neglected was the Python script for advertising route’s to other BGP peers. As things stand – the Linux server really isn’t working as a BGP speaker. That is – it’s only advertising the routes we explicitly tell it to advertise and we have no means to add more routes to that list once ExaBGP is running. What if we had another BGP peer that’s only connection on the network was through our ExaBGP server? If ExaBGP never passed along the info it learned then we’d be in trouble as return routing would likely be largely broken. Now I’ll argue that ExaBGP’s goal in life was never to be a full blown BGP speaker – but it would be an interesting experiment to see if we could make it behave like one as much as possible.
But before we do that – I want to tackle one other change which we’ll address in the next post. This change should hopefully help us abstract talking to ExaBGP and make building/receiving route advertisements more palatable. So stay tuned!