LISP – Mobility (VM Mobility)

So it’s been awhile since I talked about LISP.  Studying for my CCIE has taken up a good chunk of my free time lately.  Some discussions at work have come up around it so I decided to setup a VM mobility lab to test out how LISP mobility works.  Let’s jump right into the scenario…

This is a similar setup to one that I’ve used in the past.  The goal here is simple.  Move one of the server (192.168.0.10) to Site B while the client computer (149.149.149.2) is downloading a file from it.  Once we do that, we’ll look at some of the limitations of this kind of setup.

Let’s start with the configs…

Router dcedge-1 (Site A)
interface GigabitEthernet1
description Border1
ip address 10.0.1.2 255.255.255.252
negotiation auto
!
interface GigabitEthernet2
description Server
mac-address 0000.0d0e.010c
ip address 192.168.1.1 255.255.255.0
negotiation auto
lisp mobility east_dc
lisp mobility west_dc
!
router lisp
locator-set west_dc
  IPv4-interface GigabitEthernet1 priority 10 weight 50
  exit
!
eid-table default instance-id 0
  dynamic-eid east_dc
   database-mapping 192.168.2.0/24 locator-set west_dc
   exit
  !
  dynamic-eid west_dc
   database-mapping 192.168.1.0/24 locator-set west_dc
   exit
  !
  ipv4 use-petr 10.0.3.1
  ipv4 itr map-resolver 10.0.3.1
  ipv4 itr
  ipv4 etr map-server 10.0.3.1 key alldcs
  ipv4 etr
  exit
!
exit    

Some of this configuration should look familiar.  We are using the core router as the PXTR.  The routing table on this router looks like this…

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C        10.0.1.0/30 is directly connected, GigabitEthernet1
L        10.0.1.2/32 is directly connected, GigabitEthernet1
S        10.0.3.1/32 [1/0] via 10.0.1.1
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/24 is directly connected, GigabitEthernet2
L        192.168.1.1/32 is directly connected, GigabitEthernet2

So this router only knows how to get to the loopback on the core router, the border 1 network, and it’s local subnet that has the servers in it.  All other addresses will be reached using the PXTR node.  You’ll also notice that addition of the ‘lisp mobility’ command on the interfaces facing the servers.  This effectively enables the LISP mobility service on the interfaces.  In addition, you’ll see a locator set configuration.  The ISRs use locator sets to describe their local RLOCs which in this case is on interface gig1.  We also have the eid-table definition.  This includes the dynamic EID space for both the east data center (192.168.2.0/24) and the west data center (192.168.1.0/24).  When we define them, we also configure which locator set should be used to reach them in the case that the EID happens to roam to this router.  This router is acting as a LISP XTR.

Router dcedge-2 (Site B)
interface GigabitEthernet1
description Border2
ip address 10.0.2.2 255.255.255.252
negotiation auto
!
interface GigabitEthernet2
description Server
mac-address 0000.0d0e.010c
ip address 192.168.2.1 255.255.255.0
negotiation auto
lisp mobility east_dc
lisp mobility west_dc
!
router lisp
locator-set east_dc
  IPv4-interface GigabitEthernet1 priority 10 weight 50
  exit
!
eid-table default instance-id 0
  dynamic-eid east_dc
   database-mapping 192.168.2.0/24 locator-set east_dc
   exit
  !
  dynamic-eid west_dc
   database-mapping 192.168.1.0/24 locator-set east_dc
   exit
  !
  ipv4 use-petr 10.0.3.1
  ipv4 itr map-resolver 10.0.3.1
  ipv4 itr
  ipv4 etr map-server 10.0.3.1 key alldcs
  ipv4 etr
  exit

Same deal on the Site B router dcedge-2.  Note that each router is using the same MAC address for it’s Gig2 interface.  This is crucial to the config working.  This router is also acting as a LISP XTR.  Now let’s take a look at the core router…

Coure Router
interface Loopback0
ip address 10.0.3.1 255.255.255.255
!
interface LISP0
!
interface FastEthernet0/0.100
encapsulation dot1Q 100
ip address 10.0.1.1 255.255.255.252
!
interface FastEthernet0/0.200
encapsulation dot1Q 200
ip address 10.0.2.1 255.255.255.252
!
interface FastEthernet0/1
ip address 75.75.75.1 255.255.255.252
duplex auto
speed auto
!
router lisp
site datacenters
  description All DCs
  authentication-key alldcs
  eid-prefix 192.168.1.0/24 accept-more-specifics
  eid-prefix 192.168.2.0/24 accept-more-specifics
  exit
!
map-cache 192.168.1.0/24 map-request
map-cache 192.168.2.0/24 map-request
ipv4 map-server
ipv4 map-resolver
ipv4 proxy-etr
ipv4 proxy-itr 10.0.3.1
ipv4 itr map-resolver 10.0.3.1
exit

I’m using subinterfaces on the switch to get the amount of interfaces I needed for this.  That part is pretty straight forward, the interesting part is the LISP configuration.  Note that there is only one site.  Since we are going to move EID prefixes between router’s they need to be in the same site.  The rest of the LISP config we should be used to seeing.  This router is acting a a LISP map server and PXTR.

In addition to the LISP config, I’ve allocated a separate vMotion network with split subnets using proxy-arp to facilitate the communication…

Each respective VMware server has a vMotion interface that has one end of a /30.  The actual interface on the server has a subnet mask of /29.  The switch has the other end of the /30 on a VLAN interface that has proxy-arp configured.  When the server reaches out to vMotion, the proxy-arp on the switch will take care of facilitating the communication.  I imagine that something similar can be done with the management interface so that you you can have servers in different physical data centers part of the same VMware data center on the same logical subnet.

The other piece of this I’m not going to talk about is storage.  I have a storage network shared between each host that has common iSCSI storage.  I’m not  a storage guy, but that would need to be considered in a production build.

Now let’s get to the interesting part, the actual vMotion.  In the initial test, I’m going to start downloading a file from the 192.168.0.10 server via HTTP, and then move it over to Site B mid download. I wish I could show a video of this, but pictures will have to do.

Starting the download

Start the vMotion
 

Download continuing
 

vMotion finishing
 

On the router at Site A we see the following messages…

*Jan  9 15:22:50.954: LISP: Processing received Map-Notify message from 10.0.3.1 to 10.0.1.2
*Jan  9 15:22:50.954: LISP: Processing Map-Notify no ID-included, no MS-RTR auth, 1 record, nonce 0x7EE5E672-0xF73EAC5F, key-id 1, auth-data-len 20, hash-function sha1
*Jan  9 15:22:50.954: LISP: Processing Map-Notify mapping record for IID 0 192.168.1.10/32, ttl 1440, action none, not authoritative, 1 locator
        10.0.2.2 pri/wei=10/50 lpR
*Jan  9 15:22:50.954: LISP-0: Local dynEID west_dc IID 0 prefix 192.168.1.10/32 RLOC 10.0.1.2 pri/wei=10/50, Map-Notify does not contain our locator, dyn-EID moved (rlocs: 1/1).
*Jan  9 15:22:50.954: LISP-0: DynEID IID 0 192.168.1.10 [west_dc:GigabitEthernet2] Moved away.
*Jan  9 15:22:50.954: LISP-0: DynEID IID 0 192.168.1.10 [west_dc:GigabitEthernet2] Delete.
*Jan  9 15:22:50.955: LISP-0: DynEID IID 0 192.168.1.10 Received ping 100% -> 0% ok event, deleting.
*Jan  9 15:22:51.078: LISP: Executing work item Net_tx:_0_v4_Map-Request_send[normal]
*Jan  9 15:22:51.078: LISP: Send map request type dyn-EID SMR
*Jan  9 15:22:51.078: LISP: Send map request for EID prefix IID 0 149.149.149.2/32
*Jan  9 15:22:51.078: LISP-0: AF IID 0 IPv4, Send SMR map-request for 192.168.1.10 to 10.0.1.1.
*Jan  9 15:22:51.078: LISP-0: AF IPv4, Sending probe map-request from 10.0.1.2 to 10.0.1.1 for EID 149.149.149.2/32, ITR-RLOCs 0, nonce 0xC95C1D85-0xF3CDFA19, SMR 192.168.1.10, DoNotReply.

The debug does a pretty good job of outlining the LISP process for moving the dynamic EID space.  Pretty slick huh?  We can see that the core router now knows that the specific /32 prefix is now off of Site B…

192.168.1.10/32, uptime: 00:06:26, expires: 23:54:07, via map-reply, complete
  Locator   Uptime    State      Pri/Wgt
  10.0.2.2  00:05:46  up          10/50

Now, here’s the interesting part.  What happens when we try to move a server that isn’t talking through the XTR?  For instance, if 192.168.1.10 was talking to 192.168.1.20 and we vMotioned 192.168.1.10 to the second DC things would break.  This is because we aren’t talking to the XTR so there’s no way for LISP to know what happened.  I think there are a couple of ‘hacks’ for this so maybe I’ll follow up with another post on how to make this work in the future.

Bottom line, this is an awesome use case for LISP.