Base Config – Nexus 5K / 2K

So by this point I’m going to assume that you know the role that Nexus 5k and 2k series switches play in the data center.  The core of it all is that the 5k’s are the brain of the operation with 2ks acting as individual line cards in a virtual chassis.  The combination of the two gives us a distributed switching environment that supports FCOE and simplifies switch management. 

So you’ve just unboxed the equipment and you want to get it all configured.  Let’s start from scratch….

Configure the Management0 interface
The Nexus 5k’s come with a single out of band management interface that can be used to remotely administer the system.  I say out of band because the interface is in its very own VRF segregating it from the default VRF.  If you aren’t comfortable with the concept of VRF’s I would look into them, they play a large role in the Nexus platform. 

-Insert your relevant information between <>
-Console prompts are shown in green

Nexus5k# config t
Nexus5k(config)# interface mgmt0
Nexus5k(config-if)# ip address <Enter your IP> 
Nexus5k(config-if)# exit

Now that the IP is configured, we should have layer 2 connectivity to the Nexus.  But, we won’t have layer 3 until we give it a default route. 

Nexus5k# config t
Nexus5k(config)# vrf context management
Nexus5k(config-vrf)# ip route 0.0.0.0/0 <Default Gateway>
Nexus5k(config-vrf)# exit

Notice that the default route was defined under the VRF for the management interface.  So when I want to test connectivity from the Nexus to other subnets using ping, I need to specify the VRF.  If I don’t, it will use the default VRF and my pings won’t go anywhere since there isn’t a route.

Nexus5k# ping <Your Workstation IP> vrf management

Enable the required features
As you probably know, none of the features on a Nexus are turned on until you tell the switch to enable them.  This is done using the feature command.  For my testing, I enabled the following features…

Nexus5k# config t
Nexus5k(config)# feature fcoe
(Note: This enables FCOE on the switch)
Nexus5k(config)# feature telnet
(Note: This enables remote administration of the switch through telnet)
Nexus5k(config)# feature udld
(Note: This enables unidirectional link detection)
Nexus5k(config)# feature interface-vlan
(Note: This enables VLAN interface configuration.  The Nexus is a layer 2 switch, but its handy when testing CNA connectivity to be able to ping from the Nexus using the VLAN interfaces)
Nexus5k(config)# feature lacp
(Note: This enables 802.3ad on the switch for port-channel negotiation)
Nexus5k(config)# feature vpc
(Note: This enables the Virtual Port Channel configuration on the switch)
Nexus5k(config)# feature fex
(Note: This enables the fiber channel extender protocol which allows you to connect 2ks to the 5k)
Nexus5k(config)# feature npv
(Note: This enables N port Virtualization on the switch making it a N port proxy)
Nexus5k(config)# exit

Configure the FEX (2k) modules
As discussed, the Nexus 5k’s use the FEX service to connect and manage the 2k modules as fabric extenders.  The configuration is pretty straight forward. 

Nexus5k# config t
Nexus5k(config)# fex 100
(Note: This creates a FEX association group 100)
Nexus5k(config-fex)# pinning max-links 1
(Note: This sets the pinning to 1 since we are going to be configuring a port-channel for the connection between the 2 and 5k.)
Nexus5k(config-fex)# exit
Nexus5k(config)# interface port-channel 100
(Note: Create the port-channel interface)
Nexus5k(config-if)# switchport mode fex-fabric
(Note: You need to specify that the connection is a fex-fabric connection rather than a standard switchport)
Nexus5k(config-if)# fex associate 100 
(Note: Assign the FEX group 100 to the port-channel)
Nexus5k(config-if)# no shut
Nexus5k(config-if)# exit
Nexus5k(config)# interface <the first physical interface going to the FEX>, <the second>
(Note: In this scenario, I’m using two 10 gig links to connect the FEX to the 5k)
Nexus5k(config-if)# switchport mode fex-fabric
(Note: You need to specify that the connection is a fex-fabric connection rather than a standard switchport)
Nexus5k(config-if)# fex associate 100 
(Note: Assign the FEX group 100 to the physical interfaces)
Nexus5k(config-if)# channel-group 100 mode on
(Note: Bind the interfaces to the port-channel)
Nexus5k(config-if)# exit
Nexus5k(config)# exit

Once this is completed the link between the 5k and the 2k should come up.  Once the link is established, the 2k will sync with the 5k, get software upgrades if need be, and then reboot.  Once the reboot is done the FEX ports will show up in the running config and interface commands on the 5k.  You can check the FEX status at any time by using the ‘show fex’ command on the 5k.

Wrap up
So at this point the 5ks and the 2ks should be online.  Depending on your topology there will need to be some extra port, trunk, and port-channel configuration.  I won’t walk through that in this post.  Up next we’ll talk about connecting the 5k to the IP network, the SAN fabric, and the host CNA adapters.