It occurred to me while writing the NPV and NPIV article (coming up shortly) that there was a lot of the fiber channel protocol which I hadn’t covered. That being said, I’m going to talk a little bit more about it here. Specifically we’ll discuss FCID, WWNs, zoning, and VSANs
So if you look at the terms we listed in the last post. You might be scratching your head and wondering what the difference is between WWNs and FCID/NPIDs. This was something I had trouble understanding initially as well. For starters, let’s agree that we’ll simplify and call NPIDS/FCIDS just FCIDs going forward. The terms are used interchangeably, but for the sake of clarity, I’ll just call them FCIDs going forward.
That being said, let’s talk about about the FCID. The FCID is a 24 bit (3 byte) field used to route frames through a FC network. Let’s walk through each byte of the FCID….
1st byte – Domain ID. The Domain ID is given to each FC switch in a fabric and must be unique for each switch.
2nd byte – Area ID. The Area ID is very similar to the 3rd byte. In fact, in Cisco documentation, you’ll find it called the exact same thing as the 3rd byte. The Area ID is used to identify an N type port that is connected to a switch.
3rd byte – Port ID. The Port ID is used to identify a single FC object on the FC fabric.
In a SAN each device that is going to talk on the fabric has a WWN and a FCID. The WWN is very much like a MAC address in Ethernet speak. Each WWN is globally unique with some exceptions. There are instances where devices allow an administrator to manually configure a WWN but we won’t be discussing those instances. For basic understanding, let’s say that each WWN is globally unique. So an HBA that had multiple ports would have a WWN for itself, and then PWWN (port WWNs) for each one of its ports. Additionally, each port on a fabric switch has a FWWN (fabric WWNs) which are sometimes also called FPWNNs (fabric port WWNs).
So, a device talking on the fabric has multiple identifiers. Still with me?
The FCID is used in routing through the fabric. So in a FC frame there is S_ID (Source ID) and a D_ID (Destination ID) field. These are populated with the appropriate FCID to facilitate traffic through the fabric and to the correct node.
So what is the WWN used for? Enter zones… Put basically, for two devices to communicate, they MUST be in the same FC zone. Zones are primarily a security practice. In reality, if everything was in the same zone, and in the same fabric, everything could talk to everything. That isn’t to say that all the LUNs would show up on all the servers, its just implying that it would be possible. Zoning is a fabric configuration, not a disk array to host mapping configuration. So let’s go over a quick example so that we are all on the same page. Take the diagram below as an example. (Side note: I’ve had this wonderfully large whiteboard in my home office for almost a year and have hardly ever used it. I will waste resources no longer, look forward to more whiteboard pictures with my hard to read hand writing.)
So as we can see here (in my glorious white board depiction) we have two servers. A backup server that needs access to the disk array for its local disk, and access to the tape library to run backups. We also have a file server that just has access to the disk array. It doesn’t need access to the tape library since the backup server is accessing the file server’s files over the network, and then putting them to tape itself. That’s a pretty simple example of zoning.
To be clear, you don’t always have to use WWNs to create zones. You can also create zones based on…
-PWWN or FWWN
-FC Alias (Basically, a name associated with a WWN)
Cisco recommends using a PWWN to do zoning. This ties an HBA to a zone rather than a physical port to a zone.
Another concept of logical SAN separation is VSANs. While its usually viewed as a Cisco fabric feature I believe its recently been approved as a global standard. VSANs are much like, you guessed it, VLANs. A VSAN allows you to have logically separate fabrics without actually having to have different physical fabrics.
(Additional side note: I’m going to get colored markers to make this more straight forward)
As you can see in the drawing above, all four devices hang off of the same physical fabric, but since they are in separate VSANs, they are unable to communicate.
So let’s take a step back and talk about the entire Cisco SAN configuration. We spoke about zones and VSANs, now let’s talk about how to configure them. This is the basic list I use when talking about SAN config on an MDS.
1. Create the VSAN number you wish to use (1 is the default, not recommended to use the default for production SAN traffic)
2. Add interfaces to your VSAN (just like you do with a VLAN)
3. Do any interface configuration needed on the FC interfaces (Just turn them on in most cases)
4. Verify the cabling and ensure that you have SAN connectivity
5. Create Aliases for WWNs (makes life easier)
6. Create required zones
7. Add members to your zones (I recommend using PWWNs)
8. Create a zoneset (I think Brocade calls this a ‘config’)
9. Add your zones to your zoneset
10. Activate the zoneset on the fabric
(Don’t worry I’ll show you how to do each one of these steps in the next article)
You’ll notice that we introduced the term ‘zoneset’ in the steps. We haven’t talked about them yet, so let’s take a brief moment to discuss them. There are some basic rules I like to think of when talking about zones/zonesets that do a pretty decent job defining the term. They are…..
-Zonesets are a container for one or more zones in the fabric
-Zones need to be a member of a zoneset in order to be used. That is to say, zones once configured aren’t actually ‘on’ until they are applied to a zoneset and the zoneset is activated in a particular VSAN.
-Zones can be in more than one zoneset ‘container’. That is, a zone can belong to multiple zonesets.
-Devices (WWNs) can be members of multiple zones
-Only one zoneset can be active at one time in any given VSAN
-If an alias or WWN is not assigned to a particular zone, it is a member of the default zone (1).
-If zoning isn’t active at all, then all devices are default to being a member of the default VSAN.
So if we looked at the big picture, the layout would look like this….
I probably shouldn’t have used VSAN 1 in the diagram since thats the default, but you get the idea.
Please keep in mind that we are talking about a very basic configuration here. There are far more advanced topics/concepts in regards to all of these functional items. As you get more comfortable with the configuration, you can start playing with the additional functionality.
As I reread these posts, I’m starting to see they are a bit scatterbrained in regards to flow. I apologize for that; I’ll do my best to keep these flowing as I’m sure that makes them easier to read. Once we get all the basic FC understanding out of the way, I think the actual config posts will flow together nicely.
This is my outline for the next few posts…
-CLI config of an MDS (how to do what we’ve been talking about)
-NPV and NPIV
-Nexus 5k/2k config (just the base config)
-Nexus 5k/2k config (Connecting to the MDS and the IP network)
-Nexus 5k/2k config (Connecting to the hosts (CNAs))
-Follow up posts to answer any questions I see coming through in the comments.
Awesome Post. Great clarity.
Nice Post and It help Beginners allot .. Keep posting…
thanks man. it really helped me
I must say Now I am out of mesh of confusions Great post buddy expect few more of them soon
Great blog post! I just have one question. So is it only pertinent to create multiple VSANs only if you have multiple Disk Arrays?
I ask because instead of just creating zones for tasks, I wanted there to be a VSAN per department but they would all share the same Disk Array in the middle, so there’s little disruption to the entire fabric if I wanted to test something in a development environment that shares the same SAN.
Good question. I can tell you that Ive only ever used VSANs for multiple paths. That being said, that doesnt mean I have two seperate disk arrays. I think it depends on how big your SAN fabric is and if there is a need to do that. Sounds like zones moght work just as well.
Hello Mr. Jon Langemak.
Very nice article. Can you please write one about ISLs ?
My question is regarding zoneset.
Say i have 2 MDS9000 switches (sw1 and sw2). These are brand new switches no previous configuration. I now configure Zone Alpha in VSAN1 on SW1 and Zone Beta in vsan1 on sw2.
Should i add zone alpha and zone beta to same zoneset (say zoneset1active) on both switches . OR should i use two different zonesets (say “zoneset1active” associate to zone alpha on sw1 and other “zoneset1active” associate to zone beta on sw2) ?
Also to add to above query ?
I need both zones to be in active zoneset for vsan1 .
I need to configure zones one on each switch and can’t configure both zones on one switch .
— I did that using same zoneset name , but after redistribution i get to see just one zone , its not appending zones instead its updating to just either one (say beta in both )in active zoneset . I can see just one zone not both in running config of both switches.
How to make it work , what sequence to follow to achieve this ?
Can you explain what you are trying to accomplish from a host perspective? I think Im missing the point of what you are trying to configure.
Thank you very much. It gives some clear idea about the basics in cisco MDS which I am looking for. Great work.. Thnx alot again.
My ques is .. in order to create a zone we require host and a storage. For host we define pwwn as a zone member. But what about storage ? Its faport’s wwn . It should also be defined as a member in that zone defination.
Can you clarify the question? I think Im missing your point.
you are really talented in explaining such somehow sophisticated terminology in such a simple coherent way.
I suggest to make a blog for your articles and present them in a complete series
Wish you best of luck
I have zoning problem issue. in the zone member, there is no star in host pwwn (not running, people said). I already remove and recreate member and zone, but still not running. what what got me confused is the status of int eth, veth, and vfc is up. Thanks in advance. aris
thanks man for great explanation…
finally I can solved this. the problem was someone not install the storage controller driver. as I instali, thus pwwn active directly. Thanks.
what is command to find out interface port number by using FCID.
Thank you for the great article, one of the best! I have a question and hoping you can help in answering it. In Nexsus 5k, I have my active zonset is showing lot of zones, but I cannot find any of them when I try to run show zone name. Also, the zoneset itself isn’t shown when I run show zoneset, but it only appears when I run show zoneset active. Is this normal behavior, or the zoneset was activated and then deleted?
If I try now to add a new zone to the active zoneset, the zoneset set will have only one member zone when I run show zoneset. What will happen if I activate it? will it append the new zone to the active zoneset, or will delete all the zones and add only the new zone?
Thank you for your reply.
I have two hosts(host31 and host63), I would like the host63 to see all the drives on host31. I use EMC VMAX.
I have two Cisco MDS switches .
What is the best way to do this?
The best way to do this is with a Storage Group on your VMAX.
Excellent explanation !!
is it possible to zone single port in different vsan? or is it possible to zone same host in two different VSAN
Thank you so much for making things clear, I found the diagrams very helpful!!