You are currently browsing articles tagged Storage.

VCP – VMFS Block Sizes

A quick post on VMFS block sizes.  There are two configurable VMFS versions in vSphere 5.  VMFS-3 and VMFS-5.  VMFS-3 data stores could be configured to use different block sizes.  The reason for doing this was that based on the block size, a user could increase the maximum size of a single virtual disk.  So the block size dictates the maximum file size for a particular data store.  For instance…

Block size Max size of
1 MB 256 GB
2 MB 512 GB
4 MB 1 TB
8 MB 2 TB

So when you create a VMFS-3 data store, you get the option of selecting the block size…

Once configured, the block size can not be changed.  If you want to change the block size, you have to migrate data off, blow away the VMFS data store, recreate it, and migrate data back.

This is not the case with the newest version of VMFS, VMFS-5.  Only hosts running vSphere 5 or later support VMFS-5.  Hosts running earlier version wont even be able to see the data stores formatted with VMFS-5.  VMFS-5 has the following improvements.

-VMFS-5 uses a unified block size of 1MB.  The block size isn’t even a configurable options when you add storage as VMFS5. Despite the smaller block size the max file size in VMFS-5 is 2 TB.

-VMFS-5 data stores can be as large as 64 TB.  This is the max size for data stores with or without extents. 

-Physical RDMs in VMFS-5 can be larger than 2 TB in size. 

So there are some significant enhancements in VMFS-5.  And the bonus is that you can upgrade VMFS-3 to VMFS-5 on the fly.  The only gotcha is that the block size can’t be changed.  For instance, let’s upgrade one of my VMFS-3 data stores to VMFS-5.  So here is what we have to start with…

As you can see, it’s a VMFS-3 file system set with a block size of 8.  Now, click the link up top that says ‘Upgrade to VMFS-5…’.  You’ll get this warning…

Click OK to start the upgrade…


As you can see, once it completed I’m now running VMFS-5.  Also note that although VMFS-5 only uses a block size of 1 MB, the block size is still 8MB.   To get the best performance out of the data store, I’d likely want to rebuild it as a native VMFS-5 data store to get the same performance as newly built VMFS-5 data stores.


Storage / SAN Terminology

I realized as I reread my last posts that I used a lot of terminology related to storage and SANs that I didn’t do a very good job of explaining what they actually mean.  Additionally you’ll find that, as with any other field of IT, some terms begin to have double meaning and are used as a general term for something else.  I’d like to spend some time to review some of the more general terminology in regards to SANs and storage.  And in no particular order….

LUN – Logical Unit Number.  LUN actually comes from the SCSI world.  Basically an LUN represents an individual device (A disk) which is part of the target (The SAN Cluster).  LUN seems to be hang-over from the older SAN days and many different people have different opinions as to what its definition is.  I like to think of an LUN as a mapping to a disk.  So, in the old FC SANs I worked on, we would carve a disk out of the array, put them in a RAID configuration, and then attach an LUN number to them.  The concept is very much the same in ISCSI.  You create a disk on the SAN and it gets a IQN (defined below) attached to it.  The IQN is how the disk is identified on the SAN.

HBA – Host Bus Adapter.  Think of the HBA as the connection between a system and the SAN.  With Fiber Channel SAN it’s usually something like a QLogic Fiber card, with ISCSI it’s either the NIC that attaches you to the ISCSI SAN or a dedicated Hardware HBA that does more of the processing for you.

Zoning – Really more of an FC term.  When you were working with SAN switches you would create aliases for all the WWN (World Wide Numbers) which were the unique identifiers for SAN endpoints.  For instance, each HBA would have a WWN as well as each other device such as the disk controllers and so on.  After I aliased each WWN with a name (EX: 20:00 00:e0:8b 02:02:05 equals HBA1_Server2) I would put everything that needed to talk to each other in its own zone.  So, I might have a zone for Server2 which would include both of its HBAs and both of the disk controllers.  Or I might have a backup server that needed to talk to the disk controller and the tape library.  So that zone would include all the aliases for those devices.  Then only items in their own zone could talk to each other.  Similar to VLAN’s on a Ethernet network.

FC – Fiber Channel.  SANs were originally only FC due to the speed you needed to get for direct disk access.  The advantage to FC was that it forced you to create a separate network for disk.  No one had fiber for their normal servers so you had to put in an FC infrastructure designed just for disk access.  Now you can run ISCSI on a standard Ethernet network.  If not implemented separately/correctly you can have a lot of issues.

IQN – ISCSI Qualified Name.  Like a WWN it’s the identifier for the ISCSI network.  Each device that is going to communicate on the ISCSI network needs one.  When I create a Volume of a Snapshot on my LeftHand SAN the SAN also creates IQNs for each of them.  When I go on my ESXi host or on my server I will need to know what the IQN is for the volume I want to attach to.  Conversely you can use CHAP in which case you don’t need to know the IQN of the server you want to attach to.  As long as CHAP authenticates on the initiator side the volumes will come down to the server.

ISCSI – Internet Small Computer System Interface.  Basically let’s you encapsulate SCSI disk commands within Ethernet segments , so you have a SAN that works over long distances on existing infrastructure.

ISCSI Software Initiator – Uses your standard NIC and a piece of software to process ISCSI commands.  MS has a software initiator that you can download for free and use with their OS’s.

ISCSI Hardware Initiator – In the SAN world this is more or less an ISCSI HBA.  It’s usually a designated card that handles all of the processing of the ISCSI traffic.  It also usually has a built in bios allowing you to boot from ISCSI should you decide to do so.

TOE – TCP Offload Engine.  TOE Cards are a little more common that full ISCSI HBAs.  More often than not, a built in NIC might have TOE allowing it to offload some of its processing on the card but still requires some of the host systems CPU time for processing.

Spindle – An actual hard drive within an array

Disk / Volume – This is another term (or set of terms) that people use semi-interchangeably.  For instance on the older FC SANs I worked on, a disk represented an actual spindle.  When we carved volumes out of a SAN we specified 2 or more ‘disks’.  They were all numbered and you literally could pick which ones you wanted in each volume.  So a disk was a spindle and the volume was a group (in RAID) of spindles.  Now with the advent of Virtual SAN environments all of your storage is just seen as one large pool.  So you have an underlying RAID configuration and then the disk is presented to the SAN on top of that.  You make volumes from the available raw space.  Disks and spindles don’t really come through in the configuration side.

Tags: ,

The LeftHand Backup Script

I know I said that Part 5 would be my last LeftHand post for the time being but I couldn’t resist building a script to test out all of the scripting commands we just went through.  So I created two scripts that do the following.

-Creates a snapshot of a volume
-Assigns it to a server
-Attaches it to the server for access

-Logs the ISCSI session off
-Un-assigns it from the server (removes the target portal)
-Deletes the snapshot

Take a look at it if you’d like, feel free to use any of the code yourself.  If you are going to use the entire batch file yourself, I ask that you leave the header in place that gives credit to this blog.  I’m contemplating making a CHAP version as well but I’ll have to find some time first.

I, in no way, shape, or form guarantee this code to work.  It worked on my configuration but you might need to tweak it a little yourself.  I tried to document it as best as I could. Furthermore I am not claiming this as my own completely original work.  I used a lot of other concepts and examples that I found online to build the scripts.

Tags: , ,

« Older entries