NetApp ASA (All San Array) Lab Investigation

Just playing around with a lab-on-demand to understand how sizing works in ASA. Some things are hidden for administrative simplicity, but you can still dig down and find disks, raid-groups, aggregates, volumes, LUNs etcetera.

1) Disks

The SIM has 28 x 29127 MB (physical) and 26795 MB (Used/Usable)

Note: node run local aggr status -r output says 26795 used for spare disks and other disks.

Of those 28 disks, we have:

• RAID Group 0 (24 disks): 22 data, 2 parity (RAID-DP)
• RAID Group 1 (3 disks): 1 data, 2 parity (RAID-DP)
• 1 spare

And it looks like the disk (::> disk show) are owned by each node (shared or not owned by any node in particular.)

2) Aggregates

With ASA we don't create the aggregates, they are magically there already.

cluster1::*> aggr show -fields size,availsize,nodes,volcount
aggregate      node        availsize size     volcount
-------------- ----------- --------- -------- --------
dataFA_2_p0_i1 cluster1-01 124634MB  124637MB 1
dataFA_4_p0_i1 cluster1-02 180580MB  180597MB 1
pod_SSD_1      cluster1-01 0MB       0MB      0
rootFA_1_p0_i1 cluster1-01 110744MB  111919MB 1
rootFA_3_p0_i1 cluster1-02 110754MB  111919MB 1
5 entries were displayed.

5 aggregates, 1 root and 1 data per node, and the one called pod_SSD_1.

23 data disks x 26795 MB = 616285 MB

Sum of size above = 529072 MB - Q: How have we lost 87213 MB?

3) Volumes

I have already used the wizard in System Manager to create 1 LUN as per:

The capacity remained at 219 GiB physical used and 298 GiB available.

But the host only saw 1.98 GB and 2 GiB converted to GB is 2.15 GB, so where did the remainder go?

We have these volumes (all have space guarantee = nonoe or thin-provisioned vols):

cluster1::*> vol show -fields aggregate,size,available,snapshot-reserve-available

vserver volume    aggregate      size          avail. s.r.a[1]

------- --------- -------------- ----------- -------- ------ 

clu1-01 vol0      rootFA_1_p0_i1    105903MB  99601MB 5132MB

clu1-02 vol0      rootFA_3_p0_i1    105903MB  99589MB 5154MB

svm1    app_1     dataFA_4_p0_i1 440401920MB 180580MB    0MB

svm1    svm1_root dataFA_2_p0_i1        20MB     18MB    0MB

4 entries were displayed.

[1] = snapshot reserve available

So the app_1 volume created to hold our 2 GiB LUN is actually much bigger than the LUN and has a large available space (which is the same as aggregate available size.) It would appear the wizard has created a max size (for the platform) volume but the available space is what is remaining in the aggregate.

4) LUN 2GB

Our created LUN is of size 2048MB (hmm, this isn't 2 GiB that would be 2147MB.) Our LUN size is 2.048 GB ... hmm I'm getting very confused with units ... or is it simply 2 GB and this online calculator I am looking at is b0ll0cks!?

cluster1::> lun show -instance

                              Vserver Name: svm1

                                  LUN Path: app_1

                               Volume Name: app_1

                                  LUN Name: blocks

                                  LUN Size: 2048MB

                                   OS Type: windows

                         Space Reservation: disabled

                Space Reservations Honored: false

                          Space Allocation: enabled

            Physical Size of Logical Block: 0MB

                                 Used Size: 11MB

                       Maximum Resize Size: 134217728MB

                      Node Hosting the LUN: cluster1-02

                        Physical Used Size: 0MB

           Physical Used Size By Snapshots: 0MB

Okay, when we look at the Volumes tab for the disk we see a capacity of 2032 MB with unallocated space of 2 MB, and a capacity of 2030 MB. And 2030 / 1024 ~= 1.98. So it would seem that our LUN was indeed 2GB and we've lost a little just 18MB (NTFS Overhead perhaps.)

If we look in the empty volume (LUN) it does show some used space (15.8 MB) (could that be the NTFS overhead!?).

5) Comment

Curious about why the aggregates are smaller than I expect, but maybe this is a weird lab thing. Should be easier to understand in a real physical system.

Lets try with a bigger LUN and see what happens.

6) Bigger LUN

If I create a 100GB LUN, that is about the limit of these lab aggregates. Of course, it starts off thin so will not be reflected on the aggregate. (Note: More options includes things like QoS, snapshots and replication.)

Note: You can resize LUNs in ONTAP System Manager. You'll need to rescan from the host side and expand the volume (and 1 GiB is actually 1024MB or 1GB on the Windows host, and we don't get any more overhead ... just checked with the 2GB LUN.)

And our 100 GiB LUN appears as a 100 GB LUN in Windows. After initializing it we get 99.98 GB. Capacity is 102384 MB, 2MB unallocated again after creating the Windows volume (not sure why it always leaves 2MB). Now 100GB is 102400 MB, so again we've lost about 16MB to some overhead. Curiously we have 98.9 MB used in the volume ... is this the NTFS overhead!?

Summary

I think ASA is pretty cool that you don't need to think about aggregates, raid_groups and all that. Nor any volume stuff. Simply create LUN and go. I do not trust ONTAP System Manager though, that GiB / TiB definitely looks like GB / TB in my mind (and to Windows) but might that Windows is wrong. If Wikipedia is correct then Windows is wrong!

Gigabyte - Wikipedia