Caveat lector: This is an independent and unsupported blog. Please seek supported NetApp documentation and/or NetApp Professional Services before attempting!
Note i: I would normally follow the NetApp official documentation - Storage switches - but in this case I couldn't find what I was looking for, hence this blog.
Note ii: The scenario below is quite specific, but the theory could be applied to any similar system and moving from DAS to SAT.
Scenario
I need to migrate a NetApp AFF C800 from:
i) 2 x NS224 Shelves Direct-Attached (DAS)
- to -
ii) 2 x NS224 Shelves Switch-Attached (SAT)
This is because the maximum 2 x NS224 Direct-Attached shelves have been reach and we want to add a 3rd NS224 shelf to the system.
Initial Steps
1) Check in NetApp Hardware Universe (HWU) for a supported NX-OS and RCF (configuration file) for your ONTAP version and Storage switch.
Example output showing nxos64-cs.10.4.5.M and NX9336C-FX2-RCF-v1.13-12-Port-1-Storage are a supported NXOS + RCF configuration with the Cisco Nexus N9K-C9336C-FX2, 12-port switch and ONTAP 9.16.1:
2) Verify in the Bill of Materials (BoM) that you have all the components needed to complete you work.
Note: Interestingly, you cannot use optical modules in the X1148A card on the C800/A800. HWU says "optical modules are not supported due to thermal constraints".
3) [On-site] Configure your switches.
See: APPENDIX: Storage Switch Configuration Brief Notes
4) [On-Site] Verify the existing cabling is correct before starting.
Converting from Direct Attached NS224 to Switch-Attached
This is a non-disruptive operation (NDO) but needs to be done carefully.
- Note i: Do your usual cluster health checks before starting:
- cluster show
- system health status show
- system health subsystem show
- system health alert show
- storage shelf show -connectivity
- storage shelf show -connectivity -instance
- etcetera ...
- Note ii: Before moving cables and after each step, validate storage paths (before and after every step we should see an A and a B path for each disk on the NS224 shelves):
- node run -node NODENAME storage show disk -p
- Note iii: The process is essentially:
- 1st phase move all the Path_A (e0a on NS224) direct connections 1-by-1 to switch_A
- 2nd phase move all the Path_B (e0b on NS224) direct connections 1-by-1 to switch_B
1) Send AutoSupport - Start:
system node autosupport invoke -node * -type all -message "MAINT=2h Start: Converting NS224 from DAS to SAT"
2) Move cables from direct to switch according to the following sequence, one cable at a time:
- 1) NODE_A e5a <-> SHELF_1 / NSM_A - e0a
- NODE_A e5a <-> Switch_A e1/1
- SHELF_1 / NSM_A - e0a <-> Switch_A e1/5
- 2) NODE_B e5a <-> SHELF_1 / NSM_B - e0a
- NODE_B e5a <-> Switch_A e1/2
- SHELF_1 / NSM_B - e0a <-> Switch_A e1/6
- 3) NODE_A e3a <-> SHELF_2 / NSM_A - e0a
- NODE_A e3a <->
- SHELF_2 / NSM_A - e0a <-> Switch_A e1/7
- 4) NODE_B e3a <-> SHELF_2 / NSM_B - e0a
- NODE_B e3a <->
- SHELF_2 / NSM_B - e0a <-> Switch_A e1/8
- 5) NODE_A e3b <-> SHELF_1 / NSM_B - e0b
- NODE_A e3b <-> Switch_B e1/1
- SHELF_1 / NSM_B - e0b <-> Switch_B e1/6
- 6) NODE_B e3b <-> SHELF_1 / NSM_A - e0b
- NODE_B e3b <-> Switch_B e1/2
- SHELF_1 / NSM_A - e0b <-> Switch_B e1/5
- 7) NODE_A e5b <-> SHELF_2 / NSM_B - e0b
- NODE_A e5b <->
- SHELF_2 / NSM_B - e0b <-> Switch_B e1/8
- 8) NODE_B e5b <-> SHELF_2 / NSM_A - e0b
- NODE_B e5b <->
- SHELF_2 / NSM_A - e0b <-> Switch_B e1/7
3) Verify all is good.
4) Send AutoSupport - Finish:
system node autosupport invoke -node * -type all -message "MAINT=2h Start: Converting NS224 from DAS to SAT"
Conclusion
The process worked seamlessly. I ran through the procedure on a production system in the middle of the working day and no one noticed, which is perfect 🍻.
APPENDIX: Storage Switch Configuration Brief Notes
It's simple doing a basic configuration of a Storage Switch for NetApp ONTAP. This is an example switch configuration output. I use a simple webserver on my laptop (HFS.exe) for serving the software images. When applying the RCF you need to make sure the RCF applies successfully (i.e. take heed of any errors and read the notes section of the RCF, which may tell you to apply the RCF twice for instance.) As long as remote connectivity is good (confirm) any further required configurations can be applied remotely at a later date.
Firstly we Abort Power on Auto Provisioning and answer the prompts (many just accepting the default option). Download to the switch - NXOS, RCF and hardware EPLD. Install - NXOS (remember to say y to continue with the install before the prompt times out), EPLD, RCF. Validate all is good.
Abort Power On Auto Provisioning [yes - continue with normal setup, skip - bypass password and basic configuration, no - continue with Power On Auto Provisioning] (yes/skip/no): yes
---- System Admin Account Setup ----
Do you want to enforce secure password standard (yes/no) [y]: n
Enter the password for "admin":
Confirm the password for "admin":
---- Basic System Configuration Dialog VDC: 1 ----
This setup utility will guide you through the basic configuration of the system. Setup configures only enough connectivity for management of the system.
Please register Cisco Nexus9000 Family devices promptly with your supplier. Failure to register may affect response times for initial service calls. Nexus9000 devices must be registered to receive entitled support services.
Press Enter at anytime to skip a dialog. Use ctrl-c at anytime to skip the remaining dialogs.
Would you like to enter the basic configuration dialog (yes/no): yes
Create another login account (yes/no) [n]:
Configure read-only SNMP community string (yes/no) [n]:
Configure read-write SNMP community string (yes/no) [n]:
Enter the switch name : SWITCH1
Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]:
Mgmt0 IPv4 address : 10.10.10.11
Mgmt0 IPv4 netmask : 255.255.255.0
Configure the default gateway? (yes/no) [y]:
IPv4 address of the default gateway : 10.10.10.1
Configure advanced IP options? (yes/no) [n]:
Enable the telnet service? (yes/no) [n]:
Enable the ssh service? (yes/no) [y]:
Type of ssh key you would like to generate (dsa/rsa) [rsa]:
Number of rsa key bits <1024-2048> [1024]:
Configure the ntp server? (yes/no) [n]:
Configure default interface layer (L3/L2) [L2]:
Configure default switchport interface state (shut/noshut) [noshut]:
Configure CoPP system profile (strict/moderate/lenient/dense) [strict]:
The following configuration will be applied:
no password strength-check
switchname SWITCH1
vrf context management
ip route 0.0.0.0/0 10.10.10.1
exit
no feature telnet
ssh key rsa 1024 force
feature ssh
system default switchport
no system default switchport shutdown
copp profile strict
interface mgmt0
ip address 10.10.10.11 255.255.255.0
no shutdown
Would you like to edit the configuration? (yes/no) [n]:
Use this configuration and save it? (yes/no) [y]:
User Access Verification
SWITCH1 login: admin
Password:
SWITCH1# show version
...
Software
BIOS: version 05.47
NXOS: version 10.3(4a) [Maintenance Release]
...
SWITCH1# copy http://10.10.10.5/nxos64-cs.10.4.5.M.bin bootflash: vrf management
SWITCH1# copy http://10.10.10.5/NX9336C-FX2-RCF-v1.13-12-Port-1-Storage.txt bootflash: vrf management
SWITCH1# copy http://10.10.10.5/n9000-epld.10.4.5.M.img bootflash: vrf management
SWITCH1# install all nxos bootflash:nxos64-cs.10.4.5.M.bin
...
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] y
...
Finishing the upgrade, switch will reboot in 10 seconds.
---
User Access Verification
SWITCH1 login: admin
Password:
SWITCH1# show version
...
Software
BIOS: version 05.53
NXOS: version 10.4(5) [Maintenance Release]
...
SWITCH1# show version module 1 epld
Module 1:
EPLD Device Version
---------------------
MI FPGA 0x5
IO FPGA 0x17
SWITCH1# install epld bootflash:n9000-epld.10.4.5.M.img module 1
None of the modules can be upgraded.
SWITCH1# copy run start
SWITCH1# terminal length 0
SWITCH1# copy NX9336C-FX2-RCF-v1.13-12-Port-1-Storage.txt running-config echo-commands
...
Copy complete, now saving to disk (please wait)...
Copy complete.
SWITCH1# show run
...
SWITCH1# show banner motd
...
SWITCH1# copy run start
Comments
Post a Comment