What is the NetApp DII Data Model?

on
From Wikipedia: "A data model is an abstract model that organizes elements of data and standardizes how they relate to one another and to the properties of real-world entities."

If you search the Data Infrastructure Insights documentation for "data model" you won't get much.

Top 3 results to "data model" search in DII documentation


But the data model is pretty much what you see in the Data Infrastructure Insights Reporting Schema Diagram. 

For example we look at Storage Mapping. Essentially, this is every table with storageId in. And we can search for this super easy in this post An Exploration of the NetApp DII DWH - Columns on Tables in dwh_inventory.

The statement in the above paragraph is not 100% correct. The dwh_inventory schema is not identical to the inventory schema (at least it didn't use to be when you could look in a front-end OCI server MySQL database and compare it to the back-end OCI DWH server MySQL database), but it is close enough. Also, don't forget that there is a massive wealth of data in the the DII Data Lake which isn't in the inventory/dwh_inventory schema, also performance data too.


Note: I've highlighted some obvious logical objects in cyan - like volumes - and physical objects in yellow - like nodes.

Storage Mapping: storage.id maps to storageId in ->
  • auto_tiering_policy_constraint.storageId
  • auto_tiering_policy.storageId
  • backend_lun_to_volume.backendStorageId
  • backend_lun_to_volume.frontendStorageId
  • backend_lun.storageId
  • backend_path.storageId
  • cluster_switch_port.storageId
  • cluster_switch.storageId
  • disk_group_to_disk.storageId
  • disk_group_to_storage_pool.storageId
  • disk_group.storageId
  • disk.storageId
  • dr_internal_volume_replica.sourceStorageId
  • dr_internal_volume_replica.targetStorageId
  • dr_qtree_replica.sourceStorageId
  • dr_qtree_replica.targetStorageId
  • dr_replica.sourceStorageId
  • dr_replica.targetStorageId
  • dr_volume_replica.sourceStorageId
  • dr_volume_replica.targetStorageId
  • file_system_to_volume.storageId
  • hv_data_store_to_internal_volume.storageId
  • hv_data_store_to_volume.storageId
  • hv_lun_to_volume.storageId
  • hv_nas_share_to_share.storageId
  • hv_virtual_disk_to_volume.storageId
  • internal_volume.storageId
  • iscsi_logical.storageId
  • logical.storageId
  • nas_cr_logical.storageId
  • nas_file_share.storageId
  • nas_logical.storageId
  • nas_share_initiator.storageId
  • nas_share.storageId
  • physical_path.storageId
  • qtree.storageId
  • quota.storageId
  • storage_controller.storageId
  • storage_node_to_internal_volume.storageId
  • storage_node_to_port.storageId
  • storage_node_to_storage_pool.storageId
  • storage_node_to_volume.storageId
  • storage_node.storageId
  • storage_pool_to_backend_lun.storageId
  • storage_pool_to_disk.storageId
  • storage_pool.storageId
  • storage_port.storageId
  • storage_to_application.storageId
  • unknown_logical.storageId
  • violation.storageId
  • volume_in_storage_pool.storageId
  • volume_map.storageId
  • volume_mask.storageId
  • volume.storageId

Similarly for other mappings we can search in An Exploration of the NetApp DII DWH - Columns on Tables in dwh_inventory for the {object}Id to see which tables it is in.


APPENDIX: Service Now

The reason for the above is a question I then went and asked an AI "Design a data model for ServiceNow to injest data from NetApp Data Infrastructure Insights". And the output looked reasonable. I'll post the AI answer below with edits and over time I may edit this or create a dedicated post for the question with real-world experience (something AI does not have):

1. High-Level Architecture

To ensure data integrity and avoid duplicates, you should use ServiceNow's Identification and Reconciliation Engine (IRE)

  • Source: NetApp DII REST APIs.
  • Staging: Custom Import Set Tables (e.g., u_netapp_dii_storage, u_netapp_dii_volumes).
  • Processing: Transform Maps utilizing the CMDB Integration Studio or Robust Transform Engine (RTE) to push data through the IRE.
  • Target: ServiceNow CMDB CI Classes.

2. CMDB Class Mapping (Data Model)

Here is how the primary assets in NetApp DII map to ServiceNow's out-of-the-box (OOTB) CMDB classes:

NetApp DII Entity   ServiceNow CMDB Class 
Storage Array       cmdb_ci_storage_server
Storage Node        cmdb_ci_storage_node  
Storage Pool        cmdb_ci_storage_pool  
Storage Volume      cmdb_ci_storage_volume
LUN                 cmdb_ci_storage_lun   
SAN Switch          cmdb_ci_fc_switch     
Virtual Machine     cmdb_ci_vm_instance   

Other columns you would want to add to this table: Key Attributes to Map & Identification Rule (IRE)


3. CI Relationships (The Service Graph)

Linking CIs provides real value for root cause analysis. Suggested Transform Maps:

  • Storage Node [Contained by] Storage Array
  • Storage Pool [Hosted by] Storage Array
  • Storage Volume / LUN [Instantiated by] Storage Pool
  • Host / VM [Depends on] Storage Volume / LUN
Note: Also can look at the Storage Mappings we get from dwh_inventory


4. Implementation Steps

  1. Create a REST Message in ServiceNow to authenticate and pull JSON payloads from the NetApp DII API.
  2. Set up Scheduled Jobs to pull this data daily (for inventory/CMDB) and more frequently for events.
  3. Define Transform Maps mapping the JSON keys to the CMDB fields listed above.
  4. Test via IRE to ensure CIs update correctly without creating duplicates.

Comments

Post a Comment