[Working Blog] ONTAP Snapshot Reporting in Python - Part 5 - Recap

We've discovered where to get all the data we need for our program. Now a recap of how to get all the data in one place. Also some data structures that might be useful. Once we've recapped, onwards we shall go (at this stage of writing, I hope we'll have something usable by part 7).

(Recap) How to Get all the Needed Data

Below is collecting all the data in our Python client so we can work with the data. I'm not 100% on whether we need to get the SVMs but I've left it in there. With volumes and snapshots my ByUUID dictionaries are designed to hold just want we need (hopefully be more usable like this), but can easily revert to contain the whole .get() output.

####################################
## DATA ACQUISITION (ONE CLUSTER) ##
####################################

## 0) Import statements ##

from netapp_ontap import config, HostConnection
from netapp_ontap.resources import Cluster
from netapp_ontap.resources import SnapshotPolicy
from netapp_ontap.resources import Schedule
from netapp_ontap.resources import Svm
from netapp_ontap.resources import Volume
from netapp_ontap.resources import Snapshot

## 1) CONNECTION TO CLUSTER ##

conn = HostConnection(
  host="CLUSTERNAME",
  username="LOGIN_USER",
  password="LOGIN_PASSWORD",
  verify=False
)

config.CONNECTION = conn

## 2) GET CLUSTER (INCLUDES UTC TIME) ##

clus = Cluster()
clus.get()
utcDateTime = clus.statistics.timestamp

## 3) GET SNAPSHOT POLICIES AND SNAPSHOT POLICY SCHEDULES ##

snapPolicyUUIDs = []
snapPolicyNames = []
snapPolicyUUIDtoName = {}
snapPolicyNameToUUID = {}
snapPoliciesByUUID = {}

for policy in SnapshotPolicy.get_collection():
  policy.get()
  snapPolicyUUIDs += [policy.uuid]
  snapPolicyNames += [policy.name]
  snapPolicyUUIDtoName[policy.uuid] = policy.name
  snapPolicyNameToUUID[policy.name] = policy.uuid
  snapPoliciesByUUID[policy.uuid] = policy

## 4) GET CRON SCHEDULES ##

def cronPeriod(x,period):
  try:
    if period == "minutes":
      return x.cron.minutes
    elif period == "hours":
      return x.cron.hours
    elif period == "days":
      return x.cron.days
    elif period == "weekdays":
      return x.cron.weekdays
    elif period == "months":
      return x.cron.months
    else:
      return ['X']
  except AttributeError:
    return ['X']

schedUUIDs = []
schedNames = []
schedUUIDtoName = {}
schedNameToUUID = {}
schedByUUID = {}

for s in Schedule.get_collection(type="cron"):
  schedUUIDs += [s.uuid]
  schedNames += [s.name]
  schedUUIDtoName[s.uuid] = s.name
  schedNameToUUID[s.name] = s.uuid
  schedByUUID[s.uuid] = {}
  schedByUUID[s.uuid]["minutes"]  = cronPeriod(s, "minutes")
  schedByUUID[s.uuid]["hours"]    = cronPeriod(s, "hours")
  schedByUUID[s.uuid]["days"]     = cronPeriod(s, "days")
  schedByUUID[s.uuid]["weekdays"] = cronPeriod(s, "weekdays")
  schedByUUID[s.uuid]["months"]   = cronPeriod(s, "months")
  # Handling some conditions:
  if schedByUUID[s.uuid]["months"] != ['X']:
    if schedByUUID[s.uuid]["days"] == ['X'] and schedByUUID[s.uuid]["weekdays"] == ['X']:
      schedByUUID[s.uuid]["days"] = [1]
  if schedByUUID[s.uuid]["days"] != ['X'] or schedByUUID[s.uuid]["weekdays"] != ['X']:
    if schedByUUID[s.uuid]["hours"] == ['X']:
      schedByUUID[s.uuid]["hours"] = [0]

# Changing ONTAP day 0 = Sunday, to Python datetime day 0 = Monday
for u in schedUUIDs:
  if schedByUUID[u]['weekdays'] != ['X']:
    newWeekdays = []
    for w in schedByUUID[u]['weekdays']:
      x = w - 1
      if x == -1:
        x = 6
      newWeekdays += [x]
    schedByUUID[u]['weekdays'] = newWeekdays

## 5) GET SVMS ##

svmUUIDs = []
svmNames = []
svmUUIDtoName = {}
svmNameToUUID = {}
svmByUUID = {}

for s in Svm.get_collection():
  svmUUIDs += [s.uuid]
  svmNames += [s.name]
  svmUUIDtoName[s.uuid] = s.name
  svmNameToUUID[s.name] = s.uuid
  void = s.get()
  svmByUUID[s.uuid] = s
  
## 6) GET VOLUMES ##

volUUIDs = []
volNames = []
volUUIDtoName = {}
volNameToUUID = {}
volByUUID = {}

for v in Volume.get_collection():
  volUUIDs += [v.uuid]
  volNames += [v.name]
  volUUIDtoName[v.uuid] = v.name
  volNameToUUID[v.name] = v.uuid
  void = v.get()
  volByUUID[v.uuid] = {}
  volByUUID[v.uuid]["create_time"]     = v.create_time
  volByUUID[v.uuid]["snapshot_policy"] = v.snapshot_policy.name
  volByUUID[v.uuid]["state"]           = v.state
  volByUUID[v.uuid]["style"]           = v.style
  volByUUID[v.uuid]["svm"]             = v.svm.name

## 7) ACQUIRE SNAPSHOTS ##

snapsByVolUUID = {}

for u in volUUIDs:
  snaps = Snapshot.get_collection(u,fields="create_time")
  snapsByVolUUID[u] = []
  for s in snaps:
    snapsByVolUUID[u] += [s.name]

What's left to do?

• Work out expected snapshot, based on the volumes and their applied snapshot policies
• Compare expected and actual snapshots, do we have all the snapshots we think we should have?
• Output to CSV

Note: I might share this post on my other blog ...