Snapshot can be nightmare for vmware administrator

Snapshot is the one of best feature of VMware. It is not backup solution but a good option if want to preserve some existing configuration/settings of vm/installed applications. User can take snapshot of vm and make changes if anything goes wrong, roll-back those changes with snapshot, taken prior making  changes. Most of the people uses this as backup and left it as for long time after creation. So, question here is how can be leaving snapshot becomes headache for Administrators.

As you guys are aware, when we create snapshot of vm a delta vmdk disk created and attached with vm. Original vmdk marked as readonly and all changes will be performed at delta vmdk. As Type of delta vmdk disk is thin and can grow upto the size of configured vmdk.

Example:
Size of datastore = 50 GB
Size of vmdk = 30 GB
Size of delta vmdk few MB but can grow upto 30 GB.

Now, suppose vmdk is Thick which already consumed 30 GB space of datastore and delta vmdk can grow up to 30 GB so, total space requirement would be 60 + GB.
Most of the peoples follow 10% threshold of free space. While snapshot created, admin was having enough free space 15GB but if anyhow, vm found as high IOPs, it will easily consume free space and create outage.

If type of vmdk is thin, even outage can't be avoided but can give us more time to full space of datastore.

As a good practice, we should delete snapshot once requirement of its completed.

As a work around, we can also create alert for snapshot usage. If snapshot reaches to configured alert value, alert will be generated and remediate action can be taken.

vNUMA -Virtual NUMA.

vSphere ESXi was taking benefit of NUMA since long time, as ESXi only aware of NUMA node, esxi kernel has the responsibility to place vm right NUMA node. Guest OS wasn't aware but with vNUMA guest OS is also aware of NUMA, it exposes guest OS to physical NUMA. Both guest OS and applications take adavantage of NUMA optimization, which provide performance improvements within VM. vNUMA allows VMs to benefit from NUMA, even if the VM itself is larger than the physical size of the NUMA nodes.


vNUMA comes in picture when you configure vm with more than 8 vCPU and without HotAdd vCPU feature.

Now question is here, how guest OS create virtual NUMA Nodes and how it impact the performance of vm? I have performed some vCPU configuration with VM and below are the results of vNUMA nodes.


CoreInfo is the sysinternal tool, I have used to check the numa configuration at OS level as there is no inbuilt tool in Microsoft OS however, OS is linux, you can use numctl command to check numa at OS level.

Physical host configuration

CPU - 4
Core - 8
Logical Processor with HT - 64

NUMA node - 4 and 1 NUMA node have 8 cores.


1. Virtual machine configured with 1 * 9 vCPU

ESXi kernel has used 2 numa to satisfy vm cpu/memory requirement but at OS level only one numa node is detected.

2. Virtual machine configured with 3 * 4 vCPU

ESXi kernel has given 3 numa to satisfy vm cpu/memory requirement and at OS level 3 numa node is detected.

3. Virtual machine configured with 2 * 9 vCPU

ESXi kernel has given 3 numa to satisfy vm cpu/memory requirement but at OS level only 2 numa node is detected.

It has been observed with above vms configuration, calculation of numa at guest level is directly proportional to socket configured at vm level. What will happen if we configure sockets to vm, more than physical sockets of hosts. Below is the result.

Physical host configuration

CPU - 4
Core - 4
Logical Processor with HT - 32

NUMA node - 4 and 1 NUMA node have 4 cores.

1. Virtual machine configured with 9 * 1 vCPU
vm is placed to 3 numa nodes at esxi kernel and at guest os, 2 numa nodes calculated. 4 NUMA nodes are calculated and 1 numa node is configured with 4 cores. As vm is configured with 9 socket which is more than physical socket of host,  2 numa node is calculated to satisfy 9 socket of vm.

Please correct if you guys found any discrepancies.  :-)

vRo script to delete snapshots of vms listed in text file

vRO script will read text file containing name of VMs, snapshots to be deleted.

To read the text file in vRO script, first of all you need to assign permission to path to which script can read text file. Task is quite simple, you need to append a line containing path along with permission to file named "js-io-rights.conf". Below are the steps.

Modify the js-io-rights.conf file:
  1. vi /etc/vco/app-server/js-io-rights.conf
  2. Press the i key on the keyboard
  3. Copy & paste the following line to the end file:
  4. +rwx /tmp
  5. Press the esc key on the keyboard
  6. Type in :wq! and press the Enter key
  7.  restart vRO server with command "service vco-server restart"
Below are the screenshots would be helpful to create workflow.










 
----------------------------------------Start code of script----------------------------------
var line,tempVM,i,vmname;
var myFile = new FileReader(Path);
myFile.open();
data = myFile.readAll();
var lines = new Array();
var fvms = new Array();
lines = data.split(/\r\n|\n\r/);
System.log(lines);
var vms = VcPlugin.getAllVirtualMachines();
for (line in lines )
{
 tempVM = lines[line];
 //System.log("2nd line " + tempVM);
 for (i in vms)
 {
  vmname = vms[i];
  //System.log(tempVM + " : " + vmname.name);
  if (tempVM.match(vmname.name))
  {
   System.log(tempVM + vms[i].name + " find");
   vm = vmname;
   //System.log(vm.snapshot);
   var allSnaps = new Array();
   var allParents;
   var snapshot = vm.snapshot;
   var rootSnapshotList = snapshot.rootSnapshotList;
   for each (var rsnap in rootSnapshotList)
   {
       getSnapshotsOfVM(rsnap);
   }
      for each (var snap in allSnaps)
   {
       System.log("VM Name: " + snap.vm.name);
          System.log("Snapshot Name: " + snap.name);
          System.log("Snapshot creation time: " + snap.createTime.toLocaleString());
          System.log("Snapshot is old, removing...");
    var task = snap.snapshot.removeSnapshot_Task(false,true);
    var actionResult = System.getModule("com.vmware.library.vc.basic").vim3WaitTaskEnd(task);
   }
  }
 }
}
function getSnapshotsOfVM(snapshotTree)
{
    allSnaps.push(snapshotTree);
    var childTrees = snapshotTree.childSnapshotList;
    if(childTrees != null) {
        for(var index in childTrees) {
            if(childTrees[index] != null) {
                getSnapshotsOfVM(childTrees[index]);
            }
        }
    }
}
 ----------------------------------------End code of script----------------------------------








Code has been modified by me as per mine requirement. It is taken from internet.