Data Protection

 View Only

HDID for VMware: VMware protection in Hitachi Data Instance Director - Part1: Selection (via the REST API)

By John Wesley posted 01-23-2020 10:46

  

Previously (VMware protection in Hitachi Data Instance Director - Part 1: Selection ) I wrote about the options provided for selecting Virtual Machines via the HDID user interface. This is a great introduction to show off the capabilities of what HDID makes possible, but what if you're a DevOps type and clicking buttons isn't really "your thing"? Well, thankfully there's the HDID REST API, and even more helpfully the HDID SDK.

n this blog post I'll go through the required API calls to create the ESX node and use it to build a Policy that selects Virtual Machines by id, name and tag. I'll also try and follow up each blog post with one explaining how to perform the same operations using the REST API.

Lets get started, as a minimum you'll need a HDID install that's up and running, and I'm going to use Postman to make my REST requests.

Note: the "Content-Type" header must be set to "application/json" for all the requests shown, otherwise you may get an error about unknown content type

Login

In order to use HDID we need to login, this is done by a POST to the login action at:
https://MASTER_IP/HDID/master/UIController/services/Users/actions/login/invoke
The HDID API talks exclusively JSON, and the body for this action is the following JSON document:
{"username":"my_username","space":"master","password":"my_password"}
Note: The HDID API runs only on the HDID master node, and some requests can only be dealt with by the master, such as logging in.
 
The response body to this request, if successful, should be empty:
null
The important part is in the headers, where the Session ID is kept:
Set-Cookie →Session_ID={b0d80d06-efe9-4839-9a9e-df41c76f3c3e}; path=/

If you're using Postman is takes care of holding onto this value for all other requests, but if you're not you need to make sure it's set in the headers of all subsequent requests to HDID.

Creating a VMware ESX Agentless Node

First of all we need a node that can communicate with our vCenter, in HDID we call this a VMware ESX Agentless Node.

All Agentless Nodes are created by sending a POST request to the Node Manager on the HDID master, like so:
https://MASTER_IP/HDID/master/NodeManager/objects/VMwareESXAgentlessNodes/
 
With the following JSON body:
 
{   "nodeType": "VMwareESX",   "name": "MyVMwareNode",   "resourceGroup": null,   "configuration": {     "proxyName": "Fibre3@BX-UXUVZY-RT478H-CV2IZI-SUKNE8[0-1-1]",     "enableVADP": true,     "server": "my_vmware_server",     "authentication": {       "username": "my_vmware_username",       "password": "my_vmware_password" }     }   }
 
The tricky bit in the above body is the proxyName. In HDID this is the node that will actually do the communication with (in this instance) VMware. In this simple blog post we're going to be using a single HDID master node, but the proxy could just as easily be another HDID client node.

In order to figure out our proxy node we need to perform a GET request against the Node Manager for a list nodes, with a query to select only OS nodes, like so:
 
https://MASTER_IP/HDID/master/NodeManager/objects/Nodes/?query=(type+IN+(%22OSHost%22))
 
Note: The above shows a very simple example of filtering the response by adding a query, not all fields can be used in queries so be sure to check your responses carefully.

The response body to our GET request should look like so:
{
"node": [
{
"id": "Fibre3@BX-UXUVZY-RT478H-CV2IZI-SUKNE8[0-1-1]",
"name": "Fibre3",
"resourceId": "4294967297",
"agentless": false,
"version": "6.7",
"osDetails": {
"os": "eOS_WINDOWS",
"osVersion": "Enterprise",
"osVariant": "Windows 2008 R2",
"architecture": "x64",
"chassis": "Virtual",
"osLevel": "Server"
},
"networkDetails": {
"iPAddresses": [
"192.168.45.147"
]
},
"stateInfo": {
"authorized": true,
"hasRules": true,
"upToDate": true,
"hubState": "Connected",
"activeRules": 1543933922,
"connected": true,
"accessible": true
},
"master": true,
"type": "OSHost",
"filterAvailable": true,
"nodeAttributes": []
}
],
"pageInfo": {
"totalCount": 1,
"end": true
}
}


This JSON response is a bit more complex, but most importantly contains an array of Node objects. For more complex setups the nodes can be filtered by name and OS variants in order to make queries easier.

For our purposes selecting the Proxy is simple, as there is only a single OS node, which also contains the attribute
 

"master":true 


Now we have the Node ID of the Node we wish to use as the Proxy, we can create our VMware ESX Agentless node. The successful response body should look like this:

 

{ "name": "MyVmwareSetup", "configuration": { "server": "my_vmware_server", "proxyName": "Fibre3@BX-UXUVZY-RT478H-CV2IZI-SUKNE8[0-1-1]", "authentication": { "username": "my_vmware_username" }, "enableVADP": true }, "timestamp": "2018-12-04T14:39:26", "id": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]"}


Note: the password is not returned, and even if queried will only return an encrypted string

Browsing VMware

Now that we have an VMware ESX Agentless node, we can browse our VMware setup just as we can from the HDID user interface. Except that we can take a shortcut. In order to start browsing VMware items in the HDID user interface we first need to create a Policy, but using the REST API we can just browse immediately and make the Policy later.

Lets get a list of all Virtual Machines that are available in our vCenter, using a GET request to our new VMware ESX Agentless node, like so:

https://MASTER_IP/HDID/MyVmwareSetup%4000-C666DE-6FF0F0-425D84-DAB6F1%5B0-1-6%5D/VMwareBrowser/objects/VirtualMachines/?order-by=name+ASC 


I've added an "order-by" in here just so that the response makes sense to us humans.

The response body we get should look like this:
 

"virtualMachine": [ { "id": "my_vmware_server/VirtualMachine:vm-1048", "name": "2008R2-Test2", "v2i": false,"vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, { "id": "my_vmware_server/VirtualMachine:vm-1044", "name": "2008R2-Test3", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, { "id": "my_vmware_server/VirtualMachine:vm-1045", "name": "2008R2-Test4", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" },{ "id": "my_vmware_server/VirtualMachine:vm-15973", "name": "2008R2Test1", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, { "id": "my_vmware_server/VirtualMachine:vm-15977", "name": "2008R2Thick", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, .....


And voila, a list of our Virtual Machines with their ID's (a concatenation of their vCenter server, type and MOREF)

Perhaps though we'd rather find some tags to use in our selection, we can make a slightly different GET request to our VMware ESX Agentless node:
 

https://MASTER_IP/HDID/MyVmwareSetup%4000-C666DE-6FF0F0-425D84-DAB6F1%5B0-1-6%5D/VMwareBrowser/objects/Tags/?order-by=name+ASC

Giving a response body like so:
 

{ "managedObject": [ { "id": "my_vmware_server/Tag:Andrew's", "name": "Andrew's", "isBrowsable": true, "type": "eTag", "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" }, { "id": "my_vmware_server/Tag:Block", "name": "Block", "isBrowsable": true, "type": "eTag", "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" }, { "id": "my_vmware_server/Tag:BlockTest", "name": "BlockTest", "isBrowsable": true, "type": "eTag", "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" }, { "id": "my_vmware_server/Tag:Development", "name": "Development", "isBrowsable": true, "type": "eTag", "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" },....


This time we get a list of ManagedObjects as VMware tags can be attached to any object.

If you look carefully at the above response you'll also see the attribute:

"isBrowsable": true


This is because the Managed Object interface allows us to traverse the VMware node hierarchy. Now that we have the Tag ID, we can use it to find Managed Objects that have that tag.

Here's how we construct a GET request that allows us to see what Managed Objects are tagged with the "Development" tag:

https://MASTER_IP/HDID/MyVmwareSetup%4000-C666DE-6FF0F0-425D84-DAB6F1%5B0-1-6%5D/VMwareBrowser/objects/ManagedObjects/my_vmware_setup%2FTag%3ADevelopment/collections/ManagedObjects


Note: We need to escape the URL characters in the Tag ID

This gives the following response body:
 

{ "managedObject": [ { "id": "my_vmware_server/VirtualMachine:vm-363", "name": "[AMC DEV] WIN-73ETL3VUO31", "isBrowsable": false, "type": "eVirtualMachine", "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" } ], "pageInfo": { "totalCount": 1, "end": true }}


Which shows us all the Managed Objects with the Tag "Development", in this case a single Virtual Machine.

This Managed Object request can be used for any Managed Object that is marked as "isBrowsable". Asking for the Managed Object collections of any Managed Object allows us to see it's children, or sub-nodes.

The generic Managed Object GET requests look like this:
 

https://MASTER_IP/HDID/{vmware_esx_agentless_node_id}/VMwareBrowser/objects/ManagedObjectshttps://MASTER_IP/HDID/{vmware_esx_agentless_node_id}/VMwareBrowser/objects/ManagedObjects/{managed_object_id_URL_encoded}/collections/ManagedObjects


The first GET request will return the available Datacenters for the specified VMware setup, and the second will allow tree traversal.

Finding Objects by Name

So far I've covered browsing a VMware setup, which is selecting existing objects by ID, but what if you wanted to select by name?

In order to this we need to build a small JSON document and make use of a call that's only available to the REST API.

The VMwareBrowser has a preview method, that will accept a small section of a HDID Policy and return the resulting list of Virtual Machines.

The preview method responds to a PUT and is found here:
 

https://MASTER_IP/HDID/MyVmwareSetup%4000-C666DE-6FF0F0-425D84-DAB6F1%5B0-1-6%5D/VMwareBrowser/services/ManagedObjects/actions/preview/invoke


And the JSON required looks like so:
 

{ "include": [{ "name": "Win7*", "type": "eVirtualMachine", "_selected": false }], "exclude": [], "vmwareNodeId": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]"}


Which returns the following response:
 

{ "virtualMachine": [ { "id": "my_vmware_server/VirtualMachine:vm-2165", "name": "Win7-HDID55", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, { "id": "my_vmware_server/VirtualMachine:vm-2118", "name": "Win7x32", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" }, { "id": "my_vmware_server/VirtualMachine:vm-2135", "name": "Win7-03", "v2i": false, "vadp": true, "isBrowsable": false, "nodeID": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]", "type": "eVirtualMachine" } ], "pageInfo": { "totalCount": 3, "end": true }}


You can also include Managed Objects that have been selected too by adding them to the JSON document like so:
 

{ "include": [{ "name": "Win7*", "type": "eVirtualMachine", "_selected": false }, { "id" : "my_vmware_server/Folder:group-v375", "type": "eFolder", "_selected":true }], "exclude": [], "vmwareNodeId": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]"}


Now that we've learned about VMware selection using the REST API we can put our selection into a HDID Policy, and create it.

Creating a Policy

The PolicyHandler on the master is responsible for creating Policies, and it can be found here:
 

https://MASTER_IP/HDID/master/PolicyHandler/objects/Policies/


The full Policy JSON document looks like so:
 

{ "name": "MyTestPolicy", "description": "", "classificationChains": [{ "items": [{ "typeName": "vmwareesx", "displayName": "VMware", "normalizedTypeName": "vmwareesx", "propertyName": "vmware", "category": "Hypervisor", "_policyItemType": "classifications", "vmwareEsx": { "include": [{ "name": "Win7*", "type": "eVirtualMachine", "_selected": false }, { "id" : "my_vmware_server/Folder:group-v375", "type": "eFolder","_selected":true }], "exclude": [], "vmwareNodeId": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" }, "_selected": false }] }], "operationChains": [{ "items": [{ "typeName": "Local Snapshot", "displayName": "Snapshot", "normalizedTypeName": "snapshot", "propertyName": "snapshot", "category": "Standard", "_policyItemType": "operations", "snapshot": { "label": "Snapshot", "mode": { "snapshotMode": "eSNAPSHOT_HARDWARE", "hardwareType": "eSNAPSHOT_HITACHI_BLOCK" }, "run": { "mode": "eRUN_ANYTIME", "scheduleName": "" }, "recoveryPointObjective": { "period": 8, "units": "eRPO_HOURS" },"retention": { "period": 1, "units": "eRETENTION_WEEKS" }, "recoveryTimeObjective": { "incrementalTapeBackup": false }, "sourceOption": { "quiesceApplications": true, "preScriptEnabled": false, "postScriptEnabled": false }, "type": { "mode": "eTRANSFER_ASYNCHRONOUS_JOURNALED" } }, "_selected": false }] }]}


You can see our VMware JSON in the above, as well as an operation to perform a Hardware Snapshot, with its associated settings

If you're looking at changing these it is easiest to use HDID User Interface with your browsers developer tools enabled in order to find out the correct keys and values for your requirements.

If the request is successful then we get the Policy returned with it's ID:

 

{ "id": "cc6d61935ca045c8a16be74e9357573a", "classificationChains": [ { "items": [ { "typeName": "vmwareesx", "vmwareEsx": { "include": [ { "name": "Win7*", "type": "eVirtualMachine" }, { "id": "my_vmware_server/Folder:group-v375", "name": "Andrew", "type": "eFolder" } ], "exclude": [], "vmwareNodeId": "MyVmwareSetup@00-C666DE-6FF0F0-425D84-DAB6F1[0-1-6]" }, "id": 1, "propertyName": "vmwareEsx" } ] } ],"operationChains": [ { "items": [ { "typeName": "Local Snapshot", "snapshot": { "label": "Snapshot", "retention": { "units": "eRETENTION_WEEKS", "period": 1 }, "recoveryPointObjective": { "units": "eRPO_HOURS", "period": 8 }, "run": { "mode": "eRUN_ANYTIME"}, "sourceOption": { "quiesceApplications": true, "preScriptEnabled": false, "postScriptEnabled": false }, "mode": { "snapshotMode": "eSNAPSHOT_HARDWARE", "hardwareType": "eSNAPSHOT_HITACHI_BLOCK" } }, "id": 1, "propertyName": "snapshot" } ] } ], "description": "", "nextClassificationId": 2, "nextOperationId": 2, "version": 2, "permissions": [ { "propertyType": "pt_user", "pt_user": { "upn": "Administrator@master" }, "access": "eA_READWRITE" } ], "name": "MyTestPolicy"}


And there we have it, you can now build, test and create a HDID Vmware Policy using just the REST API. Thanks for reading and feel free to put questions into the comments section.


#ThoughtLeadership
#DataProtection
#Blog
1 comment
9 views

Permalink

Comments

05-19-2022 14:00

Excellent !!