Monthly Archives: April 2017

Add PowerShell Host fails in vRO 7.x

While making an attempt to add PowerShell host in vRO 7.x fails with an error

Initial Error: ‘Add a PowerShell host/item8’, state: ‘failed’, business state: ‘null’, exception: ‘Clients credentials have been revoked (18) (Dynamic Script Module name : addPowerShellHost#12)’
workflow: ‘Add a PowerShell host’ (EF8180808080808080808080808080803D80808001270557368849c62c352aa82)
|  ‘attribute’: name=errorCode type=string value=Clients credentials have been revoked (18) (Dynamic Script Module name : addPowerShellHost#12)

investigation into this came up with some very basic issues.

vRO was configured with External MS SQL Db which was being authorized by particular AD account credentials, and that account itself was locked out. This might have happened due to multiple wrong password logon attempts and that created communication between vRO and DB server to fail.

this was tracked by going in vRO configurator page where it was throwing error:

Error! Error occured while retrieving nodes configuration. org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection
Error in both the vRO node is: couldn’t connect to database server.

but when checked with DB admin, they said DB is healthy enough and vRO node was able to reach DB server via ping as well. But when tried to test connection with service account, it was found to be locked. Unlocking same resolve everything, and was able to add powershell host also successfully.


Network and Security Inventory data collection fails in vRA 7.1

One of the customer reporting that their vRA 7.1 has started reporting deployment failures, and they were suspecting that this is happening due to Network and Security inventory data collection failures can be seen in vRA Infrastrastructure -> Compute Resource tab under all the Compute Resources.

Customer also revealed that they recently changed vRA -> External vRO -> NSX plugin configuration user credentails with a different username than the one in use earlier. and they were under the impression that due to this probably they started noticing Inventory data collection for network and security is failing now.

looking into Infrastracture -> monitoring -> logs

Error logs can be seen are as bellow:
Workflow ‘vSphereVCNSInventory’ failed with the following exception:
vRealize Orchestrator returned an error: Not Found.

DEM Worker at the same time was reporting errors as listed bellow:

2017-04-11T02:31:47.382Z CUA44494VPA100 vcac: [component=”iaas:DynamicOps.DEM.exe” priority=”Error” thread=”2768″] [sub-thread-Id=”52″  context=””  token=””]
Workflow ‘vSphereVCNSInventory’ failed with the following exception:
System.Net.WebException: vRealize Orchestrator returned an error: Not Found.
at DynamicOps.VcoModel.Common.VcoClient.DecodeJsonResponse(IRestResponse response)    at DynamicOps.VcoModel.Common.VcoInventoryReader.ReadInventory(VcoInventoryItemToken inventoryToken, String queryObject)    at DynamicOps.VCNSModel.Workflows.vSphereVCNSInventory_CompiledExpressionRoot.InvokeExpression(Int32 expressionId, IList`1 locations, ActivityContext activityContext)
at Microsoft.CSharp.Activities.CSharpValue`1.Execute(CodeActivityContext context)

while Server.log of Orchestrator node was reporting following:

2017-04-11 01:51:05.155-0400 [http-nio-] WARN  {} [SDKFinder] Unable to execute ‘fetchRelation’ for type : EdgePage : java.lang.NumberFormatException: For input string: “389,636,1012,2014,2020”
at sun.reflect.GeneratedMethodAccessor409.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke( at java.lang.reflect.Method.invoke( at ch.dunes.vso.sdk.DirectInvoker.invoke( at ch.dunes.vso.sdk.SDKPluginFactoryInvoker.fetchRelation( at ch.dunes.vso.sdk.SDKFinder.fetchRelation(
at ch.dunes.vso.sdk.SDKFinder._findRelation(
at ch.dunes.vso.sdk.SDKFinder.findRelation(
at ch.dunes.vso.sdk.ModulesFactory.findRelation( at com.vmware.o11n.sdk.EnhancedScriptingSDK.findRelation(

in this environment, this was going on since last 1 year, which cusotmer failed to notice, and I found that they are using vRO-NSX plugin version 1.0.4 which is affected by a known issue.

VMware KB
where even if Network and Security inventory collection fails, we don’t have to worry about it because this is happening due to vCO-NSX plugin version 1.0.4 which is in use currently in this environment, solution to this is included in vRO-NSX plugin 1.1 as mentioned in the quoted KB

As long as your vRO end point data collection is successful, it’s going to still let you use all the NSX components in your blueprint and deployments should not fail. If you still find deployments failing, I would suggest to open a Support Request with VMware.


Change Lease to Never Expire in vRA 7.1

working with vRA 7.1

  1. created a blueprint with Minimum Lease as 10 Days and left Maximum lease as empty.
  2. created a deployment out of it with 10 days lease and now I have a need to change the lease of this same deployment to Never Expire.
  3. Action entitlement is there to do this and I went ahead with Change Lease, a dialog box popped up with Lable “Change the lease for a machine. Leave empty for indefinite” with Expiration date and time boxes.
  4. since the requirement was to set lease to Never expire, left Expiration date empty and tried to submit the request.
  5. Ended up seeing following error “An unexpected error occurred while validating your request.”
  6. Since deployment was created from Blueprint which was configured with no Max Lease duration, in vRA 7.1 if you try to submit above dialog box with some date which is 20 years from now also, that works perfectly fine.

Only point of concern is the lable on Change Lease dialog box which says Leave empty for indefinite, no matter what you do that doesn’t work in vRA 7.1

I have explored same part in vRA 7.2 where Change Lease dialog box lable is changed and if I try step 1 through 5 as above example, it works perfectly well. So I would say this issue is resolved in vRA 7.2


Unallocate IP from IP Range in vRA 6.2.x

This came as a requirement from one of the clients. They are using vRA 6.2.2 where they have migrated few Deployments (vSphere VMs) from one Reservation to Other and ended up in a situation that old Reservation based Network profile which was having IP Range configured from where these set of VMs were holding IPs allocation remained as it was and at the same time same set of VMs which are now moved to new reservation from where they picked up new IPs from the local IP range of that reservation.

Simple ask from customer was to make sure that old IP Range based IPs which were allocated in the name of moved VMs should be released so those can be used by new deployments

Little research on IaaS sql DB and found a way to resolve this.

Used following select statement to see if particular VM Name is having multiple network IP allocated againsts it.

select * from staticipv4address where virtualmachineid in (select virtualmachineid from virtualmachine where virtualmachinename in (”));

if above statement returns 2 or more entries for one VM, execute following update statement to set IP address to be unallocated which is not being used.

update staticipv4address set virtualmachineid = NULL, StaticIPv4AddressState = 1 where virtualmachineid in (select virtualmachineid from virtualmachine where virtualmachinename in (”)) and IPv4Address=”;

I did make sure to keep Backup of IaaS DB in case if that’s needed.
There will not be a need of vRA service outage.