Description of problem: provisioning two vms in one provisioning operation leads to an error during poll_clone_complete : MIQ(ManageIQ::Providers::Vmware::InfraManager::Provision#provision_error) [[RuntimeError]: The operation is not supported on the object.] encountered during phase [poll_clone_complete] Version-Release number of selected component (if applicable): 5.6.1.2 How reproducible: very frequently on customer environment Steps to Reproduce: 1. provision 2 vms from a template 2. 3. Actual results: the provision stops at the 2nd vm with the error MIQ(ManageIQ::Providers::Vmware::InfraManager::Provision#provision_error) [[RuntimeError]: The operation is not supported on the object.] encountered during phase [poll_clone_complete] Expected results: the provision of both vms passes. customer also hit by memory issues but they do notseem to affect this issue directly Additional info: more data from the customer environment available in private notes
Billy - Please work with Tina on this issue, but I think you will also want to consult with Adam as the returned error is coming from VMware. Felix - Can you verify if this is standard provisioning or service provisioning? Also, would like to know from the customer if they were provisioning this way in the past or did they only start provisioning multiple VMs recently? Since the error is being raised from VMware we would want to know if any errors or recent changes within the VMware environment might be contributing to this failure.
I am seeing some broker errors in the evm.log. 362964 [----] E, [2016-10-10T15:16:44.146271 #22526:377990] ERROR -- : MIQ(MiqFaultTolerantVim._connect) EMS: [Noida] [Broker] Unable to connect to: [innoivcs1.pbi.global.pvt] beca use Broker is not available (connection error). 362965 [----] W, [2016-10-10T15:16:44.151091 #22526:377990] WARN -- : MIQ(MiqVimBrokerWorker.broker_unavailable) The following error was encountered, 'Broker is not available (con nection error).', the broker server should be restarted on the next heartbeat 362966 [----] I, [2016-10-10T15:16:44.194687 #22526:377990] INFO -- : MIQ(MiqQueue.put) Message id: [3000173993630], id: [], Zone: [worker noida], Role: [], Server: [b28a3e1e-110 4-11e1-89a4-005056a8007d], Ident: [miq_server], Target id: [], Instance id: [3000000000002], Task id: [], Command: [MiqServer.message_for_worker], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [3000001045086, "broker_unavailable"] 362967 [----] W, [2016-10-10T15:16:44.195072 #22526:377990] WARN -- : MIQ(ManageIQ::Providers::Vmware::InfraManager#with_provider_connection) Reported the broker unavailable 362968 [----] E, [2016-10-10T15:16:44.195263 #22526:377990] ERROR -- : MIQ(MiqQueue#m_callback) Message id: [3000173993615]: Broker is not available (connection error). 362969 [----] E, [2016-10-10T15:16:44.195365 #22526:377990] ERROR -- : MIQ(MiqQueue#m_callback) backtrace: /var/www/miq/vmdb/gems/pending/util/miq_fault_tolerant_vim.rb:191:in `res cue in _connect_with_broker'
There seem to be two different errors here, I see "The operation is not supported on the object." on multiple appliances but the broker_unavailable error only on one. "The operation is not supported on the object" is a general fault [0] thrown typically when: 1. Trying to run a vc-only method on a host 2. A feature is disabled due to licensing 3. An invalid object (e.g.: Folder) was selected to do an operation (e.g.: cloneVM_Task) #3 is most likely in this case, I'll go through the clone task to see if anything stands out as invalid. Can the customer try to clone a VM through the vCenter client with exactly the same selections and user as one of the failed provision tasks? [0]: https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vmodl.fault.NotSupported.html
the broker errors are likely due to the memory issues of the environment and can be discarded. I've asked the customer, waiting for his feedback.
Felix, Any feedback from the customer?
the customer's issue disappeared after application of scripts and memory changes.
Felix, Can I close this ticket ? Billy
I think yes
Thanks