Bug 1617520

Summary: Unable to provision VM with "choose automatic option"
Product: Red Hat CloudForms Management Engine Reporter: Saurabh Heda <sheda>
Component: ProvisioningAssignee: Lucy Fu <lufu>
Status: CLOSED CURRENTRELEASE QA Contact: Jaroslav Henner <jhenner>
Severity: medium Docs Contact:
Priority: high    
Version: 5.9.3CC: dmetzger, gmccullo, jhardy, jhenner, mfeifer, obarenbo, simaishi, smallamp
Target Milestone: GAKeywords: TestOnly, ZStream
Target Release: 5.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.10.0.17 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1631557 (view as bug list) Environment:
Last Closed: 2019-02-12 16:51:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1631557    

Description Saurabh Heda 2018-08-16 03:55:33 UTC
Description of problem:
When deploying virtual machines from templates in CloudForms, when we check Choose Automatic option on the Environment tab it fails

Version-Release number of selected component (if applicable):
5.9.3.4

Steps to Reproduce:
(1) Compute -> infrastructure -> Select Virtual Machine
(2) Select a template, Life cycle - Select provisioning of the virtual machine using this template
(3) Select the Environment tab and Select Choose Automatically.

Additional info:

[2018-08-08T12:15:13.415389 #9008:de4c34] ERROR -- : MiqAeServiceModelBase.ar_method raised: <RuntimeError>: <Resource <Storage> <111000000000005:VDP_Datastore01> is not an eligible resource for this provisioning instance.>

Comment 3 Lucy Fu 2018-08-16 21:30:52 UTC
What is the relationship between "vSAN Default Storage Policy" and VDP_Datastore01? Is VDP_Datastore01 in "vSAN Default Storage Policy"?

Comment 5 Saurabh Heda 2018-08-17 09:11:15 UTC
Datastore                                               Type        Policy                                      NOTE
------------------------------------------------------------------------------------------------------
VxRail-Virtual-SAN-Datastore         			vSAN      vSAN Default storage policy    		DellEMC VxRail

VDP_Datastore                                     	NFS         n/a                                         NFS

DE700XXXXX69                                     	VMFS        n/a                                         ESXi host storage

DE700XXXXX70                                     	VMFS     	n/a                                         ESXi host storage

DE700XXXXX71                                       	VMFS     	n/a                                         ESXi host storage

Comment 6 Lucy Fu 2018-08-17 13:29:52 UTC
This is the log extracted from comment #2.

[----] E, [2018-08-08T12:15:13.415230 #9008:de4c34] ERROR -- : The following error occurred during instance method <set_resource> for AR object <#<ManageIQ::Providers::Vmware::InfraManager::Provision id:111000000000048, description: "Provision from [test-template-custom-nicx1] to [te...", state: "pending", request_type: "template", userid: "admin", options: {:initial_pass=>true, :src_vm_id=>[111000000000044, "test-template-custom-nicx1"], 
...
:placement_storage_profile=>[111000000000013, "vSAN Default Storage Policy"], 

:placement_cluster_name=>[nil, nil], :cluster_filter=>[nil, nil], :host_fi  12195 lter=>[nil, nil], :ds_filter=>[nil, nil], :placement_host_name=>[111000000000001, "esx-b03.vxrail.in-dpss"], :placement_ds_name=>[nil, nil], :rp_filter=>[nil, nil], :placement_auto=>[true, 1], ....>
[----] E, [2018-08-08T12:15:13.415389 #9008:de4c34] ERROR -- : MiqAeServiceModelBase.ar_method raised: <RuntimeError>: <Resource <Storage> <111000000000005:VDP_Datastore01> is not an eligible resource for this provisioning instance.>

"vSAN Default Storage Policy" was selected as the storage profile which caused the provision failure.

Please try provision without storage profile or select a storage profile that has VDP_Datastore01.

Comment 7 Saurabh Heda 2018-08-20 04:32:06 UTC
Customer has mentioned that they are just using "Choose Automatically" option and explicitly they are not selecting the datastore or the Storage Policy.

Thanks
Saurabh

Comment 9 Lucy Fu 2018-08-20 18:44:13 UTC
Please back-up original file vmware_best_fit_least_utilized.rb under ManageIQ/Infrastructure/VM/Provisioning/Placement.class/__methods__. 
Then apply the modified version.

Try provision with the same template with "Choose Automatically" option.
Thanks.

Comment 12 CFME Bot 2018-09-05 11:48:33 UTC
New commit detected on ManageIQ/manageiq-automation_engine/master:

https://github.com/ManageIQ/manageiq-automation_engine/commit/78b43da37e8a07548428146071ec242dc215d9a2
commit 78b43da37e8a07548428146071ec242dc215d9a2
Author:     Lucy Fu <lufu>
AuthorDate: Tue Sep  4 10:09:44 2018 -0400
Commit:     Lucy Fu <lufu>
CommitDate: Tue Sep  4 10:09:44 2018 -0400

    Expose storage_profiles association in Storage's service model.

    https://bugzilla.redhat.com/show_bug.cgi?id=1617520

 lib/miq_automation_engine/service_models/miq_ae_service_storage.rb | 1 +
 1 file changed, 1 insertion(+)

Comment 13 CFME Bot 2018-09-20 20:00:12 UTC
New commits detected on ManageIQ/manageiq-content/master:

https://github.com/ManageIQ/manageiq-content/commit/7d90a0bd588c887b9d9fb76bae241d373487792c
commit 7d90a0bd588c887b9d9fb76bae241d373487792c
Author:     Lucy Fu <lufu>
AuthorDate: Wed Aug 29 15:18:51 2018 -0400
Commit:     Lucy Fu <lufu>
CommitDate: Wed Aug 29 15:18:51 2018 -0400

    Refactor Vmware_best_fit_least_utilized method.

    https://bugzilla.redhat.com/show_bug.cgi?id=1617520

 content/automate/ManageIQ/Infrastructure/VM/Provisioning/Placement.class/__methods__/vmware_best_fit_least_utilized.rb | 89 +-
 spec/automation/unit/method_validation/vmware_best_fit_least_utilized_spec.rb | 99 -
 spec/content/automate/ManageIQ/Infrastructure/VM/Provisioning/Placement.class/__methods__/vmware_best_fit_least_utilized_spec.rb | 131 +
 3 files changed, 190 insertions(+), 129 deletions(-)


https://github.com/ManageIQ/manageiq-content/commit/aa0c72c03d132b32aa9c7112882a361e521eb07a
commit aa0c72c03d132b32aa9c7112882a361e521eb07a
Author:     Lucy Fu <lufu>
AuthorDate: Fri Sep  7 14:30:16 2018 -0400
Commit:     Lucy Fu <lufu>
CommitDate: Fri Sep  7 14:30:16 2018 -0400

    Honor the selected storage profile.

    https://bugzilla.redhat.com/show_bug.cgi?id=1617520

 content/automate/ManageIQ/Infrastructure/VM/Provisioning/Placement.class/__methods__/vmware_best_fit_least_utilized.rb | 8 +
 spec/content/automate/ManageIQ/Infrastructure/VM/Provisioning/Placement.class/__methods__/vmware_best_fit_least_utilized_spec.rb | 25 +-
 2 files changed, 31 insertions(+), 2 deletions(-)

Comment 15 Jaroslav Henner 2018-10-23 16:06:29 UTC
I tried the steps in the bug description and it works.

Comment 16 Jaroslav Henner 2018-10-24 15:50:17 UTC
When provisioning on our env-nvc60-vcsa provider I got 

[----] E, [2018-10-24T11:41:25.240471 #11954:9eef80] ERROR -- : Q-task_id([r1_miq_provision_1]) MIQ(ManageIQ::Providers::Vmware::InfraManager::Provision#provision_error) [[RuntimeError]: pbm.fault.PBMFault.summary] encountered during phase [poll_clone_complete]
[----] E, [2018-10-24T11:41:25.240567 #11954:9eef80] ERROR -- : Q-task_id([r1_miq_provision_1]) (drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /opt/rh/cfme-gemset/gems/vmware_web_service-0.3.0/lib/VMwareWebService/MiqVimInventory.rb:2094:in `pollTask'
(drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /usr/share/ruby/drb/drb.rb:1624:in `perform_without_block'
(drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /usr/share/ruby/drb/drb.rb:1584:in `perform'
(drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /usr/share/ruby/drb/drb.rb:1668:in `block (2 levels) in main_loop'
(drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /usr/share/ruby/drb/drb.rb:1664:in `loop'
(drbunix:///tmp/MiqVimBroker20181024-1919-all7ws) /usr/share/ruby/drb/drb.rb:1664:in `block in main_loop'
/opt/rh/cfme-gemset/gems/vmware_web_service-0.3.0/lib/VMwareWebService/miq_fault_tolerant_vim.rb:41:in `block in method_missing'
/opt/rh/cfme-gemset/gems/vmware_web_service-0.3.0/lib/VMwareWebService/miq_fault_tolerant_vim.rb:86:in `_execute_with_broker'
/opt/rh/cfme-gemset/gems/vmware_web_service-0.3.0/lib/VMwareWebService/miq_fault_tolerant_vim.rb:75:in `_execute'
/opt/rh/cfme-gemset/gems/vmware_web_service-0.3.0/lib/VMwareWebService/miq_fault_tolerant_vim.rb:41:in `method_missing'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/cloning.rb:5:in `block in do_clone_task_check'
/var/www/miq/vmdb/app/models/mixins/provider_object_mixin.rb:6:in `block in with_provider_connection'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/vim_connect_mixin.rb:24:in `with_provider_connection'
/var/www/miq/vmdb/app/models/mixins/provider_object_mixin.rb:5:in `with_provider_connection'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/cloning.rb:3:in `do_clone_task_check'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/state_machine.rb:28:in `poll_clone_complete'
/var/www/miq/vmdb/app/models/miq_request_task/state_machine.rb:21:in `signal'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/state_machine.rb:24:in `start_clone_task'
/var/www/miq/vmdb/app/models/miq_request_task/state_machine.rb:21:in `signal'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/state_machine.rb:12:in `determine_placement'
/var/www/miq/vmdb/app/models/miq_request_task/state_machine.rb:21:in `signal'
/opt/rh/cfme-gemset/bundler/gems/cfme-providers-vmware-5558b303efbc/app/models/manageiq/providers/vmware/infra_manager/provision/state_machine.rb:3:in `create_destination'
/var/www/miq/vmdb/app/models/miq_request_task/state_machine.rb:21:in `signal'
/var/www/miq/vmdb/app/models/miq_provision/state_machine.rb:4:in `run_provision'
/var/www/miq/vmdb/app/models/miq_request_task/state_machine.rb:21:in `signal'
/var/www/miq/vmdb/app/models/miq_provision_task.rb:14:in `do_request'
/var/www/miq/vmdb/app/models/miq_request_task.rb:201:in `execute'
/var/www/miq/vmdb/app/models/miq_queue.rb:455:in `block in dispatch_method'
/usr/share/ruby/timeout.rb:93:in `block in timeout'
/usr/share/ruby/timeout.rb:33:in `block in catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:33:in `catch'
/usr/share/ruby/timeout.rb:108:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:453:in `dispatch_method'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `block in deliver'
/var/www/miq/vmdb/app/models/user.rb:267:in `with_user_group'
/var/www/miq/vmdb/app/models/miq_queue.rb:430:in `deliver'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:104:in `deliver_queue_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:137:in `deliver_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:155:in `block in do_work'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `loop'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:149:in `do_work'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:329:in `block in do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:326:in `do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:127:in `start'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:22:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:402:in `block in start_runner_via_fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.4/lib/nakayoshi_fork.rb:23:in `fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:400:in `start_runner_via_fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:390:in `start_runner'
/var/www/miq/vmdb/app/models/miq_worker.rb:441:in `start'
/var/www/miq/vmdb/app/models/miq_worker.rb:271:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:154:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_worker.rb:154:in `times'
/var/www/miq/vmdb/app/models/miq_worker.rb:154:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:53:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server.rb:148:in `start'
/var/www/miq/vmdb/app/models/miq_server.rb:240:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:27:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:48:in `start'
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'


The machine creation failed. I do not know what it means, but I think it is not good, so setting back to ASSIGNED.

Comment 17 Lucy Fu 2018-10-24 19:26:54 UTC
Can you verify the cause of the provisioning failure? 

If it failed for the same reason reported in the BZ, then it makes sense to put the BZ back to the developer. If it failed for another reason, you may want to open a new BZ to track the issue.

To help identify the issue, it may help to do a provision without choosing automatic option.

Comment 18 Greg McCullough 2018-11-07 16:58:01 UTC
Jaroslav - Any update on this issue?  It is marked as a 5.10 blocker and has not been updated for 2 weeks.

Comment 19 Lucy Fu 2018-11-12 22:09:33 UTC
Change Status back to ON_QA for testing as still waiting for QE's response on comment 17.

Comment 20 Jaroslav Henner 2018-11-22 13:43:26 UTC
My automatic test passed with 5.10 on our vsphere6.7-nested so if there is a problem, it is probably something provider-specific.