Bug 1202816

Summary: Rhev PXE provision error with monitoring power states
Product: Red Hat CloudForms Management Engine Reporter: Josh Carter <jocarter>
Component: AutomateAssignee: Brandon Dunne <bdunne>
Status: CLOSED ERRATA QA Contact: Dave Johnson <dajohnso>
Severity: medium Docs Contact:
Priority: high    
Version: 5.3.0CC: jhardy, mfeifer, mkanoor, obarenbo, ssainkar, sshveta, tfitzger
Target Milestone: GA   
Target Release: 5.4.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Machine state was stale in VMDB. As a result, the VM PXE provision failed with RhevmApi error. Code was updated to verify destination is off in the provider before queuing a refresh. VM PXE provisioning is now successful.
Story Points: ---
Clone Of:
: 1209653 (view as bug list) Environment:
Last Closed: 2015-06-16 12:53:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1209653    

Description Josh Carter 2015-03-17 13:43:50 UTC
Description of problem:

VM PXE provision fails with RhevmApi ERROR. 

[----] E, [2015-03-11T14:46:22.094404 #30493:77b014] ERROR -- : Q-task_id([miq_provision_123000000000007]) MIQ(MiqProvisionRedhatViaPxe#provision_error) [[RhevmApiError]: [Cannot run VM because it is in Powering Up status.]] encountered during phase [autostart_destination]
[----] E, [2015-03-11T14:46:22.094544 #30493:77b014] ERROR -- : Q-task_id([miq_provision_123000000000007]) /var/www/miq/lib/RedHatEnterpriseVirtualizationManagerAPI/rhevm_service.rb:234:in `parse_error_response'


It appears during the provision the power states of the instance is not being monitored within real time and this is causing the kickstart file to be removed before the install has been started. 

poll_destination_powered_off_in_vmdb

[----] I, [2015-03-11T14:45:55.680884 #15667:118e010]  INFO -- : Q-task_id([miq_provision_123000000000007]) Starting Phase <poll_destination_powered_off_in_vmdb>
[----] I, [2015-03-11T14:45:55.766041 #15667:118e010]  INFO -- : Q-task_id([miq_provision_123000000000007]) MIQ(MiqProvisionRedhatViaPxe#poll_destination_powered_off_in_vmdb) Vm [testecloud] is not yet powered off, will retry

[----] I, [2015-03-11T14:46:07.760368 #30493:77b014]  INFO -- : Q-task_id([miq_provision_123000000000007]) MIQ(MiqProvisionRedhatViaPxe#poll_destination_powered_off_in_vmdb) Vm [testecloud] is not yet powered off, will retry

[----] E, [2015-03-11T14:46:22.094404 #30493:77b014] ERROR -- : Q-task_id([miq_provision_123000000000007]) MIQ(MiqProvisionRedhatViaPxe#provision_error) [[RhevmApiError]: [Cannot run VM because it is in Powering Up status.]] encountered during phase [autostart_destination]


Version-Release number of selected component (if applicable):
cfme-5.3.3
rhev-3.5

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 Greg McCullough 2015-03-17 14:24:26 UTC
Brandon - I worked with Josh to track this one down and it seems pretty clear we are getting a slightly stale refresh that causes the state machine to progress from the poll_destination_powered_off_in_vmdb too early and causes an error.  We may need to make a live call to the provider to verify the VM machine state to avoid this.  Let's discuss.

Comment 5 CFME Bot 2015-04-07 20:25:59 UTC
New commit detected on manageiq/master:
https://github.com/ManageIQ/manageiq/commit/00c89a787495edb00f983762737a9bcce357bb1a

commit 00c89a787495edb00f983762737a9bcce357bb1a
Author:     Brandon Dunne <bdunne>
AuthorDate: Tue Apr 7 14:39:01 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Tue Apr 7 14:39:05 2015 -0400

    Verify destination is off in the provider before queueing a refresh
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1202816

 vmdb/app/models/miq_provision/state_machine.rb            |  2 +-
 vmdb/app/models/miq_provision_redhat/state_machine.rb     | 15 ++++++++++++++-
 .../models/miq_provision_redhat_via_iso/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat_via_pxe/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat/state_machine_spec.rb     |  9 ++++++++-
 5 files changed, 25 insertions(+), 5 deletions(-)

Comment 6 CFME Bot 2015-04-07 22:30:51 UTC
New commit detected on cfme/5.3.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=a79fb0c0fa447a10162da75505b6390556abb73d

commit a79fb0c0fa447a10162da75505b6390556abb73d
Author:     Brandon Dunne <bdunne>
AuthorDate: Tue Apr 7 14:39:01 2015 -0400
Commit:     Brandon Dunne <bdunne>
CommitDate: Tue Apr 7 18:00:05 2015 -0400

    Verify destination is off in the provider before queueing a refresh
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1202816

 vmdb/app/models/miq_provision/state_machine.rb            |  2 +-
 vmdb/app/models/miq_provision_redhat/state_machine.rb     | 15 ++++++++++++++-
 .../models/miq_provision_redhat_via_iso/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat_via_pxe/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat/state_machine_spec.rb     |  9 ++++++++-
 5 files changed, 25 insertions(+), 5 deletions(-)

Comment 7 CFME Bot 2015-04-07 22:30:55 UTC
New commit detected on cfme/5.3.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=2be582a90e74e300559f796038672d313ab41e9e

commit 2be582a90e74e300559f796038672d313ab41e9e
Merge: 7c0d9b1 a79fb0c
Author:     Greg McCullough <gmccullo>
AuthorDate: Tue Apr 7 18:26:31 2015 -0400
Commit:     Greg McCullough <gmccullo>
CommitDate: Tue Apr 7 18:26:31 2015 -0400

    Merge branch '53z_check_destination_is_off_in_provider_first' into '5.3.z'
    
    Verify destination is off in the provider before queueing a refresh
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1202816
    
    See merge request !28

 vmdb/app/models/miq_provision/state_machine.rb            |  2 +-
 vmdb/app/models/miq_provision_redhat/state_machine.rb     | 15 ++++++++++++++-
 .../models/miq_provision_redhat_via_iso/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat_via_pxe/state_machine.rb  |  2 +-
 .../models/miq_provision_redhat/state_machine_spec.rb     |  9 ++++++++-
 5 files changed, 25 insertions(+), 5 deletions(-)

Comment 9 Shveta 2015-04-24 22:44:47 UTC
VM PXE provision is working in build 5.4.0.0.22.20150420163946_26004d1

Comment 11 errata-xmlrpc 2015-06-16 12:53:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1100.html