Bug 1300559

Summary: Unable to retire VMs which are provisioned using cloudforms 3.0 version
Product: Red Hat CloudForms Management Engine Reporter: Prasad Mukhedkar <pmukhedk>
Component: AutomateAssignee: Tina Fitzgerald <tfitzger>
Status: CLOSED ERRATA QA Contact: Milan Falešník <mfalesni>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 5.4.0CC: cpelland, fdewaley, jhardy, jkrocil, jocarter, jprause, mfeifer, mkanoor, obarenbo, pmukhedk, tfitzger
Target Milestone: GAKeywords: ZStream
Target Release: 5.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.6.0.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1301149 (view as bug list) Environment:
Last Closed: 2016-06-29 15:32:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1300738, 1301149    
Attachments:
Description Flags
production automation_code.zip
none
lab automation.log (automation_task_509000000004421)
none
retire_autotask_prod_bad.log none

Description Prasad Mukhedkar 2016-01-21 07:08:01 UTC
Created attachment 1116826 [details]
production automation_code.zip

Description of problem:


We are using Cloudforms 3.2 (upgraded from 3.0 to 3.4) and we have a retirement state machine:
[/Customer/Intuit/Retirement/StateMachines/VMRetirementLocal/Default]

This state machine has a state called "DeleteFromProvider" that calls a factory instance to 
delete a vm:

[/ManageIQ/Infrastructure/VM/Retirement/StateMachines/Methods/RemoveFromProvider]

OnEntry of the state "DeleteFromProvider", a special method to dump
root is run:

[/Customer/Intuit/Retirement/StateMachines/VMRetirementLocal/update_retirement_status]

We find that the logs show a VM exists:

[----] I, [2016-01-19T18:51:07.597732 #32632:883239c]  INFO -- : Q-task_id([automation_task_702000000008929]) <AEMethod update_retirement_status> update_retirement_status - Root:<$evm.root> Attribute - vm_id: 702000000004926
[----] I, [2016-01-19T18:51:07.596606 #32632:883239c]  INFO -- : Q-task_id([automation_task_702000000008929]) <AEMethod update_retirement_status> update_retirement_status - Root:<$evm.root> Attribute - vm: pprdqbows40a.ie.intuit.net

In the factory method "remove_from_provider", if vm is not nil AND (vm has miq_provision OR tagged with lifecycle/retire_full), then it should retire.
But it does not retire. The log statements show that it never enters this block.

Further investigation showed that it looks like VM objects created in CF3.0 do not have a miq_provision reference while VM objects created in CF3.2 do have a miq_provision reference.


Should 3.0 original objects have been given a "dummy" reference? Should old objects be tagged with lifecyle/retire_full manually?

Is there any possiblity to add "miq_provision" reference to the old virtual machines? How? 

How viable is tagging old objects with lifecyle/retire_full manually workaround for this issue ?

I'm attaching the following:

(1) production automation.log (automation_task_702000000008929)
(2) production automation_code.zip
(3) lab automation.log (automation_task_509000000004421)


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Prasad Mukhedkar 2016-01-21 07:09:17 UTC
Created attachment 1116827 [details]
lab automation.log (automation_task_509000000004421)

Comment 2 Prasad Mukhedkar 2016-01-21 07:10:37 UTC
Created attachment 1116828 [details]
retire_autotask_prod_bad.log

Comment 4 Greg McCullough 2016-01-21 15:25:17 UTC
*** Bug 1300720 has been marked as a duplicate of this bug. ***

Comment 5 Greg McCullough 2016-01-21 15:29:05 UTC
*** Bug 1300738 has been marked as a duplicate of this bug. ***

Comment 6 Tina Fitzgerald 2016-01-21 16:09:22 UTC
Hi Prasad,

We're in the process of changing that code to remove the requirement for miq_provision and/or tags.  

Could you try the workaround which is to copy the remove_from_provider automate method to a writable, enabled domain and modify it to not rely on the miq_provision and/or tag?  

Using the Automate Explorer, 
1. Navigate to the/ManageIQ/Infrastructure/VM/Retirement/StateMachines/Methods/RemoveFromProvider class
2. Copy the remove_from_provider method to a writable enabled domain.
3. Modified the copied method to comment out the lines shown below:
....................

#category = "lifecycle"
#tag = "retire_full"

.........................
  when "remove_from_disk"
#    if vm.miq_provision || vm.tagged_with?(category, tag)
      $evm.log('info', "Removing VM:<#{vm.name}> from provider:<#{ems.try(:name)}>")
      vm.remove_from_disk(false)
      $evm.set_state_var('vm_removed_from_provider', true)
#    end

Here's my modified method:

#
# Description: This method removes the VM from the provider
#

# Get vm from root object
vm = $evm.root['vm']
#category = "lifecycle"
#tag = "retire_full"

removal_type = $evm.inputs['removal_type'].downcase
$evm.set_state_var('vm_removed_from_provider', false)

if vm
  ems = vm.ext_management_system
  case removal_type
  when "remove_from_disk"
#    if vm.miq_provision || vm.tagged_with?(category, tag)
      $evm.log('info', "Removing VM:<#{vm.name}> from provider:<#{ems.try(:name)}>")
      vm.remove_from_disk(false)
      $evm.set_state_var('vm_removed_from_provider', true)
#    end
  when "unregister"
    $evm.log('info', "Unregistering VM:<#{vm.name}> from provider:<#{ems.try(:name)}")
    vm.unregister
    $evm.set_state_var('vm_removed_from_provider', true)
  else
    $evm.log('info', "Unknown retirement type for VM:<#{vm.name}> from provider:<#{ems.try(:name)}")
    exit MIQ_ABORT
  end
end

Thanks,
Tina

Comment 8 CFME Bot 2016-01-22 17:37:03 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/3e1b9e236fff5ef0d346c60c206c437d5caf0261

commit 3e1b9e236fff5ef0d346c60c206c437d5caf0261
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Thu Jan 21 12:13:06 2016 -0500
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Thu Jan 21 12:24:28 2016 -0500

    Automate - Retirement - Remove miq_provision and/or tag code from
    cloud/infra remove_from_provider methods.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1300559

 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../unit/method_validation/remove_from_provider_spec.rb        |  1 -
 3 files changed, 6 insertions(+), 15 deletions(-)

Comment 9 CFME Bot 2016-02-10 19:53:35 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=ef777510e86bace4fae62c961a78197f4c95ef87

commit ef777510e86bace4fae62c961a78197f4c95ef87
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Thu Jan 21 12:13:06 2016 -0500
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Fri Jan 22 12:36:14 2016 -0500

    Automate - Retirement - Remove miq_provision and/or tag code from
    cloud/infra remove_from_provider methods.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1300559

 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../unit/method_validation/remove_from_provider_spec.rb        |  1 -
 3 files changed, 6 insertions(+), 15 deletions(-)

Comment 10 CFME Bot 2016-02-10 19:54:30 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=0b8f8d758237a37a9d29998dc48da5086410c1ea

commit 0b8f8d758237a37a9d29998dc48da5086410c1ea
Merge: e0f444e ef77751
Author:     Greg McCullough <gmccullo>
AuthorDate: Wed Feb 10 14:27:40 2016 -0500
Commit:     Greg McCullough <gmccullo>
CommitDate: Wed Feb 10 14:27:40 2016 -0500

    Merge branch 'cherry_pick_retirement_tag_fix' into '5.5.z'
    
    Automate - Retirement - Remove miq_provision and tag code
    
    from the cloud/infra remove_from_provider methods.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1301149
    
    Original:
    https://bugzilla.redhat.com/show_bug.cgi?id=1300559
    https://github.com/ManageIQ/manageiq/pull/6290
    
    Clean cherry-pick.
    
    See merge request !745

 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../Methods.class/__methods__/remove_from_provider.rb          | 10 +++-------
 .../unit/method_validation/remove_from_provider_spec.rb        |  1 -
 3 files changed, 6 insertions(+), 15 deletions(-)

Comment 12 Milan Falešník 2016-06-13 17:32:34 UTC
Checked in 5.6.0.10, as Tina pointed out a VM provisioned outside CFME and then retired should get retired and not deleted. It worked as expected.

Comment 13 Tina Fitzgerald 2016-06-13 18:55:05 UTC
Thanks Milan.

Comment 15 errata-xmlrpc 2016-06-29 15:32:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348