Bug 1325167

Hi Nikhil,

The default retirement state machine powers off the vm(if necessary), removes the vm from the provider, and deletes the vm from the vmdb.

I suspect the reason retirement is not removing the vm is because we had code that prevented the vm removal unless, the vm was provisioned by us, or the vm was tagged with lifecycle/full.

1. Was the vm provisioned there?
2. Was the retirement code changed not to remove the vm? 

There are 3 retirement settings shown in the vm summary: retired(which would be true/blank), retires_on(the date), and retirement_state(retiring, retired, error, or blank).  

3. What is the value of those settings for the vm?
4. What is the power state of the vm afterwards?

Thanks,
Tina

Hi Tina,

I know, CFME only delete the instance completely from provider if it is provisioned by CFME or tagged as retire_full:
~~~
# Get vm from root object
vm = $evm.root['vm']
category = "lifecycle"
tag = "retire_full"
~~~

Note that in this case,the instance was created using the openstack console and it is not tagged with retire_full option. 

We tried to retire using the Cloudforms console. As expected the instance went into a "Suspended" state. 

Retirement code is not changed, it is default.

After resuming this instance, the power state of the VM in the cloudforms is "Retired" and in the openstack is "active".

As per my understanding, once the instance state is changed to "Retired" on CloudForms, even if someone tries to resume it from provider side, it should not resume or CloudForms status should be updated to "ON".

Regards,
Nikhil

Hi Nikhil,

Thanks for the information.

"After resuming this instance, the power state of the VM in the cloudforms is "Retired" and in the openstack is "active"."

Cloudforms should show the VM in a "Retired" state after retirement. What is the actual CloudForms power state after retirement? Is it still "Suspended"?

After resuming the instance, what is the CloudForms power state?

We have a builtin policy that should deny any power on events and power off any retired VMs.

I'm going to run some tests here. If you have a test environment setup, could you send me the details so I can check it out? 

Thanks,
Tina

Hi Nikhil,

What instance can I look at that exhibits the "retired", and "on" state?

Thanks,
Tina

Hi Nikhil,

If there isn't an instance in that state, is there an instance I use for testing?

Thanks,
Tina

Hi Nikhil,

Could you check out the environment?

I'm getting the fog error below:

Thanks,
Tina

[----] I, [2016-05-02T17:30:28.606783 #20655:130b988]  INFO -- : MIQ(MiqQueue#deliver) Message id: [99000002328516], Delivering...
[----] I, [2016-05-02T17:30:28.618542 #20655:130b988]  INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) Refreshing all targets...
[----] I, [2016-05-02T17:30:28.618755 #20655:130b988]  INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) EMS: [OSP7], id: [99000000000018] Refreshing targets for EMS: [OSP7], id: [99000000000018]...
[----] I, [2016-05-02T17:30:28.618869 #20655:130b988]  INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager::Refresher#refresh) EMS: [OSP7], id: [99000000000018]   ManageIQ::Providers::Openstack::CloudManager [OSP7] id [99000000000018]
[----] E, [2016-05-02T17:30:29.543178 #20655:130b988] ERROR -- : <Fog> excon.error     #<Excon::Errors::SocketError: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol (OpenSSL::SSL::SSLError)>

[----] W, [2016-05-02T17:30:30.313597 #20652:1151994]  WARN -- : MIQ(ManageIQ::Providers::Redhat::InfraManager::RefreshParser.host_inv_to_ip) IP lookup for host in VIM inventory data...Failed. Falling back to reverse lookup.
[----] I, [2016-05-02T17:30:30.991523 #63427:e15988]  INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 2.

Hi Nikhil,

After further investigation, it appears there are multiple reasons the retired VM is not getting powered off.

1. I didn't see events coming from the Openstack provider. There is an Openstack configuration change that will resolve this issue.

2. Since the VM is suspended instead of powered off, our built in policies would not stop this VM.  
We need to map the incoming event:
compute.instance.resume.end --> add builtin policy to suspend the VM.
compute.instance.unpause.end --> add builtin policy to pause the VM.
 
Assigning the ticket to Lucy for event and policy changes.

Regards,
Tina

Thanks Tina!

https://github.com/ManageIQ/manageiq/pull/8872

New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/4a847416e666cb53dd5a1f828f39aa032f5bfd59

commit 4a847416e666cb53dd5a1f828f39aa032f5bfd59
Author:     Lucy Fu <lufu>
AuthorDate: Fri May 20 17:07:59 2016 -0400
Commit:     Lucy Fu <lufu>
CommitDate: Mon Jun 13 16:14:07 2016 -0400

    Add built in policy for event vm_resume.
    
    Retired instances may be left in suspended state on Openstack server.
    The build in policy will keep the retired instance in suspended state if it is powered on from the Openstack server.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1325167

 product/policy/built_in_policies.yml | 14 ++++++++++++++
 spec/models/miq_policy_spec.rb       | 30 ++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)

https://github.com/ManageIQ/manageiq/pull/9133

New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=a710139802b28b5387b00d088ce63022b938ed9a

commit a710139802b28b5387b00d088ce63022b938ed9a
Author:     Lucy Fu <lufu>
AuthorDate: Fri May 20 17:07:59 2016 -0400
Commit:     Lucy Fu <lufu>
CommitDate: Tue Jun 14 16:46:42 2016 -0400

    Add built in policy for event vm_resume.
    
    Retired instances may be left in suspended state on Openstack server.
    The build in policy will keep the retired instance in suspended state if it is powered on from the Openstack server.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1325167

 app/models/miq_policy.rb       | 10 ++++++++++
 spec/models/miq_policy_spec.rb | 30 ++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)

https://github.com/ManageIQ/manageiq/pull/7620