Bug 1351262 - Takes too long for Amazon VM instances to service retire
Summary: Takes too long for Amazon VM instances to service retire
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate
Version: 5.4.0
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: GA
: 5.7.0
Assignee: William Fitzgerald
QA Contact: Shveta
URL:
Whiteboard: ec2:service:retire
Depends On:
Blocks: 1372771
TreeView+ depends on / blocked
 
Reported: 2016-06-29 14:39 UTC by Tsai Li Ming
Modified: 2017-01-04 12:56 UTC (History)
8 users (show)

Fixed In Version: 5.7.0.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1372771 (view as bug list)
Environment:
Last Closed: 2017-01-04 12:56:37 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0012 0 normal SHIPPED_LIVE CFME 5.7.0 bug fixes and enhancement update 2017-01-04 17:50:36 UTC

Description Tsai Li Ming 2016-06-29 14:39:32 UTC
Description of problem:
1. An Amazon instance is provisioned using a service catalogue.
2. Service catalogue is then retired using Lifecycle -> Retire this service

The instances goes to the "powering down" state and gets stuck for a long time. It doesn't go to the "power off" state unless I force a provider refresh from the UI (I believe VM refresh state will be the same). 

I had to wait till the provider got refreshed before the state will change, which will then continue the retirement process. This took about 30mins. 

Compared to RHOS, RHOS is significantly faster in the service retirement process.

Version-Release number of selected component (if applicable):
Latest CF 3.2 (5.4.4)

How reproducible:
Always

Steps to Reproduce:
1. Provision an Amazon instance using a service catalogue
2. Retire the service under My Services
3. Observe the slowness and state of the VM

Actual results:
VM doesn't retire / gets stuck in the retirement process

Expected results:
VM should retire as soon as possible.

Additional info:

evm logs:
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:10:12.997874 #14980:4bfc1e8]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:12:24.423405 #27746:5a338b0]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:13:30.178788 #27746:5df541c]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:14:37.548781 #14980:4f59c50]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:16:49.380637 #14980:52a2970]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
[----] I, [2016-06-29T22:19:01.864513 #14980:53ca6cc]  INFO -- : <AutomationEngine> <AEMethod amazon_check_pre_retirement> Instanc
e:<test_retirement_aws_009002> on EMS:<Amazon has Power State:<powering_down>
.... many more ...

Comment 2 Tina Fitzgerald 2016-06-29 14:53:57 UTC
I think this is an issue with VM retirement. 
Can you retire an Amazon VM using VM retirement to see if you get the same results?

Thanks,
Tina

Comment 3 Tsai Li Ming 2016-06-29 15:13:02 UTC
Same issue if I use VM lifecycle retirement. Has been > 10 mins.

Comment 4 William Fitzgerald 2016-06-29 19:50:34 UTC
Can I have access to your appliance?

Thanks

Bill

Comment 6 William Fitzgerald 2016-06-30 13:35:32 UTC
I didn't know this was happening at a customer site.  Can I get a copy of the automation.log and evm.log when this problem occurred.  If this is a customer site, then don't try to get me accees, just the logs.

Thanks

Bill

Comment 11 William Fitzgerald 2016-07-19 18:15:02 UTC
Changed retirement state machine to do a vm refresh for Amazon.

This issue has been resolved in the following PR:

https://github.com/ManageIQ/manageiq/pull/9706

Comment 14 Shveta 2016-09-20 19:07:20 UTC
Verified in 5.7.0.0.20160906172503_12f65cb

Comment 16 errata-xmlrpc 2017-01-04 12:56:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0012.html


Note You need to log in before you can comment on or make changes to this bug.