Bug 1310067

Summary: The AWS instance don't become archived after termination
Product: Red Hat CloudForms Management Engine Reporter: Taras Lehinevych <tlehinev>
Component: ProvidersAssignee: James Wong <jwong>
Status: CLOSED ERRATA QA Contact: Taras Lehinevych <tlehinev>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.5.0CC: cpelland, dajohnso, jfrey, jhardy, mhild, nachandr, obarenbo, simaishi, tlehinev
Target Milestone: GA   
Target Release: 5.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: ec2:vm:power
Fixed In Version: 5.6.0.0 Doc Type: Bug Fix
Doc Text:
Cause: ManageIQ::Providers::Amazon::CloudManager::Refresher is reading VMDB::Config.new("vmdb").config[:ems_refresh][:amazon] which doesn't exist. The correct trailing hash key should be [:ec2]. Consequence: Without access to a flag in the said config hash, ManageIQ::Providers::Amazon::CloudManager::RefreshParser.parse_instance won't disconnect a terminated instance (which is the "archived" state). Fix: Make ManageIQ::Providers::Amazon::CloudManager::Refresher to access the config hash with correct key (its ems_type() method) Result: now a terminated aws instance would be flagged correctly as archived state.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-29 15:38:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1322015, 1322885    
Bug Blocks:    
Attachments:
Description Flags
instance termination none

Description Taras Lehinevych 2016-02-19 11:28:07 UTC
Description of problem:
When you terminate vm, the vm status doesn't change to Archived

Version-Release number of selected component (if applicable):
5.5.2.4

How reproducible:
Always

Steps to Reproduce:
1. Open AWS instance
2. Terminate it
3. Wait for changing the power status to terminated

Actual results:
The vm doesn't become archived 

Expected results:
The vm should have archived status

Additional info:

Comment 4 James Wong 2016-02-25 20:35:14 UTC
Hi Tara, 

A couple clarification questions,


1. how did you terminate the VM? through AWS management console or ManageIQ console?  

2. And after it's terminated, is it still there in "instances"? Or it's completely gone from the ManageIQ console?

Comment 5 Taras Lehinevych 2016-02-26 11:57:19 UTC
Hi James, 

Sorry for not full description of the issue. 

1. I terminate VM through ManageIQ.
Instances -> Choose Instance -> Power -> Terminate. 

2. Yes, VM is still there in "instances". I attached screen shot.

Comment 6 Taras Lehinevych 2016-02-26 11:57:49 UTC
Created attachment 1130825 [details]
instance termination

Comment 9 Marcel Hild 2016-05-10 15:47:46 UTC
@james: have a look here 

https://github.com/ManageIQ/manageiq/pull/7477#discussion_r57366105

my guess is, that there is a kind of race condition going on.

The refresh calls #disconnect_env and removes the instance from the ems. But then automate cannot archive/retire the instance. But I'm not sure if this is the real problem. AFAIR the above PR retired a terminated instance...

Comment 10 James Wong 2016-05-10 20:23:31 UTC
The cause is that there's VMDB::Config.new("vmdb").config[:ems_refresh][:ec2]. But the refresh_parser is trying to get VMDB::Config.new("vmdb").config[:ems_refresh][:amazon] which doesn't exist. 

Without that options (and its ignore_terminated_instances=true to be exact),  ManageIQ::Providers::Amazon::CloudManager::RefreshParser.parse_instance won't be able to move a terminated instance into the flow of disconnect_inv.

Comment 11 James Wong 2016-05-10 22:57:58 UTC
@Marcel/@GregB, seems like we need this PR https://github.com/ManageIQ/manageiq/pull/5868 to fix this issue.

Comment 13 Marcel Hild 2016-05-11 11:20:16 UTC
@greg I wonder if ignore_terminated_instances=true makes sense at all.
AWS removes terminated instances after a threshold and then we'd archive them on our side. While they are terminated on aws IMHO they should be terminated on our side too...

Comment 14 James Wong 2016-05-11 19:25:26 UTC
@Marcel, that's right, that PR (https://github.com/ManageIQ/manageiq/pull/5868) was already merged to upstream and is in Darga.  Now just realized this bug is targeted at 5.6 which is Darga

So this bug can be closed after QE verification with the coming Darga release.

PS Also, the ems_type call was triggered from https://github.com/ManageIQ/manageiq/blob/master/gems/manageiq-providers-amazon/app/models/manageiq/providers/amazon/cloud_manager/refresher.rb#L5 which won't get the Amazon CloudManager class's ems_type.

Comment 15 Marcel Hild 2016-05-12 08:22:43 UTC
@james ok. good to see this might be a non-issue :)

regarding your PS I'm confused:

in rails console:
ManageIQ::Providers::Amazon::CloudManager::Refresher.ems_type 
=> :ec2

maybe lets take that conversation to gitter

Comment 16 James Wong 2016-05-13 18:06:53 UTC
For clarification:

Original ems_type() is calling
  @ems_type ||= parent.short_token.underscore.to_sym
parent.short_token returns "Amazon" which fails us.

The fix is calling instead,
  @ems_type ||= parent.ems_type.to_sym
parent.ems_type is CloudManager.ems_tyep => "ec2"

Comment 17 Taras Lehinevych 2016-05-17 10:59:33 UTC
Verified in 5.6.0.6-beta2.5

Comment 19 errata-xmlrpc 2016-06-29 15:38:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348