Bug 1310067 - The AWS instance don't become archived after termination
The AWS instance don't become archived after termination
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers (Show other bugs)
Unspecified Unspecified
medium Severity medium
: GA
: 5.6.0
Assigned To: James Wong
Taras Lehinevych
Depends On: 1322015 1322885
  Show dependency treegraph
Reported: 2016-02-19 06:28 EST by Taras Lehinevych
Modified: 2016-08-02 21:33 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: ManageIQ::Providers::Amazon::CloudManager::Refresher is reading VMDB::Config.new("vmdb").config[:ems_refresh][:amazon] which doesn't exist. The correct trailing hash key should be [:ec2]. Consequence: Without access to a flag in the said config hash, ManageIQ::Providers::Amazon::CloudManager::RefreshParser.parse_instance won't disconnect a terminated instance (which is the "archived" state). Fix: Make ManageIQ::Providers::Amazon::CloudManager::Refresher to access the config hash with correct key (its ems_type() method) Result: now a terminated aws instance would be flagged correctly as archived state.
Story Points: ---
Clone Of:
Last Closed: 2016-06-29 11:38:25 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
instance termination (71.72 KB, image/png)
2016-02-26 06:57 EST, Taras Lehinevych
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1348 normal SHIPPED_LIVE CFME 5.6.0 bug fixes and enhancement update 2016-06-29 14:50:04 EDT

  None (edit)
Description Taras Lehinevych 2016-02-19 06:28:07 EST
Description of problem:
When you terminate vm, the vm status doesn't change to Archived

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Open AWS instance
2. Terminate it
3. Wait for changing the power status to terminated

Actual results:
The vm doesn't become archived 

Expected results:
The vm should have archived status

Additional info:
Comment 4 James Wong 2016-02-25 15:35:14 EST
Hi Tara, 

A couple clarification questions,

1. how did you terminate the VM? through AWS management console or ManageIQ console?  

2. And after it's terminated, is it still there in "instances"? Or it's completely gone from the ManageIQ console?
Comment 5 Taras Lehinevych 2016-02-26 06:57:19 EST
Hi James, 

Sorry for not full description of the issue. 

1. I terminate VM through ManageIQ.
Instances -> Choose Instance -> Power -> Terminate. 

2. Yes, VM is still there in "instances". I attached screen shot.
Comment 6 Taras Lehinevych 2016-02-26 06:57 EST
Created attachment 1130825 [details]
instance termination
Comment 9 Marcel Hild 2016-05-10 11:47:46 EDT
@james: have a look here 


my guess is, that there is a kind of race condition going on.

The refresh calls #disconnect_env and removes the instance from the ems. But then automate cannot archive/retire the instance. But I'm not sure if this is the real problem. AFAIR the above PR retired a terminated instance...
Comment 10 James Wong 2016-05-10 16:23:31 EDT
The cause is that there's VMDB::Config.new("vmdb").config[:ems_refresh][:ec2]. But the refresh_parser is trying to get VMDB::Config.new("vmdb").config[:ems_refresh][:amazon] which doesn't exist. 

Without that options (and its ignore_terminated_instances=true to be exact),  ManageIQ::Providers::Amazon::CloudManager::RefreshParser.parse_instance won't be able to move a terminated instance into the flow of disconnect_inv.
Comment 11 James Wong 2016-05-10 18:57:58 EDT
@Marcel/@GregB, seems like we need this PR https://github.com/ManageIQ/manageiq/pull/5868 to fix this issue.
Comment 13 Marcel Hild 2016-05-11 07:20:16 EDT
@greg I wonder if ignore_terminated_instances=true makes sense at all.
AWS removes terminated instances after a threshold and then we'd archive them on our side. While they are terminated on aws IMHO they should be terminated on our side too...
Comment 14 James Wong 2016-05-11 15:25:26 EDT
@Marcel, that's right, that PR (https://github.com/ManageIQ/manageiq/pull/5868) was already merged to upstream and is in Darga.  Now just realized this bug is targeted at 5.6 which is Darga

So this bug can be closed after QE verification with the coming Darga release.

PS Also, the ems_type call was triggered from https://github.com/ManageIQ/manageiq/blob/master/gems/manageiq-providers-amazon/app/models/manageiq/providers/amazon/cloud_manager/refresher.rb#L5 which won't get the Amazon CloudManager class's ems_type.
Comment 15 Marcel Hild 2016-05-12 04:22:43 EDT
@james ok. good to see this might be a non-issue :)

regarding your PS I'm confused:

in rails console:
=> :ec2

maybe lets take that conversation to gitter
Comment 16 James Wong 2016-05-13 14:06:53 EDT
For clarification:

Original ems_type() is calling
  @ems_type ||= parent.short_token.underscore.to_sym
parent.short_token returns "Amazon" which fails us.

The fix is calling instead,
  @ems_type ||= parent.ems_type.to_sym
parent.ems_type is CloudManager.ems_tyep => "ec2"
Comment 17 Taras Lehinevych 2016-05-17 06:59:33 EDT
Verified in
Comment 19 errata-xmlrpc 2016-06-29 11:38:25 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.