Bug 1514270

Summary: Automate handlers are missing for some AWS events
Product: Red Hat CloudForms Management Engine Reporter: Satoe Imaishi <simaishi>
Component: ProvidersAssignee: Ladislav Smola <lsmola>
Status: CLOSED CURRENTRELEASE QA Contact: Matouš Mojžíš <mmojzis>
Severity: high Docs Contact:
Priority: high    
Version: 5.9.0CC: gblomqui, izapolsk, jfrey, jhardy, obarenbo
Target Milestone: GA   
Target Release: 5.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: ec2:refresh
Fixed In Version: 5.9.0.10 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1511621 Environment:
Last Closed: 2018-07-12 17:24:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: AWS Target Upstream Version:
Embargoed:
Bug Depends On: 1511621    
Bug Blocks:    

Comment 2 CFME Bot 2017-11-29 20:29:50 UTC
New commit detected on ManageIQ/manageiq-providers-amazon/gaprindashvili:
https://github.com/ManageIQ/manageiq-providers-amazon/commit/2b0d95fac8d56e7d9d50267c055306bcccec2a03

commit 2b0d95fac8d56e7d9d50267c055306bcccec2a03
Author:     Bronagh Sorota <bsorota>
AuthorDate: Thu Nov 16 10:44:43 2017 -0500
Commit:     Satoe Imaishi <simaishi>
CommitDate: Fri Nov 17 10:47:32 2017 -0500

    Merge pull request #366 from Ladas/enhance_list_of_events_supported
    
    Enhance list of events supported
    (cherry picked from commit 5de90a5bee870b5b7bea006529990ef4f316a049)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1514270

 .../amazon/cloud_manager/event_target_parser.rb    |   4 +
 .../AWS_API_CALL_AssociateDhcpOptions.json         |  11 +
 .../AWS_API_CALL_AssociateRouteTable.json          |  11 +
 .../AWS_API_CALL_AttachInternetGateway.json        |  11 +
 .../cloud_watch/AWS_API_CALL_CopySnapshot.json     |  11 +
 .../AWS_API_CALL_CreateInternetGateway.json        |  10 +-
 .../cloud_watch/AWS_API_CALL_CreateRoute.json      |  11 +
 .../cloud_watch/AWS_API_CALL_CreateRouteTable.json |  11 +
 .../cloud_watch/AWS_API_CALL_CreateVpc.json        |  10 +-
 .../AWS_API_CALL_DeleteInternetGateway.json        |  11 +
 .../AWS_API_CALL_DeleteNetworkInterface.json       |  11 +
 .../cloud_watch/AWS_API_CALL_DeleteRoute.json      |  11 +
 .../cloud_watch/AWS_API_CALL_DeleteRouteTable.json |  11 +
 .../cloud_watch/AWS_API_CALL_DeleteVpc.json        |  11 +
 .../cloud_watch/AWS_API_CALL_DeregisterImage.json  |  11 +
 .../AWS_API_CALL_DetachInternetGateway.json        |  11 +
 .../AWS_API_CALL_DisassociateRouteTable.json       |  11 +
 .../AWS_API_CALL_EnableVpcClassicLink.json         |  11 +
 ...WS_API_CALL_EnableVpcClassicLinkDnsSupport.json |  11 +
 .../cloud_watch/AWS_API_CALL_ImportKeyPair.json    |  11 +
 .../AWS_API_CALL_ModifyVpcAttribute.json           |  11 +
 .../AWS_API_CALL_ReplaceRouteTableAssociation.json |  11 +
 .../AWS_API_CALL_RevokeSecurityGroupEgress.json    |  11 +
 .../AWS_API_CALL_RevokeSecurityGroupIngress.json   |  11 +
 .../config/AWS_EC2_InternetGateway_CREATE.json     |  12 +
 .../config/AWS_EC2_InternetGateway_DELETE.json     |  12 +
 .../config/AWS_EC2_NetworkAcl_CREATE.json          |  12 +
 .../config/AWS_EC2_NetworkAcl_UPDATE.json          |  12 +
 .../config/AWS_EC2_RouteTable_CREATE.json          |  12 +
 .../config/AWS_EC2_RouteTable_DELETE.json          |  12 +
 .../config/AWS_EC2_RouteTable_UPDATE.json          |  12 +
 .../config/AWS_EC2_Subnet_CREATE.json              |  12 +
 .../event_catcher/config/AWS_EC2_VPC_CREATE.json   |  12 +
 .../event_catcher/config/AWS_EC2_VPC_DELETE.json   |  12 +
 .../cloud_manager/event_target_parser_spec.rb      | 378 ++++++++++++++++++++-
 35 files changed, 737 insertions(+), 16 deletions(-)
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_AssociateDhcpOptions.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_AssociateRouteTable.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_AttachInternetGateway.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_CopySnapshot.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_CreateRoute.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_CreateRouteTable.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeleteInternetGateway.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeleteNetworkInterface.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeleteRoute.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeleteRouteTable.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeleteVpc.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DeregisterImage.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DetachInternetGateway.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_DisassociateRouteTable.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_EnableVpcClassicLink.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_EnableVpcClassicLinkDnsSupport.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_ImportKeyPair.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_ModifyVpcAttribute.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_ReplaceRouteTableAssociation.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_RevokeSecurityGroupEgress.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/cloud_watch/AWS_API_CALL_RevokeSecurityGroupIngress.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_InternetGateway_CREATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_InternetGateway_DELETE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_NetworkAcl_CREATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_NetworkAcl_UPDATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_RouteTable_CREATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_RouteTable_DELETE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_RouteTable_UPDATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_Subnet_CREATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_VPC_CREATE.json
 create mode 100644 spec/models/manageiq/providers/amazon/cloud_manager/event_catcher/config/AWS_EC2_VPC_DELETE.json

Comment 3 CFME Bot 2017-11-29 20:32:22 UTC
New commit detected on ManageIQ/manageiq-content/gaprindashvili:
https://github.com/ManageIQ/manageiq-content/commit/101f05f55f5831b0b66de24fe02d364b730df587

commit 101f05f55f5831b0b66de24fe02d364b730df587
Author:     Greg McCullough <gmccullo>
AuthorDate: Thu Nov 16 10:51:31 2017 -0500
Commit:     Satoe Imaishi <simaishi>
CommitDate: Fri Nov 17 10:32:55 2017 -0500

    Merge pull request #217 from Ladas/add_more_aws_event_handlers
    
    Add more aws event handlers
    (cherry picked from commit bba2391ced0fe83cee347ac2b879446852ffb9a4)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1514270

 .../Amazon.class/aws_api_call_associatedhcpoptions.yaml      | 12 ++++++++++++
 .../Amazon.class/aws_api_call_associateroutetable.yaml       | 12 ++++++++++++
 .../Amazon.class/aws_api_call_attachinternetgateway.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_createkeypair.yaml    | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_createroute.yaml      | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_createroutetable.yaml | 12 ++++++++++++
 .../Amazon.class/aws_api_call_deleteinternetgateway.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_deletekeypair.yaml    | 12 ++++++++++++
 .../Amazon.class/aws_api_call_deletenetworkinterface.yaml    | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_deleteroute.yaml      | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_deleteroutetable.yaml | 12 ++++++++++++
 .../Event/EmsEvent/Amazon.class/aws_api_call_deletevpc.yaml  | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_deregisterimage.yaml  | 12 ++++++++++++
 .../Amazon.class/aws_api_call_detachinternetgateway.yaml     | 12 ++++++++++++
 .../Amazon.class/aws_api_call_disassociateroutetable.yaml    | 12 ++++++++++++
 .../aws_api_call_enablevpcclassicdnssupport.yaml             | 12 ++++++++++++
 .../Amazon.class/aws_api_call_enablevpcclassiclink.yaml      | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_api_call_importkeypair.yaml    | 12 ++++++++++++
 .../Amazon.class/aws_api_call_modifyvpcattribute.yaml        | 12 ++++++++++++
 .../aws_api_call_replaceroutetableassociation.yaml           | 12 ++++++++++++
 .../Amazon.class/aws_api_call_revokesecuritygroupegress.yaml | 12 ++++++++++++
 .../aws_api_call_revokesecuritygroupingress.yaml             | 12 ++++++++++++
 .../Amazon.class/aws_ec2_internetgateway_create.yaml         | 12 ++++++++++++
 .../Amazon.class/aws_ec2_internetgateway_delete.yaml         | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_ec2_networkacl_create.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_ec2_networkacl_update.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_ec2_routetable_create.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_ec2_routetable_delete.yaml     | 12 ++++++++++++
 .../EmsEvent/Amazon.class/aws_ec2_routetable_update.yaml     | 12 ++++++++++++
 .../Event/EmsEvent/Amazon.class/aws_ec2_subnet_create.yaml   | 12 ++++++++++++
 .../Event/EmsEvent/Amazon.class/aws_ec2_vpc_create.yaml      | 12 ++++++++++++
 .../Event/EmsEvent/Amazon.class/aws_ec2_vpc_delete.yaml      | 12 ++++++++++++
 32 files changed, 384 insertions(+)
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_associatedhcpoptions.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_associateroutetable.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_attachinternetgateway.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_createkeypair.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_createroute.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_createroutetable.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deleteinternetgateway.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deletekeypair.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deletenetworkinterface.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deleteroute.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deleteroutetable.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deletevpc.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_deregisterimage.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_detachinternetgateway.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_disassociateroutetable.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_enablevpcclassicdnssupport.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_enablevpcclassiclink.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_importkeypair.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_modifyvpcattribute.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_replaceroutetableassociation.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_revokesecuritygroupegress.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_api_call_revokesecuritygroupingress.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_internetgateway_create.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_internetgateway_delete.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_networkacl_create.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_networkacl_update.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_routetable_create.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_routetable_delete.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_routetable_update.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_subnet_create.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_vpc_create.yaml
 create mode 100644 content/automate/ManageIQ/System/Event/EmsEvent/Amazon.class/aws_ec2_vpc_delete.yaml

Comment 4 Matouš Mojžíš 2018-03-01 19:27:24 UTC
AWS Networks & Network Routers are working correctly.
Image Delete event is not working correctly because AWS sends delete event sooner than image is deleted.
Key Pairs are still investigated, but there is probably issue on AWS side that KeyPairs events are not sent to SQS so we can't catch them.
Also I am testing all events from this file:
https://github.com/Ladas/manageiq-providers-amazon/blob/a9a281f710075e348354e853614b30d8faed0460/spec/models/manageiq/providers/amazon/cloud_manager/event_target_parser_spec.rb#L537
But didn't have enough time to go through all these events.

Comment 5 Matouš Mojžíš 2018-03-06 18:25:15 UTC
Verified in 5.9.0.22. The issue with key pairs was actually in our environment and with some modification it works correctly.