Bug 1903277

Summary: Deprovisioning Not Deleting Security Groups [VpcLimitExceeded on e2e-aws tests]
Product: OpenShift Container Platform Reporter: Steve Kuznetsov <skuznets>
Component: InstallerAssignee: Matthew Staebler <mstaeble>
Installer sub component: openshift-installer QA Contact: Yunfei Jiang <yunjiang>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: unspecified CC: afield, mstaeble, wking
Version: 4.5   
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Rules on the default security group that source another security group prevent the destroyer from deleting that other security group. Consequence: Cluster destroy will never complete and AWS resources will remain. Fix: Delete rules from the default security group, unblocking the delete of the other security groups. Result: Cluster destroy completes and all AWS resources for the cluster are deleted.
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-24 15:37:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Steve Kuznetsov 2020-12-01 18:29:06 UTC
See details here https://issues.redhat.com/browse/DPTP-1770

This is causing infrastructure outages.

Comment 1 Matthew Staebler 2020-12-02 15:15:31 UTC
(for help with build cop associated this bug with outages)
e2e-aws broken by aws limits

Comment 3 Yunfei Jiang 2020-12-04 09:55:28 UTC
verified. PASS.
version: 4.7.0-0.nightly-2020-12-04-013308

Verify steps:
1. install an IPI cluster on AWS
2. add ingress and egress rules in default security group.
3. destroy cluster.

>> before (4.6.0 GA):
12-04 17:31:53  level=info msg=Deleted arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41 table=rtb-08f9ed676bc701dda
12-04 17:31:53  level=debug msg="Skipping default security group" arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41 security group=sg-08474120bbc3035a0
<--snip-->
12-04 17:31:54  level=info msg=Deleted arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41
<--snip-->
12-04 17:32:20  level=debug msg="Purging asset \"Cluster\" from disk"
12-04 17:32:20  level=info msg="Time elapsed: 6m59s"

>> after (4.7.0-0.nightly-2020-12-04-013308):
12-04 17:33:27  level=info msg=Deleted id=vpc-02984f978709042b6 table=rtb-0446dc211b43503f6
12-04 17:33:27  level=debug msg=Revoked ingress permissions id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
12-04 17:33:27  level=debug msg=Revoked egress permissions id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
12-04 17:33:27  level=debug msg=Skipping default security group id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
<--snip-->
12-04 17:33:28  level=info msg=Deleted id=vpc-02984f978709042b6
<--snip-->
12-04 17:33:50  level=debug msg=Purging asset "Cluster" from disk
12-04 17:33:50  level=info msg=Time elapsed: 7m42s

Comment 6 errata-xmlrpc 2021-02-24 15:37:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633