Bug 1903277 - Deprovisioning Not Deleting Security Groups [VpcLimitExceeded on e2e-aws tests]
Summary: Deprovisioning Not Deleting Security Groups [VpcLimitExceeded on e2e-aws tests]
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.7.0
Assignee: Matthew Staebler
QA Contact: Yunfei Jiang
Depends On:
TreeView+ depends on / blocked
Reported: 2020-12-01 18:29 UTC by Steve Kuznetsov
Modified: 2021-02-24 15:37 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Rules on the default security group that source another security group prevent the destroyer from deleting that other security group. Consequence: Cluster destroy will never complete and AWS resources will remain. Fix: Delete rules from the default security group, unblocking the delete of the other security groups. Result: Cluster destroy completes and all AWS resources for the cluster are deleted.
Clone Of:
Last Closed: 2021-02-24 15:37:08 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4440 0 None closed Bug 1903277: destroy/aws: remove rules from default security group 2021-02-17 20:31:23 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:37:38 UTC

Description Steve Kuznetsov 2020-12-01 18:29:06 UTC
See details here https://issues.redhat.com/browse/DPTP-1770

This is causing infrastructure outages.

Comment 1 Matthew Staebler 2020-12-02 15:15:31 UTC
(for help with build cop associated this bug with outages)
e2e-aws broken by aws limits

Comment 3 Yunfei Jiang 2020-12-04 09:55:28 UTC
verified. PASS.
version: 4.7.0-0.nightly-2020-12-04-013308

Verify steps:
1. install an IPI cluster on AWS
2. add ingress and egress rules in default security group.
3. destroy cluster.

>> before (4.6.0 GA):
12-04 17:31:53  level=info msg=Deleted arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41 table=rtb-08f9ed676bc701dda
12-04 17:31:53  level=debug msg="Skipping default security group" arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41 security group=sg-08474120bbc3035a0
12-04 17:31:54  level=info msg=Deleted arn="arn:aws:ec2:us-east-2:301721915996:vpc/vpc-0272486bbd6e01b41" id=vpc-0272486bbd6e01b41
12-04 17:32:20  level=debug msg="Purging asset \"Cluster\" from disk"
12-04 17:32:20  level=info msg="Time elapsed: 6m59s"

>> after (4.7.0-0.nightly-2020-12-04-013308):
12-04 17:33:27  level=info msg=Deleted id=vpc-02984f978709042b6 table=rtb-0446dc211b43503f6
12-04 17:33:27  level=debug msg=Revoked ingress permissions id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
12-04 17:33:27  level=debug msg=Revoked egress permissions id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
12-04 17:33:27  level=debug msg=Skipping default security group id=vpc-02984f978709042b6 security group=sg-04b59f8c1f71edaec
12-04 17:33:28  level=info msg=Deleted id=vpc-02984f978709042b6
12-04 17:33:50  level=debug msg=Purging asset "Cluster" from disk
12-04 17:33:50  level=info msg=Time elapsed: 7m42s

Comment 6 errata-xmlrpc 2021-02-24 15:37:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.