Bug 2012233
| Summary: | [IBMCLOUD] IPI: "Exceeded limit of remote rules per security group (the limit is 5 remote rules per security group)" | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Christopher J Schaefer <cschaefe> |
| Component: | Installer | Assignee: | aos-install |
| Installer sub component: | openshift-installer | QA Contact: | Pedro Amoedo <pamoedom> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | unspecified | CC: | mstaeble, pamoedom, scuppett |
| Version: | 4.10 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.10.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-03-10 16:18:05 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 2009791 | ||
[QA Summary]
[Version]
~~~
$ ./openshift-install-local version
./openshift-install-local unreleased-master-5104-g527c46172e6c4fe28838e1a18e0d758eec61fb02
built from commit 527c46172e6c4fe28838e1a18e0d758eec61fb02
release image registry.ci.openshift.org/origin/release:4.8
release architecture amd64
$ git --no-pager log --oneline --first-parent origin/master -3
527c46172 (HEAD -> master, upstream/master, origin/master, origin/HEAD) Merge pull request #5289 from cjschaef/bz_2012233
41523104b Merge pull request #5222 from santos1709/ovirt_upi_devel
681fef731 Merge pull request #5286 from rna-afk/aws_remove_tags_from_user_iam
~~~
[Environment]
~~~
apiVersion: v1
baseDomain: ibmcloud.qe.devcluster.openshift.com
compute:
- architecture: amd64
hyperthreading: Enabled
name: worker
platform:
ibmcloud:
type: bx2-4x16
replicas: 2
controlPlane:
architecture: amd64
hyperthreading: Enabled
name: master
platform:
ibmcloud:
type: bx2-8x32
replicas: 3
metadata:
creationTimestamp: null
name: pamoedo-test
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 10.0.0.0/16
networkType: OpenShiftSDN
serviceNetwork:
- 172.30.0.0/16
platform:
ibmcloud:
region: eu-de
resourceGroupName: pamoedom-rg
publish: External
pullSecret:
sshKey:
~~~
[Results]
~~~
$ DIGEST=$(skopeo inspect --authfile pull-secret docker://registry.ci.openshift.org/ocp/release:4.10.0-0.nightly-2021-10-13-081040 | grep Digest | cut -d '"' -f4)
$ export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=registry.ci.openshift.org/ocp/release@${DIGEST}
$ export IC_API_KEY='<obfuscated>'
$ ./openshift-install-local create cluster --dir test37/ --log-level debug
DEBUG OpenShift Installer unreleased-master-5104-g527c46172e6c4fe28838e1a18e0d758eec61fb02
DEBUG Built from commit 527c46172e6c4fe28838e1a18e0d758eec61fb02
...
DEBUG Apply complete! Resources: 60 added, 0 changed, 0 destroyed.
DEBUG
DEBUG Outputs:
DEBUG
DEBUG control_plane_security_group_id_list = [
DEBUG "r010-6a6c0109-3b56-413f-8769-f2e09314654c",
DEBUG "r010-8624c529-7c01-4563-ac6e-7bd7d86fc470",
DEBUG "r010-b193d9de-0d15-4fa6-bb09-16c4f3093e20",
DEBUG "r010-3a5c82e4-5102-4b68-ba04-a46ce6799171",
DEBUG ]
DEBUG control_plane_subnet_id_list = [
DEBUG "02b7-9db6e4f0-d0d1-4fa9-b9f5-b26afb517e84",
DEBUG "02c7-519c5479-1142-47d8-b1bf-71848785c0d6",
DEBUG "02d7-1cab091f-888f-45c6-826d-20513c4eb14d",
DEBUG ]
DEBUG control_plane_subnet_zone_list = [
DEBUG "eu-de-1",
DEBUG "eu-de-2",
DEBUG "eu-de-3",
DEBUG ]
...
~~~
*** PASSED ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056 |
Version: 4.10.0-0.nightly-2021-10-08-090421 Platform: ibmcloud Please specify: IPI What happened? Creating a new IBM Cloud using IPI fails due to a limitation of 5 rules per SecurityGroup on IBM Cloud. ERROR Error: Error while creating Security Group Rule Exceeded limit of remote rules per security group (the limit is 5 remote rules per security group). ERROR ERROR Adding a rule would exceed the limit of remote rules per security group. Consider creating another security group. ERROR { ERROR "StatusCode": 400, ERROR "Headers": { ERROR "Cache-Control": [ ERROR "max-age=0, no-cache, no-store, must-revalidate" ERROR ], ERROR "Cf-Cache-Status": [ ERROR "DYNAMIC" ERROR ], ERROR "Cf-Ray": [ ERROR "699ffed098146653-MAD" ERROR ], ERROR "Connection": [ ERROR "keep-alive" ERROR ], ERROR "Content-Length": [ ERROR "373" ERROR ], ERROR "Content-Type": [ ERROR "application/json" ERROR ], ERROR "Date": [ ERROR "Wed, 06 Oct 2021 15:41:42 GMT" ERROR ], ERROR "Expect-Ct": [ ERROR "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" ERROR ], ERROR "Expires": [ ERROR "-1" ERROR ], ERROR "Pragma": [ ERROR "no-cache" ERROR ], ERROR "Server": [ ERROR "cloudflare" ERROR ], ERROR "Strict-Transport-Security": [ ERROR "max-age=31536000; includeSubDomains" ERROR ], ERROR "Vary": [ ERROR "Accept-Encoding" ERROR ], ERROR "X-Content-Type-Options": [ ERROR "nosniff" ERROR ], ERROR "X-Request-Id": [ ERROR "2f23ff2f-9acc-447a-86e4-2730be710d37" ERROR ], ERROR "X-Xss-Protection": [ ERROR "1; mode=block" ERROR ] ERROR }, ERROR "Result": { ERROR "errors": [ ERROR { ERROR "code": "over_quota", ERROR "message": "Exceeded limit of remote rules per security group (the limit is 5 remote rules per security group).\n\nAdding a rule would exceed the limit of remote rules per security group. Consider creating another security group.", ERROR "more_info": "https://cloud.ibm.com/docs/vpc?topic=vpc-quotas" ERROR } ERROR ], ERROR "trace": "2f23ff2f-9acc-447a-86e4-2730be710d37" ERROR }, ERROR "RawResult": null ERROR } ERROR ERROR ERROR on ../../../../tmp/openshift-install-network-003889995/vpc/security-groups.tf line 268, in resource "ibm_is_security_group_rule" "control_plane_machine_config_lb_inbound": ERROR 268: resource "ibm_is_security_group_rule" "control_plane_machine_config_lb_inbound" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change What did you expect to happen? IPI installation would properly deploy a cluster on IBM Cloud How to reproduce it (as minimally and precisely as possible)? Using the latest installer builds, attempt to create a new cluster on IBM Cloud # openshift-install create cluster Anything else we need to know? The issue was caused by this change, putting one SG over the 5 rule limit. https://github.com/openshift/installer/pull/5105/files#diff-b237721806432aac5fdc1b5a78484516c931af5affbbcea351e8824b5f9fcd54 IBM Cloud is attempting to find a way to add the changes above in a way that meets the 5 rule limit per SG requirement.