Bug 1958154
| Summary: | Custom AWS user tags limit not supported (openshift/api says max=25), install fails when >=10 | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Andrew McDermott <amcdermo> |
| Component: | Installer | Assignee: | Aditya Narayanaswamy <anarayan> |
| Installer sub component: | openshift-installer | QA Contact: | Yunfei Jiang <yunjiang> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | low | ||
| Priority: | low | CC: | anarayan, gshereme, mstaeble |
| Version: | 4.8 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.9.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
AWS S3 bucket allows only a maximum of 10 custom tags and hence we should restrict the number of custom user tags to 8 since we use 2 of our own to tag the resources and to figure out what resources to delete during cluster destruction.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-10-18 17:31:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
That's a good find. Since s3 only supports 10 tags, we need to drastically lower the number of user tags that OpenShift claims to support. Need to address, not prioritized. verified. PASS.
OCP version: 4.9.0-0.nightly-2021-08-25-185404
install-config.yaml:
platform:
aws:
region: us-east-2
userTags:
customTag-1: value-1
customTag-2: value-2
customTag-3: value-3
customTag-4: value-4
customTag-5: value-5
customTag-6: value-6
customTag-7: value-7
customTag-8: value-8
customTag-9: value-9
./openshift-install create ignition-configs --dir cluster1
FATAL failed to fetch Kubeconfig Admin Client: failed to load asset "Install Config": invalid "install-config.yaml" file: platform.aws.userTags: Invalid value: 9: number of user tags cannot be more than 8
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759 |
I was doing a demo of the experimental AWS user tags feature and had the following custom tags defined. platform: aws: experimentalPropagateUserTags: true userTags: managedBy: redhat.com administrator: root customTag-1: customValue-2021-05-07-100959 customTag-2: customValue-2021-05-07-101004 customTag-3: customValue-2021-05-07-101009 customTag-4: customValue-2021-05-07-101014 customTag-5: customValue-2021-05-07-101019 customTag-6: customValue-2021-05-07-101024 customTag-7: customValue-2021-05-07-101029 customTag-8: customValue-2021-05-07-101034 Creating the cluster I get the following error: $ ./openshift-install create cluster --dir=/Users/aim/r/clusters/4.8.0-0.ci-2021-05-03-215023/aws-amcdermo-2021-05-07-1035 INFO Credentials loaded from the "default" profile in file "/Users/aim/.aws/credentials" INFO Consuming Install Config from target directory INFO Creating infrastructure resources... ERROR ERROR Error: Error putting object in S3 bucket (amcdermo-2021-05-07-1-lmbxp-bootstrap): BadRequest: Object tags cannot be greater than 10 ERROR status code: 400, request id: 4NC1X50HD9A552TR, host id: qWGyIRj9oW5n337vZKf1l6NKzKxj6O5XcmtFRRbeJVTl1+2DdCJtS+10Av4BNREd/hEgUznEC94= ERROR ERROR on ../../../../../private/var/folders/f7/r2n_jh4s3c39sk4m1c9r0z380000z8/T/openshift-install-869013030/bootstrap/main.tf line 26, in resource "aws_s3_bucket_object" "ignition": ERROR 26: resource "aws_s3_bucket_object" "ignition" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change My install version was: 4.8.0-0.ci-2021-05-03-215023 We say that we can support 25 additional tags: https://github.com/openshift/api/blob/master/config/v1/0000_10_config-operator_01_infrastructure.crd.yaml#L252 resourceTags: description: resourceTags is a list of additional tags to apply to AWS resources created for the cluster. See https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html for information on tagging AWS resources. AWS supports a maximum of 50 tags per resource. OpenShift reserves 25 tags for its use, leaving 25 tags available for the user. type: array maxItems: 25 This AWS doc says a maximum of 10: https://docs.aws.amazon.com/AmazonS3/latest/userguide/tagging-managing.html "You can associate up to 10 tags with an object. Tags associated with an object must have unique tag keys." And this AWS doc says 50: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions "Maximum number of tags per resource – 50" I tried another install with: platform: aws: region: ${AWS_REGION} experimentalPropagateUserTags: true userTags: managedBy: redhat.com administrator: root customTag-1: customValue-2021-05-07-100959 customTag-2: customValue-2021-05-07-101004 customTag-3: customValue-2021-05-07-101009 customTag-4: customValue-2021-05-07-101014 customTag-5: customValue-2021-05-07-101019 customTag-6: customValue-2021-05-07-101024 customTag-7: customValue-2021-05-07-101029 And got: $ ./openshift-install create cluster --dir=/Users/aim/r/clusters/4.8.0-0.ci-2021-05-03-215023/aws-amcdermo-2021-05-07-1044 INFO Credentials loaded from the "default" profile in file "/Users/aim/.aws/credentials" INFO Consuming Install Config from target directory INFO Creating infrastructure resources... ERROR ERROR Error: Error putting object in S3 bucket (amcdermo-2021-05-07-1-kwckf-bootstrap): BadRequest: Object tags cannot be greater than 10 ERROR status code: 400, request id: WBARWX5WYZBDV5S2, host id: S4Ljlxgc39zxEL9EjdQzGPoAdpkSh1aCmO5pv+6tNDh0gpMcYzGVLx//pasSijRWX4ST9N2iEp4= ERROR ERROR on ../../../../../private/var/folders/f7/r2n_jh4s3c39sk4m1c9r0z380000z8/T/openshift-install-774161306/bootstrap/main.tf line 26, in resource "aws_s3_bucket_object" "ignition": ERROR 26: resource "aws_s3_bucket_object" "ignition" { ERROR ERROR FATAL failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: failed to complete the change ^^ so that one is weird given that there are not 10 tags listed. But as these are "additional" tags perhaps the installer adds other tags that I'm not aware of which would make this >= 10. I tried again with: platform: aws: region: ${AWS_REGION} experimentalPropagateUserTags: true userTags: managedBy: redhat.com administrator: root customTag-1: customValue-2021-05-07-100959 customTag-2: customValue-2021-05-07-101004 which was successful. $ ./openshift-install create cluster --dir=/Users/aim/r/clusters/4.8.0-0.ci-2021-05-03-215023/aws-amcdermo-2021-05-07-1121 INFO Credentials loaded from the "default" profile in file "/Users/aim/.aws/credentials" INFO Consuming Install Config from target directory INFO Creating infrastructure resources... INFO Waiting up to 20m0s for the Kubernetes API at https://api.amcdermo-2021-05-07-1121.devcluster.openshift.com:6443...