Bug 1766691

Summary: Timeout for aws_route53_record
Product: OpenShift Container Platform Reporter: Oleg Bulatov <obulatov>
Component: InstallerAssignee: Abhinav Dahiya <adahiya>
Installer sub component: openshift-installer QA Contact: Johnny Liu <jialiu>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: low    
Version: 4.3.0   
Target Milestone: ---   
Target Release: 4.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Rate limiting from AWS would cause failure to create records for the cluster Consequence: Rate limiting from AWS would cause failure to create records for the cluster Fix: Use exponential back-off to allow for longer wait timeout. Result: Less failures due to Rate limiting
 Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-13 17:12:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Oleg Bulatov 2019-10-29 16:46:58 UTC 
https://ci-search-ci-search-next.svc.ci.openshift.org/?search=timeout+while+waiting+for+state+to+become+%27accepted%27&maxAge=336h&context=2&type=all

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_cluster-openshift-controller-manager-operator/129/pull-ci-openshift-cluster-openshift-controller-manager-operator-master-e2e-aws/536
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-serial-4.3/1324

level=error msg="Error: [ERR]: Error building changeset: timeout while waiting for state to become 'accepted' (timeout: 5m0s)"
level=error
level=error msg="  on ../tmp/openshift-install-771650541/route53/base.tf line 33, in resource \"aws_route53_record\" \"api_external\":"
level=error msg="  33: resource \"aws_route53_record\" \"api_external\" {"
level=error
level=error
level=fatal msg="failed to fetch Cluster: failed to generate asset \"Cluster\": failed to create cluster: failed to apply using Terraform"
Comment 1 Scott Dodson 2020-01-31 18:43:29 UTC 
Proposed to bump the timeout.
Comment 3 Abhinav Dahiya 2020-02-04 23:03:44 UTC 
proposed addition of timeout to the resource upstream with the PR https://github.com/terraform-providers/terraform-provider-aws/pull/11895
Only after upsteam accepts the change, we can bump our vendor, and bumping aws provider is blocked on Go modules migration. So in inclined to moving it to 4.5. but let's wait and see.
Comment 6 Johnny Liu 2020-03-02 10:20:33 UTC 
From the output of https://search.svc.ci.openshift.org/?search=timeout+while+waiting+for+state+to+become+%27accepted%27&maxAge=336h&context=1&type=all, found 3 failed job due to the same issue.

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-aws-upgrade/19491
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-ocp-installer-e2e-aws-serial-4.2/1070
https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/24568/pull-ci-openshift-origin-master-e2e-aws-serial/12735

All these are not happened in 4.5, so move this bug to verified.
Comment 8 errata-xmlrpc 2020-07-13 17:12:05 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409