Bug 1977184
| Summary: | Image registry Degraded caused by requesting to aws sts global endpoint timeout when installing sts cluster in a disconnected network | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | wang lin <lwan> |
| Component: | Image Registry | Assignee: | Oleg Bulatov <obulatov> |
| Status: | CLOSED WONTFIX | QA Contact: | Wenjing Zheng <wzheng> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.8 | CC: | aos-bugs, arane, ctauchen, jdiaz, jrouth, lwan, obulatov, wzheng, yunjiang |
| Target Milestone: | --- | Keywords: | TestBlocker |
| Target Release: | 4.8.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1939842 | Environment: | |
| Last Closed: | 2021-07-20 11:23:42 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1939842 | ||
| Bug Blocks: | 1974499 | ||
Met below 500 error: err.detail="s3aws: WebIdentityErr: failed to retrieve credentials\ncaused by: RequestError: send request failed\ncaused by: Post \"https://sts.amazonaws.com/\": dial tcp 52.46.134.192:443: i/o timeout" when trigger build with STS+disconnected cluster time="2021-07-02T02:14:59.351119646Z" level=info msg="authorized request" go.version=go1.15.7 http.request.host="image-registry.openshift-image-registry.svc:5000" http.request.id=662f0911-5cdc-4b59-b89c-7cca83b3d5ba http.request.method=GET http.request.remoteaddr="10.129.2.24:33082" http.request.uri="/v2/openshift/httpd/manifests/sha256:e48906d6ce958d7b545808fc3b115bb7e60bde9c7a61b4049bf16fa16e480537" http.request.useragent="containers/5.10.6 (github.com/containers/image)" openshift.auth.user="system:serviceaccount:wzheng1:builder" vars.name=openshift/httpd vars.reference="sha256:e48906d6ce958d7b545808fc3b115bb7e60bde9c7a61b4049bf16fa16e480537" time="2021-07-02T02:19:00.004341792Z" level=error msg="response completed with error" err.code=unknown err.detail="s3aws: WebIdentityErr: failed to retrieve credentials\ncaused by: RequestError: send request failed\ncaused by: Post \"https://sts.amazonaws.com/\": dial tcp 52.46.134.192:443: i/o timeout" err.message="unknown error" go.version=go1.15.7 http.request.host="image-registry.openshift-image-registry.svc:5000" http.request.id=662f0911-5cdc-4b59-b89c-7cca83b3d5ba http.request.method=GET http.request.remoteaddr="10.129.2.24:33082" http.request.uri="/v2/openshift/httpd/manifests/sha256:e48906d6ce958d7b545808fc3b115bb7e60bde9c7a61b4049bf16fa16e480537" http.request.useragent="containers/5.10.6 (github.com/containers/image)" http.response.contenttype="application/json; charset=utf-8" http.response.duration=4m0.669550251s http.response.status=500 http.response.written=104 openshift.auth.user="system:serviceaccount:wzheng1:builder" vars.name=openshift/httpd vars.reference="sha256:e48906d6ce958d7b545808fc3b115bb7e60bde9c7a61b4049bf16fa16e480537" |
Verified with cluster-bot image with pr#701 merged, cluster image registry can respect serviceEndpoint in infrastructure and work well. $ oc get co image-registry NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE image-registry 4.8.0-0.ci.test-2021-06-30-021220-ci-ln-81p79j2-latest True False False 3h45m $ oc get infrastructure cluster -o yaml apiVersion: config.openshift.io/v1 kind: Infrastructure metadata: creationTimestamp: "2021-06-30T02:39:23Z" generation: 1 name: cluster resourceVersion: "680" uid: d9bd533b-4254-4daa-8921-d74bb153cc0f spec: cloudConfig: name: "" platformSpec: aws: serviceEndpoints: - name: sts url: https://sts.us-east-2.amazonaws.com type: AWS status: apiServerInternalURI: https://api-int.lwanipid0630.qe.devcluster.openshift.com:6443 apiServerURL: https://api.lwanipid0630.qe.devcluster.openshift.com:6443 controlPlaneTopology: HighlyAvailable etcdDiscoveryDomain: "" infrastructureName: lwanipid0630-vsbb5 infrastructureTopology: HighlyAvailable platform: AWS platformStatus: aws: region: us-east-2 serviceEndpoints: - name: sts url: https://sts.us-east-2.amazonaws.com type: AWS