Bug 1893956
| Summary: | Installation always fails at "failed to initialize the cluster: Cluster operator image-registry is still updating" | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Xingxing Xia <xxia> | ||||
| Component: | Image Registry | Assignee: | Oleg Bulatov <obulatov> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Wenjing Zheng <wzheng> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 4.7 | CC: | aos-bugs, juzhao, lisowski | ||||
| Target Milestone: | --- | Keywords: | TestBlocker | ||||
| Target Release: | 4.7.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | No Doc Update | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1936984 (view as bug list) | Environment: | |||||
| Last Closed: | 2021-02-24 15:29:22 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1936984, 1940877 | ||||||
| Attachments: |
|
||||||
|
Description
Xingxing Xia
2020-11-03 08:09:00 UTC
# oc -n openshift-image-registry get pod image-registry-6fccd7bf5f-5g9l2 -oyaml
...
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2020-11-04T07:20:07Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2020-11-04T07:20:07Z"
message: 'containers with unready status: [registry]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2020-11-04T07:20:07Z"
message: 'containers with unready status: [registry]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2020-11-04T07:20:07Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: cri-o://326af805108456268c97896caa847e596f220bb9c6565ddfa35f6f2783ad6d31
image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c54f78026566c7fe18df411ee0d9b230c1ff8f2c696e52882909951a7d9efca2
imageID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c54f78026566c7fe18df411ee0d9b230c1ff8f2c696e52882909951a7d9efca2
lastState:
terminated:
containerID: cri-o://326af805108456268c97896caa847e596f220bb9c6565ddfa35f6f2783ad6d31
exitCode: 1
finishedAt: "2020-11-04T08:17:17Z"
reason: Error
startedAt: "2020-11-04T08:17:17Z"
name: registry
ready: false
restartCount: 16
started: false
state:
waiting:
message: back-off 5m0s restarting failed container=registry pod=image-registry-6fccd7bf5f-5g9l2_openshift-image-registry(6a2040ca-cdab-4658-a386-9bb5c16525af)
reason: CrashLoopBackOff
hostIP: 10.0.32.3
phase: Running
podIP: 10.128.2.8
podIPs:
- ip: 10.128.2.8
qosClass: Burstable
startTime: "2020-11-04T07:20:07Z"
continue with Comment 2, 3 image-registry pods, but the deployment only required 2 replicas, image-registry-7bc845d666-fbvhc and image-registry-6fccd7bf5f-5g9l2 are in the same node "zhsun114gcp-tk9lc-worker-c-rw8cg.c.openshift-qe.internal", # oc -n openshift-image-registry get pod -o wide | grep -Ev "Running|Completed" NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES image-registry-6fccd7bf5f-5g9l2 0/1 CrashLoopBackOff 20 78m 10.128.2.8 zhsun114gcp-tk9lc-worker-c-rw8cg.c.openshift-qe.internal <none> <none> image-registry-7bc845d666-5stcl 0/1 CrashLoopBackOff 20 78m 10.131.0.17 zhsun114gcp-tk9lc-worker-b-n9l4h.c.openshift-qe.internal <none> <none> image-registry-7bc845d666-fbvhc 0/1 CrashLoopBackOff 20 78m 10.128.2.7 zhsun114gcp-tk9lc-worker-c-rw8cg.c.openshift-qe.internal <none> <none> # oc -n openshift-image-registry get deploy image-registry NAME READY UP-TO-DATE AVAILABLE AGE image-registry 0/2 1 0 81m there are conflict replicas in spec and status section # oc -n openshift-image-registry get deploy image-registry -oyaml ... spec: progressDeadlineSeconds: 600 replicas: 2 ... status: conditions: - lastTransitionTime: "2020-11-04T07:20:06Z" lastUpdateTime: "2020-11-04T07:20:06Z" message: Deployment does not have minimum availability. reason: MinimumReplicasUnavailable status: "False" type: Available - lastTransitionTime: "2020-11-04T07:30:08Z" lastUpdateTime: "2020-11-04T07:30:08Z" message: ReplicaSet "image-registry-6fccd7bf5f" has timed out progressing. reason: ProgressDeadlineExceeded status: "False" type: Progressing observedGeneration: 2 replicas: 3 unavailableReplicas: 3 updatedReplicas: 1 Created attachment 1726494 [details]
image-registry deployment file
Verified in 4.7.0-0.nightly-2020-11-05-010603 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 *** Bug 1936006 has been marked as a duplicate of this bug. *** |