Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1767119

Summary:	OCP GCP install Cluster operator console has not yet reported success
Product:	OpenShift Container Platform	Reporter:	Chance Zibolski <chancez>
Component:	Management Console	Assignee:	Samuel Padgett <spadgett>
Status:	CLOSED DUPLICATE	QA Contact:	Yadan Pei <yapei>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	4.3.0	CC:	aos-bugs, jokerman, wking
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-30 19:13:07 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chance Zibolski 2019-10-30 17:54:52 UTC

Description of problem:

I'm seeing console fail to become successful in GCP based install CI, with the following error occurring:

level=fatal msg="failed to initialize the cluster: Cluster operator console has not yet reported success"


Version-Release number of selected component (if applicable): 4.3.0


How reproducible: according to https://ci-search-ci-search-next.svc.ci.openshift.org/?search=failed+to+initialize+the+cluster%3A+Cluster+operator+console+has+not+yet+reported+success&maxAge=336h&context=2&type=all at the time this bug was filed (Oct 30 10:54 AM PST), it's occurred 14 times in the last 24 hours.


Steps to Reproduce:
1.
2.
3.

Actual results: Console is not ready

Expected results: Console is ready and install succeeds


Additional info:

Comment 1 W. Trevor King 2019-10-30 18:27:14 UTC

Looks like most of these are blocked on the deployment, although it's not clear from the error message which deployment or how many replicas are expected:

$ curl -s 'https://ci-search-ci-search-next.svc.ci.openshift.org/search?search=info.*Cluster+operator+console+Available+is+False+with&context=0' | jq -r '. | to_entries[].value | to_entries[].value[].context[]' | sort | uniq | sed 's/at version .*/at version .../' | uniq -c | sort -n | tail
      7 level=info msg="Cluster operator console Available is False with DeploymentAvailableFailedUpdate: DeploymentAvailable: 1 replicas ready at version ...
     19 level=info msg="Cluster operator console Available is False with DeploymentAvailableFailedUpdate: DeploymentAvailable: 2 replicas ready at version ...

Full messages look like [1]:

level=info msg="Cluster operator console Available is False with DeploymentAvailableFailedUpdate: DeploymentAvailable: 2 replicas ready at version 0.0.1-2019-10-30-075903"
...
level=fatal msg="failed to initialize the cluster: Cluster operator console has not yet reported success" 

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/release-openshift-origin-installer-e2e-gcp-4.3/454

Comment 2 Samuel Padgett 2019-10-30 18:48:58 UTC

It looks like 2 of 3 replicas were ready, but one failed due to

> failed: Head https://oauth-openshift.apps.ci-op-hy71il25-18274.origin-ci-int-gce.dev.openshift.com: dial tcp 35.185.40.2:443: connect: connection refused

This doesn't seem like a console problem, rather something to do with networking.

Comment 3 Samuel Padgett 2019-10-30 19:11:33 UTC

Our deployment only has 2 replicas, but something triggered a console rollout. One of the new replicas didn't become ready due to the error from comment #2.

Comment 4 Samuel Padgett 2019-10-30 19:13:07 UTC

This looks to be identical to Bug 1766664

*** This bug has been marked as a duplicate of bug 1766664 ***