Bug 1810025

Summary: Missing status in the catalogSource
Product: OpenShift Container Platform Reporter: Petr Balogh <pbalogh>
Component: OLMAssignee: Evan Cordell <ecordell>
OLM sub component: OLM QA Contact: Jian Zhang <jiazha>
Status: CLOSED DUPLICATE Docs Contact:
Severity: urgent    
Priority: urgent Keywords: Automation
Version: 4.3.0   
Target Milestone: ---   
Target Release: 4.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-12 14:30:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Logs of catalog sources none

Description Petr Balogh 2020-03-04 12:40:10 UTC 
Created attachment 1667481 [details]
Logs of catalog sources

Description of problem:
From some reason there is missing status on every catalogSource object.

I see this issue on one of the deployment on VmWare done here:
https://ocs4-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/qe-deploy-ocs-cluster/5134/


We do add of custom catalog source for deployment of OCS and checking for READY state. But from some reason in this installation we do not see any status in any of the catalogsource. This actually failed our job for deployment of OCS.

On other deployment I have on AWS I see this part in yaml output for catalogsource:

status:
  connectionState:
    address: ocs-catalogsource.openshift-marketplace.svc:50051
    lastConnect: "2020-03-04T11:27:04Z"
    lastObservedState: READY
  registryService:
    createdAt: "2020-03-04T11:23:00Z"
    port: "50051"
    protocol: grpc
    serviceName: ocs-catalogsource
    serviceNamespace: openshift-marketplace

But this part is missing on every catalogsource from some reason.


Version-Release number of selected component (if applicable):
 4.3.0-0.nightly-2020-03-03-144847


How reproducible:
Haven't tried to reproduce this yet.


Steps to Reproduce:
1. Install OCP 4.3 on Vmware
2. There is missing status under catalogSource

Actual results:
No status for any catalog source


Expected results:
See status of catalog source


Additional info:
Must gather logs:
http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/ebondare/ebondare_20200304T083449/logs/failed_testcase_ocs_logs_1583311786/deployment_ocs_logs/
Comment 1 Evan Cordell 2020-03-12 14:30:15 UTC 
The logs for the catalog operator indicate that it could not start successfully:

2020-03-04T09:27:33.236516275Z time="2020-03-04T09:27:33Z" level=info msg="log level info"
2020-03-04T09:27:33.236516275Z time="2020-03-04T09:27:33Z" level=info msg="TLS keys set, using https for metrics"
2020-03-04T09:27:33.467714412Z time="2020-03-04T09:27:33Z" level=info msg="Using in-cluster kube client config"
2020-03-04T09:27:33.470530138Z time="2020-03-04T09:27:33Z" level=info msg="Using in-cluster kube client config"
2020-03-04T09:27:33.479522185Z time="2020-03-04T09:27:33Z" level=info msg="Using in-cluster kube client config"
2020-03-04T09:28:03.74109388Z time="2020-03-04T09:28:03Z" level=info msg="operator not ready: communicating with server failed: Get https://172.30.0.1:443/version?timeout=32s: dial tcp 172.30.0.1:443: i/o timeout"

http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/ebondare/ebondare_20200304T083449/logs/failed_testcase_ocs_logs_1583311786/deployment_ocs_logs/ocp_must_gather/quay-io-openshift-origin-must-gather-sha256-ee4eae4c297a6f0c80de95d12266c61f7348349a3e72d909a294644e8371e3aa/namespaces/openshift-operator-lifecycle-manager/pods/catalog-operator-5f979d5bfd-45qq2/catalog-operator/catalog-operator/logs/current.log

There was a bug previously where this type of error would not cause the pod to fail, so the pod would hang indefinitely and not be reschedule to fix the problem.

This bug has since been fixed: 
4.5: https://bugzilla.redhat.com/show_bug.cgi?id=1807128
4.4: https://bugzilla.redhat.com/show_bug.cgi?id=1808418
4.3: https://bugzilla.redhat.com/show_bug.cgi?id=1808419

it should be rolled out in the next release of 4.3.

*** This bug has been marked as a duplicate of bug 1807128 ***