Bug 1802212
| Summary: | MCG/NooBaa noobaa-core and PV Pool statefulset pods are created using an upstream image/tag | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat OpenShift Container Storage | Reporter: | Jason Montleon <jmontleo> |
| Component: | ocs-operator | Assignee: | Jacky Albo <jalbo> |
| Status: | CLOSED ERRATA | QA Contact: | Ben Eli <belimele> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.2 | CC: | ebenahar, etamir, madam, nbecker, ocs-bugs, sostapov |
| Target Milestone: | --- | Keywords: | AutomationBackLog |
| Target Release: | OCS 4.3.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-04-14 09:45:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
ocs-operator has env vars for these images but they are not being used:
$ oc describe po ocs-operator-5694b5bc77-mnk7l | grep -e mcg -e mongo
NOOBAA_CORE_IMAGE: registry.redhat.io/ocs4/mcg-core-rhel8@sha256:1906a018df2774f1d0338483255c5f314fe707afcf12271f53a346790de9fdcf
NOOBAA_DB_IMAGE: registry.redhat.io/rhscl/mongodb-36-rhel7@sha256:506e4daa19721199e35249f2e0e67b02cbcbee1fed653f7eb91d5595b74c3bf8
For 4.3, we provided the ability to set ENV on the ocs-operator and provide the desired images (attached the PR). Nimrod I see these Env vars on the ocs-operator per my comment https://bugzilla.redhat.com/show_bug.cgi?id=1802212#c2 Is there a bug in 4.2.1 where they're not being used that's fixed or getting fixed in OpenShift Container Storage 4.3? @Jason Exactly. In 4.2.x we did not use them, now we added them and we should use (pass the correct) value which is targeted for 4.3 The feature freeze build has this Patch: https://ceph-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OCS%20Build%20Pipeline%204.3/82/ Seems to have been fixed. > oc describe po mcg-pv-pool-bs-noobaa-noobaa-2 | grep Image Image: quay.io/rhceph-dev/mcg-core@sha256:9fdf00038cfb64f044ea71e4aaa76980c5e0407b462eb326e051377d9669a3b6 Image ID: quay.io/rhceph-dev/mcg-core@sha256:9fdf00038cfb64f044ea71e4aaa76980c5e0407b462eb326e051377d9669a3b6 > oc describe po noobaa-core-0 | grep Image Image: quay.io/rhceph-dev/mcg-core@sha256:9fdf00038cfb64f044ea71e4aaa76980c5e0407b462eb326e051377d9669a3b6 Image ID: quay.io/rhceph-dev/mcg-core@sha256:9fdf00038cfb64f044ea71e4aaa76980c5e0407b462eb326e051377d9669a3b6 Verified. OCS 4.3.0-373.ci Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:1437 |
Description of problem (please be detailed as possible and provide log snippests): When installing OpenShift Container Storage from OLM on OpenShift 4.3 the noobaa-core-0 and noobaa statefulset pods for PV Pools are created using an upstream image. Version of all relevant components (if applicable): 4.2.1 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? It works, but our requirement to work in disconnected/restricted network environments is limited due to this issue. Is there any workaround available to the best of your knowledge? No. Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 2 Can this issue reproducible? Yes. Can this issue reproduce from the UI? No. If this is a regression, please provide more details to justify this: Steps to Reproduce: 1. Install OCS operator 2. Create a NooBaa System 3. Create a PV Pool Backing Store apiVersion: noobaa.io/v1alpha1 kind: BackingStore metadata: finalizers: - noobaa.io/finalizer labels: app: noobaa name: mcg-pv-pool-bs namespace: openshift-storage spec: pvPool: numVolumes: 3 resources: requests: storage: 50Gi storageClass: gp2 type: pv-pool Actual results: The downstream operator: $ oc describe po noobaa-operator-59894b6668-l6l6s | grep Image Image: registry.redhat.io/ocs4/mcg-rhel8-operator@sha256:ecb1c4e486aca77bda000fa71d190f90e06173447cff1bed983fb6e246975863 Image ID: registry.redhat.io/ocs4/mcg-rhel8-operator@sha256:ecb1c4e486aca77bda000fa71d190f90e06173447cff1bed983fb6e246975863 is creating the noobaa core and statefulset pods using upstream images: $ oc describe po mcg-pv-pool-bs-noobaa-noobaa-2 | grep Image Image: noobaa/noobaa-core:5.2.11 Image ID: docker.io/noobaa/noobaa-core@sha256:4389a1bb98083946b7837837613e7e64829333b166a2acd664093ced3e034fa6 $ oc describe po mcg-pv-pool-bs-noobaa-noobaa-2 | grep Image Image: noobaa/noobaa-core:5.2.11 Image ID: docker.io/noobaa/noobaa-core@sha256:4389a1bb98083946b7837837613e7e64829333b166a2acd664093ced3e034fa6 [jmontleo@jmontleo mig-operator]$ oc describe po noobaa-core-0 | grep Image Image: noobaa/noobaa-core:5.2.11 Image ID: docker.io/noobaa/noobaa-core@sha256:4389a1bb98083946b7837837613e7e64829333b166a2acd664093ced3e034fa6 Image: noobaa/noobaa-core:5.2.11 Image ID: docker.io/noobaa/noobaa-core@sha256:4389a1bb98083946b7837837613e7e64829333b166a2acd664093ced3e034fa6 Image: centos/mongodb-36-centos7 Image ID: docker.io/centos/mongodb-36-centos7@sha256:48a9fc021f16138cda94dc36914610df78caac36d751218ae499cc9c6363194b Expected results: The noobaa core and statefulset pods are created using the downstream images referenced in the CSV: registry.redhat.io/ocs4/mcg-core-rhel8@sha256:1906a018df2774f1d0338483255c5f314fe707afcf12271f53a346790de9fdcf registry.redhat.io/rhscl/mongodb-36-rhel7@sha256:506e4daa19721199e35249f2e0e67b02cbcbee1fed653f7eb91d5595b74c3bf8 Additional info: Tying into https://bugzilla.redhat.com/show_bug.cgi?id=1798052 tags cannot be used to mirror images in OpenShift 4 because mirror by tag is turned off in OpenShift 4 cri-o, making it impossible to use noobaa in a disconnected environment even if you do determine the list of images to mirror and do so manually.