Bug 1771203

Summary: a CSV generated by cdi-operator could not be used with rh-verified-operators AND redhat-operators-stage
Product: Container Native Virtualization (CNV) Reporter: Simone Tiraboschi <stirabos>
Component: StorageAssignee: Michael Henriksen <mhenriks>
Status: CLOSED ERRATA QA Contact: Kevin Alon Goldblatt <kgoldbla>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 2.1.1CC: alitke, awels, cnv-qe-bugs, mhenriks, ncredi, ngavrilo, sgordon, stirabos, ycui
Target Milestone: ---Flags: alitke: needinfo-
Target Release: 2.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: virt-cdi-operator-container-v2.2.0-3 hco-bundle-registry-container-v2.2.0-62 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-30 16:27:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Simone Tiraboschi 2019-11-11 23:08:25 UTC
Description of problem:
cdi-operator starts other CDI components creating their deployment spec at runtime.
The image pull URLs are rendered at runtime according to the value of:

DOCKER_REPO, DOCKER_TAG, CONTROLLER_IMAGE... specified on HCO CSV on cdi-operator deployment.

In order to deploy from rh-verified-operators currently we need something like:

                - name: DOCKER_REPO
                  value: registry-proxy.engineering.redhat.com/rh-osbs
                - name: DOCKER_TAG
                  value: v2.1.1-24
                - name: CONTROLLER_IMAGE
                  value: container-native-virtualization-virt-cdi-controller
                - name: IMPORTER_IMAGE
                  value: container-native-virtualization-virt-cdi-importer
                - name: CLONER_IMAGE
                  value: container-native-virtualization-virt-cdi-cloner
                - name: APISERVER_IMAGE
                  value: container-native-virtualization-virt-cdi-apiserver
                - name: UPLOAD_SERVER_IMAGE
                  value: container-native-virtualization-virt-cdi-uploadserver
                - name: UPLOAD_PROXY_IMAGE
                  value: container-native-virtualization-virt-cdi-uploadproxy

while in order to deploy from redhat-operators-stage we need something like:


                - name: DOCKER_REPO
                  value: registry.stage.redhat.io/container-native-virtualization
                - name: DOCKER_TAG
                  value: v2.1.1-24
                - name: CONTROLLER_IMAGE
                  value: virt-cdi-controller
                - name: IMPORTER_IMAGE
                  value: virt-cdi-importer
                - name: CLONER_IMAGE
                  value: virt-cdi-cloner
                - name: APISERVER_IMAGE
                  value: virt-cdi-apiserver
                - name: UPLOAD_SERVER_IMAGE
                  value: virt-cdi-uploadserver
                - name: UPLOAD_PROXY_IMAGE
                  value: virt-cdi-uploadproxy

The value of DOCKER_REPO can be potentially modified by RCM scripts pushing from rh-verified-operators to redhat-operators-stage but nothing is going to correctly take care of *_IMAGE values.

So, according to HCO CSV content, the deployment can work from rh-verified-operators OR from redhat-operators-stage but not from both with the same CSV content.

The best option on my opinion is to pass the whole images pull URLs as entire values and avoid collating their pieces at runtime to be sure that RCM automation can properly handle them moving though different testing stages.


Version-Release number of selected component (if applicable):
virt-cdi-operator:v2.1.1-24

How reproducible:
100%

Steps to Reproduce:
1. deploy CNV though various testing stages
2.
3.

Actual results:
According to DOCKER_TAG and *_IMAGES values in HCO CSV the deployment can be performed from rh-verified-operators OR from redhat-operators-stage.

Expected results:
cdi-operator expects complete pull URLs for other images and doesn't internally manipulate them

Additional info:

Comment 1 Adam Litke 2019-11-12 18:15:35 UTC
Is this a new requirement?  I'm pretty sure our CSV was working at one point.  Can you point me to documentation explaining the requirements because it's not clear at all to me.

Comment 4 Adam Litke 2019-11-13 15:28:31 UTC
Moving this to 2.2.0.  We need to do a somewhat invasive change to change the way we specify container images in the csv and 2.2.0 is the first place we will target the fix.  Hopefully we can continue to employ the current 2.1.0 workaround for 2.1.1.  If that is not possible then we will clone this bug to 2.1.1.

Comment 7 Alexander Wels 2019-11-27 16:24:14 UTC
Upstream release v1.11.0 contains this fix.

@Adam did we make a d/s release of that?

Comment 8 Natalie Gavrielov 2019-12-18 13:19:48 UTC
We can verify this using redhat coreos workers.

Comment 9 Kevin Alon Goldblatt 2019-12-25 14:35:00 UTC
I successfully deployed 2 environments on OCP 4.3 CNV 2.2 :

Brew:
--------------------------------------------------------
[cnv-qe-jenkins@cnv-executor-kev4322e ~]$ export KUBECONFIG=/home/cnv-qe-jenkins/oc4/working/auth/kubeconfig 
[cnv-qe-jenkins@cnv-executor-kev4322e ~]$ oc -n openshift-cnv get pod -oyaml|grep "image:"|sort -u
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-bridge-marker:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cluster-network-addons-operator:v2.2.0-6
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cnv-containernetworking-plugins:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-hostpath-provisioner-rhel8-operator:v2.2.0-8
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-hostpath-provisioner-rhel8:v2.2.0-5
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-hyperconverged-cluster-operator:v2.2.0-10
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubemacpool:v2.2.0-4
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubernetes-nmstate-handler-rhel8:v2.2.0-12
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-cpu-model-nfd-plugin:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-cpu-node-labeller:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-kvm-info-nfd-plugin:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-ssp-operator:v2.2.0-14
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-template-validator:v2.2.0-4
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-node-maintenance-operator:v2.2.0-2
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-ovs-cni-marker:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-ovs-cni-plugin:v2.2.0-3
    - image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-ovs-cni-plugin:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-api:v2.2.0-10
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-cdi-apiserver:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-cdi-controller:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-cdi-operator:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-cdi-uploadproxy:v2.2.0-3
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-controller:v2.2.0-10
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-handler:v2.2.0-10
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-launcher:v2.2.0-10
      image: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-virt-operator:v2.2.0-10


Stage:
-----------------------------
oc -n openshift-cnv get pod -oyaml|grep -i "image:"|sort -u
      containerImage: registry.stage.redhat.io/container-native-virtualization/hyperconverged-cluster-operator:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/bridge-marker:v2.1.0-3
      image: registry.stage.redhat.io/container-native-virtualization/cluster-network-addons-operator:v2.1.0-13
      image: registry.stage.redhat.io/container-native-virtualization/cnv-containernetworking-plugins:v2.1.0-6
      image: registry.stage.redhat.io/container-native-virtualization/hyperconverged-cluster-operator:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/kubemacpool:v2.1.0-8
      image: registry.stage.redhat.io/container-native-virtualization/kubernetes-nmstate-handler-rhel8:v2.1.0-7
      image: registry.stage.redhat.io/container-native-virtualization/kubevirt-ssp-operator:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/kubevirt-template-validator:v2.1.0-12
      image: registry.stage.redhat.io/container-native-virtualization/node-maintenance-operator:v2.1.0-11
      image: registry.stage.redhat.io/container-native-virtualization/ovs-cni-marker:v2.1.0-9
      image: registry.stage.redhat.io/container-native-virtualization/ovs-cni-plugin:v2.1.0-9
    - image: registry.stage.redhat.io/container-native-virtualization/ovs-cni-plugin:v2.1.0-9
      image: registry.stage.redhat.io/container-native-virtualization/virt-api:v2.1.0-17
      image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-apiserver:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-controller:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-operator:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-uploadproxy:v2.1.0-22
    - image: registry.stage.redhat.io/container-native-virtualization/virt-cdi-uploadproxy:v2.1.0-22
      image: registry.stage.redhat.io/container-native-virtualization/virt-controller:v2.1.0-17
    - image: registry.stage.redhat.io/container-native-virtualization/virt-controller:v2.1.0-17
      image: registry.stage.redhat.io/container-native-virtualization/virt-handler:v2.1.0-17
    - image: registry.stage.redhat.io/container-native-virtualization/virt-handler:v2.1.0-17
      image: registry.stage.redhat.io/container-native-virtualization/virt-operator:v2.1.0-17


@stirabos Does this satisfy this issue?

Comment 11 Simone Tiraboschi 2020-01-08 10:34:51 UTC
(In reply to Kevin Alon Goldblatt from comment #9)
> I successfully deployed 2 environments on OCP 4.3 CNV 2.2 :

> Brew:

> Stage:

> @stirabos Does this satisfy this issue?

Yes, sorry for the delay in my response.

Comment 12 Natalie Gavrielov 2020-01-08 13:04:11 UTC
Following comment #9 and comment #11, moving to verified.

Comment 14 errata-xmlrpc 2020-01-30 16:27:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:0307