Bug 2046434

Summary:	Submariner 0.11.1 pulls version 0.11.0
Product:	Red Hat Advanced Cluster Management for Kubernetes	Reporter:	Noam Manos <nmanos>
Component:	Submariner	Assignee:	Mike Kolesnik <mkolesni>
Status:	CLOSED ERRATA	QA Contact:	Noam Manos <nmanos>
Severity:	high	Docs Contact:	Christopher Dawson <cdawson>
Priority:	unspecified
Version:	rhacm-2.4.z	CC:	mbabushk, nyechiel, skitt
Target Milestone:	---	Flags:	bot-tracker-sync: rhacm-2.4.z+
Target Release:	rhacm-2.4.2
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-02-28 18:36:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Noam Manos 2022-01-26 16:49:31 UTC

Description of problem:
When creating SubmarinerConfig to be used by Submariner ACM addon, 
and configuring CSV for Submariner version 0.11.1 - Then Submariner is installed, but the Submariner Gateway pods are pulling previous image version 0.11.0.

Version-Release number of selected component (if applicable):
ACM 2.4.2
Submariner 0.11.1

How reproducible:
Always

Steps to Reproduce:
https://qe-jenkins-csb-skynet.apps.ocp4.prod.psi.redhat.com/view/acm-2.4.2/job/ACM-2.4.2-Submariner-0.11.1-OSP-AWS-SDN/7/Test-Report/

1. Install ACM 2.4.2
2. Create submarinerconfig with starting CSV of "submariner.v0.11.1" .
3. Watch the Submariner gateway and pods, and see which image version is being pulled (it should be 0.11.1)


Actual results:


# Created submarinerconfig with starting CSV of "submariner.v0.11.1" :

$ oc  describe submarinerconfig submariner -n acm-nmanos-devcluster-a2-aws
 
 Name:         submariner
 Namespace:    acm-nmanos-devcluster-a2-aws
 Labels:       <none>
 Annotations:  <none>
 API Version:  submarineraddon.open-cluster-management.io/v1alpha1
 Kind:         SubmarinerConfig
 Metadata:
   Creation Timestamp:  2022-01-26T09:48:19Z
   Finalizers:
     submarineraddon.open-cluster-management.io/config-cleanup
   Generation:  3
   Managed Fields:
     API Version:  submarineraddon.open-cluster-management.io/v1alpha1
     Fields Type:  FieldsV1
     fieldsV1:
       f:metadata:
         f:finalizers:
           .:
           v:"submarineraddon.open-cluster-management.io/config-cleanup":
     Manager:      submariner
     Operation:    Update
     Time:         2022-01-26T09:48:19Z
     API Version:  submarineraddon.open-cluster-management.io/v1alpha1
     Fields Type:  FieldsV1
     fieldsV1:
       f:status:
         .:
         f:conditions:
         f:managedClusterInfo:
           .:
           f:clusterName:
           f:infraId:
           f:platform:
           f:region:
           f:vendor:
     Manager:      submariner
     Operation:    Update
     Subresource:  status
     Time:         2022-01-26T09:48:24Z
     API Version:  submarineraddon.open-cluster-management.io/v1alpha1
     Fields Type:  FieldsV1
     fieldsV1:
       f:metadata:
         f:annotations:
           .:
           f:kubectl.kubernetes.io/last-applied-configuration:
       f:spec:
         .:
         f:IPSecIKEPort:
         f:IPSecNATTPort:
         f:NATTDiscoveryPort:
         f:NATTEnable:
         f:cableDriver:
         f:credentialsSecret:
           .:
           f:name:
         f:gatewayConfig:
           .:
           f:aws:
             .:
             f:instanceType:
           f:gateways:
         f:imagePullSpecs:
           .:
           f:lighthouseAgentImagePullSpec:
           f:lighthouseCoreDNSImagePullSpec:
           f:submarinerImagePullSpec:
           f:submarinerRouteAgentImagePullSpec:
         f:subscriptionConfig:
           .:
           f:channel:
           f:source:
           f:sourceNamespace:
           f:startingCSV:
     Manager:         kubectl-client-side-apply
     Operation:       Update
     Time:            2022-01-26T12:58:23Z
   Resource Version:  378587
   UID:               6d6ccb0c-fefc-4d61-9196-b69d2ff37182
 Spec:
   IP Sec IKE Port:      502
   IP Sec NATT Port:     4502
   NATT Discovery Port:  4900
   NATT Enable:          true
   Cable Driver:         libreswan
   Credentials Secret:
     Name:  acm-nmanos-devcluster-a2-aws-aws-creds
   Gateway Config:
     Aws:
       Instance Type:  c5d.large
     Gateways:         1
   Image Pull Specs:
     Lighthouse Agent Image Pull Spec:        
     Lighthouse Core DNS Image Pull Spec:     
     Submariner Image Pull Spec:              
     Submariner Route Agent Image Pull Spec:  
   Subscription Config:
     Channel:           alpha-0.11
     Source:            my-submariner-catalog
     Source Namespace:  submariner-operator
     Starting CSV:      submariner.v0.11.1
 Status:
   Conditions:
     Last Transition Time:  2022-01-26T09:48:24Z
     Message:               Submariner cluster environment was prepared
     Reason:                SubmarinerClusterEnvPrepared
     Status:                True
     Type:                  SubmarinerClusterEnvironmentPrepared
     Last Transition Time:  2022-01-26T09:48:31Z
     Message:               SubmarinerConfig was applied
     Reason:                SubmarinerConfigApplied
     Status:                True
     Type:                  SubmarinerConfigApplied
     Last Transition Time:  2022-01-26T09:51:29Z
     Message:               1 node(s) ("ip-10-0-0-95.us-west-1.compute.internal") are labeled as gateways
     Reason:                Success
     Status:                True
     Type:                  SubmarinerGatewaysLabeled
   Managed Cluster Info:
     Cluster Name:  acm-nmanos-devcluster-a2-aws
     Infra Id:      nmanos-devcluster-a2-qgnxp
     Platform:      AWS
     Region:        us-west-1
     Vendor:        OpenShift
 Events:            <none>



# But Submariner object was created with version v0.11.0 :

$ oc  get Submariner -n submariner-operator -o yaml
 
 apiVersion: v1
 items:
 - apiVersion: submariner.io/v1alpha1
   kind: Submariner
   metadata:
     creationTimestamp: "2022-01-26T13:00:15Z"
     generation: 1
     name: submariner
     namespace: submariner-operator
     ownerReferences:
     - apiVersion: work.open-cluster-management.io/v1
       kind: AppliedManifestWork
       name: e010f7b2bbb6e1cedae9ba8bdc286fef93e0af71937a6ef89284003a006057bd-submariner-operator
       uid: f5570e7e-275f-49a4-9103-e58e82154d51
     resourceVersion: "393642"
     uid: 9e9862eb-9a90-4e38-9ff2-3ba11eee0151
   spec:
     broker: k8s
     brokerK8sApiServer: api.nmanos-devcluster-a2-aws.devcluster.openshift.com:6443
     brokerK8sApiServerToken: ...
     brokerK8sCA: ...
     brokerK8sRemoteNamespace: submariner-broker
     cableDriver: libreswan
     ceIPSecDebug: false
     ceIPSecIKEPort: 502
     ceIPSecNATTPort: 4502
     ceIPSecPSK: 7eK0MHgRyvP7g6XkJ7euXInW87JLHgkozTtX8qQc4aC+xKFsN9wfIzbiJ2ICnvWP
     clusterCIDR: ""
     clusterID: acm-nmanos-devcluster-a2-aws
     colorCodes: blue
     debug: false
     namespace: submariner-operator
     natEnabled: true
     repository: registry.redhat.io/rhacm2-tech-preview
     serviceCIDR: ""
     serviceDiscoveryEnabled: true
     version: v0.11.0
   status:
     clusterCIDR: 10.128.0.0/14
     clusterID: acm-nmanos-devcluster-a2-aws
     colorCodes: blue
     deploymentInfo: {}
     gatewayDaemonSetStatus:
       lastResourceVersion: "386409"
       mismatchedContainerImages: false
       nonReadyContainerStates: []
       status:
         currentNumberScheduled: 1
         desiredNumberScheduled: 1
         numberAvailable: 1
         numberMisscheduled: 0
         numberReady: 1
         observedGeneration: 1
         updatedNumberScheduled: 1
     gateways:
     - connections:
       - endpoint:
           backend: libreswan
           backend_config:
             natt-discovery-port: "4490"
             preferred-server: "false"
             udp-port: "4502"
           cable_name: submariner-cable-acm-api-default-cl1-devcluster-openshift-com-10-167-3-176
           cluster_id: acm-api-default-cl1-devcluster-openshift-com
           healthCheckIP: 10.231.0.1
           hostname: default-cl1-7cdfw-worker-0-gfck4
           nat_enabled: true
           private_ip: 10.167.3.176
           public_ip: 66.187.232.127
           subnets:
           - 100.97.0.0/16
           - 10.228.0.0/14
         latencyRTT:
           average: 68.037891ms
           last: 67.852893ms
           max: 70.305875ms
           min: 67.834789ms
           stdDev: 196.691��s
         status: connected
         statusMessage: ""
         usingIP: 66.187.232.127
         usingNAT: true
       haStatus: active
       localEndpoint:
         backend: libreswan
         backend_config:
           natt-discovery-port: "4490"
           preferred-server: "false"
           udp-port: "4502"
         cable_name: submariner-cable-acm-nmanos-devcluster-a2-aws-10-0-0-95
         cluster_id: acm-nmanos-devcluster-a2-aws
         healthCheckIP: 10.130.2.1
         hostname: ip-10-0-0-95
         nat_enabled: true
         private_ip: 10.0.0.95
         public_ip: 54.215.111.168
         subnets:
         - 172.30.0.0/16
         - 10.128.0.0/14
       statusFailure: ""
       version: v0.11.0
     globalnetDaemonSetStatus:
       mismatchedContainerImages: false
     loadBalancerStatus: {}
     natEnabled: true
     networkPlugin: OpenShiftSDN
     routeAgentDaemonSetStatus:
       lastResourceVersion: "386407"
       mismatchedContainerImages: false
       nonReadyContainerStates: []
       status:
         currentNumberScheduled: 7
         desiredNumberScheduled: 7
         numberAvailable: 7
         numberMisscheduled: 0
         numberReady: 7
         observedGeneration: 1
         updatedNumberScheduled: 7
     serviceCIDR: 172.30.0.0/16
 kind: List
 metadata:
   resourceVersion: ""
   selfLink: ""


# And also the Gateway was created with version v0.11.0 :

$ oc  describe Gateway -n submariner-operator 
   
 Name:         ip-10-0-0-95
 Namespace:    submariner-operator
 Labels:       <none>
 Annotations:  update-timestamp: 1643202385
 API Version:  submariner.io/v1
 Kind:         Gateway
 Metadata:
   Creation Timestamp:  2022-01-26T13:00:33Z
   Generation:          52
   Managed Fields:
     API Version:  submariner.io/v1
     Fields Type:  FieldsV1
     fieldsV1:
       f:metadata:
         f:annotations:
           .:
           f:update-timestamp:
       f:status:
         .:
         f:connections:
         f:haStatus:
         f:localEndpoint:
           .:
           f:backend:
           f:backend_config:
             .:
             f:natt-discovery-port:
             f:preferred-server:
             f:udp-port:
           f:cable_name:
           f:cluster_id:
           f:healthCheckIP:
           f:hostname:
           f:nat_enabled:
           f:private_ip:
           f:public_ip:
           f:subnets:
         f:statusFailure:
         f:version:
     Manager:         submariner-gateway
     Operation:       Update
     Time:            2022-01-26T13:00:33Z
   Resource Version:  394208
   UID:               977baa18-1ca3-47cf-b1de-8d00c5daa756
 Status:
   Connections:
     Endpoint:
       Backend:  libreswan
       backend_config:
         Natt - Discovery - Port:  4490
         Preferred - Server:       false
         Udp - Port:               4502
       cable_name:                 submariner-cable-acm-api-default-cl1-devcluster-openshift-com-10-167-3-176
       cluster_id:                 acm-api-default-cl1-devcluster-openshift-com
       Health Check IP:            10.231.0.1
       Hostname:                   default-cl1-7cdfw-worker-0-gfck4
       nat_enabled:                true
       private_ip:                 10.167.3.176
       public_ip:                  66.187.232.127
       Subnets:
         100.97.0.0/16
         10.228.0.0/14
     Latency RTT:
       Average:       68.042884ms
       Last:          68.174307ms
       Max:           70.305875ms
       Min:           67.834789ms
       Std Dev:       203.762��s
     Status:          connected
     Status Message:  
     Using IP:        66.187.232.127
     Using NAT:       true
   Ha Status:         active
   Local Endpoint:
     Backend:  libreswan
     backend_config:
       Natt - Discovery - Port:  4490
       Preferred - Server:       false
       Udp - Port:               4502
     cable_name:                 submariner-cable-acm-nmanos-devcluster-a2-aws-10-0-0-95
     cluster_id:                 acm-nmanos-devcluster-a2-aws
     Health Check IP:            10.130.2.1
     Hostname:                   ip-10-0-0-95
     nat_enabled:                true
     private_ip:                 10.0.0.95
     public_ip:                  54.215.111.168
     Subnets:
       172.30.0.0/16
       10.128.0.0/14
   Status Failure:  
   Version:         v0.11.0
 Events:            <none>


# Submariner gateway pod pulled image "registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8:v0.11.0" :

$ oc  describe pod submariner-gateway-x2cdz -n submariner-operator

 Name:         submariner-gateway-x2cdz
 Namespace:    submariner-operator
 Priority:     0
 Node:         ip-10-0-0-95.us-west-1.compute.internal/10.0.0.95
 Start Time:   Wed, 26 Jan 2022 15:00:16 +0200
 Labels:       app=submariner-gateway
               controller-revision-hash=7ddcb767d8
               gateway.submariner.io/node=ip-10-0-0-95.us-west-1.compute.internal
               gateway.submariner.io/status=active
               pod-template-generation=1
 Annotations:  openshift.io/scc: submariner-agent-scc
 Status:       Running
 IP:           10.0.0.95
 IPs:
   IP:           10.0.0.95
 Controlled By:  DaemonSet/submariner-gateway
 Containers:
   submariner-gateway:
     Container ID:  cri-o://987106628d0e0c33f5a7eda64f1d572bee043d4129f31c9e5254841c79e55e06
     Image:         registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8:v0.11.0
     Image ID:      registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8@sha256:51be5ffd22c895df73b2d89ef8583351f47a0891fab63ff0d8219fb7a47da82f
     Ports:         4502/UDP, 4490/UDP
     Host Ports:    4502/UDP, 4490/UDP
     Command:
       submariner.sh
     State:          Running
       Started:      Wed, 26 Jan 2022 15:02:15 +0200
     Last State:     Terminated
       Reason:       Error
       Exit Code:    255
       Started:      Wed, 26 Jan 2022 15:02:08 +0200
       Finished:     Wed, 26 Jan 2022 15:02:14 +0200
     Ready:          True
     Restart Count:  2
     Environment:
       SUBMARINER_NAMESPACE:                      submariner-operator
       SUBMARINER_CLUSTERCIDR:                    10.128.0.0/14
       SUBMARINER_SERVICECIDR:                    172.30.0.0/16
       SUBMARINER_GLOBALCIDR:                     
       SUBMARINER_CLUSTERID:                      acm-nmanos-devcluster-a2-aws
       SUBMARINER_COLORCODES:                     blue
       SUBMARINER_DEBUG:                          false
       SUBMARINER_NATENABLED:                     true
       SUBMARINER_BROKER:                         k8s
       SUBMARINER_CABLEDRIVER:                    libreswan
       BROKER_K8S_APISERVER:                      api.nmanos-devcluster-a2-aws.devcluster.openshift.com:6443
       BROKER_K8S_APISERVERTOKEN:                 ...
       BROKER_K8S_REMOTENAMESPACE:                submariner-broker
       BROKER_K8S_CA:                             ...
       CE_IPSEC_PSK:                              7eK0MHgRyvP7g6XkJ7euXInW87JLHgkozTtX8qQc4aC+xKFsN9wfIzbiJ2ICnvWP
       CE_IPSEC_DEBUG:                            false
       SUBMARINER_HEALTHCHECKENABLED:             true
       SUBMARINER_HEALTHCHECKINTERVAL:            1
       SUBMARINER_HEALTHCHECKMAXPACKETLOSSCOUNT:  5
       NODE_NAME:                                  (v1:spec.nodeName)
       POD_NAME:                                  submariner-gateway-x2cdz (v1:metadata.name)
       CE_IPSEC_IKEPORT:                          502
       CE_IPSEC_NATTPORT:                         4502
       CE_IPSEC_PREFERREDSERVER:                  false
       CE_IPSEC_FORCEENCAPS:                      false
     Mounts:
       /etc/ipsec.d from ipsecd (rw)
       /lib/modules from libmodules (ro)
       /var/lib/ipsec/nss from ipsecnss (rw)
       /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rr7hv (ro)
 Conditions:
   Type              Status
   Initialized       True 
   Ready             True 
   ContainersReady   True 
   PodScheduled      True 
 Volumes:
   ipsecd:
     Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
     Medium:     
     SizeLimit:  <unset>
   ipsecnss:
     Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
     Medium:     
     SizeLimit:  <unset>
   libmodules:
     Type:          HostPath (bare host directory volume)
     Path:          /lib/modules
     HostPathType:  
   kube-api-access-rr7hv:
     Type:                    Projected (a volume that contains injected data from multiple sources)
     TokenExpirationSeconds:  3607
     ConfigMapName:           kube-root-ca.crt
     ConfigMapOptional:       <nil>
     DownwardAPI:             true
     ConfigMapName:           openshift-service-ca.crt
     ConfigMapOptional:       <nil>
 QoS Class:                   BestEffort
 Node-Selectors:              submariner.io/gateway=true
 Tolerations:                 op=Exists
                              node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                              node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                              node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                              node.kubernetes.io/not-ready:NoExecute op=Exists
                              node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                              node.kubernetes.io/unreachable:NoExecute op=Exists
                              node.kubernetes.io/unschedulable:NoSchedule op=Exists
 Events:
   Type     Reason        Age                    From               Message
   ----     ------        ----                   ----               -------
   Normal   Scheduled     6m23s                  default-scheduler  Successfully assigned submariner-operator/submariner-gateway-x2cdz to ip-10-0-0-95.us-west-1.compute.internal
   Normal   Pulling       6m23s                  kubelet            Pulling image "registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8:v0.11.0"
   Normal   Pulled        6m10s                  kubelet            Successfully pulled image "registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8:v0.11.0" in 12.971639034s
   Normal   Created       6m10s                  kubelet            Created container submariner-gateway
   Normal   Started       6m9s                   kubelet            Started container submariner-gateway
   Warning  NodeNotReady  5m15s                  node-controller    Node is not ready
   Normal   Pulled        4m25s (x2 over 4m31s)  kubelet            Container image "registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8:v0.11.0" already present on machine
   Normal   Created       4m24s (x2 over 4m31s)  kubelet            Created container submariner-gateway
   Normal   Started       4m24s (x2 over 4m31s)  kubelet            Started container submariner-gateway


Expected results:
All Submariner resources and pods should indicate that the version is as requested in SubmarinerConfig (0.11.1)


**Environment**:

### submariner-operator-bundle-index Image ###
id=image-registry.openshift-image-registry.svc:5000/submariner-operator/submariner-operator-bundle-index@sha256:586a61757a1fd2c5f76ea581e7a75f3d64508ceba80c76a08d3e8acada7bbf33
name=openshift/ose-operator-registry
release=202201210948.p0.g9379df5.assembly.art3171
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.10.0-202201210948.p0.g9379df5.assembly.art3171 
version=v4.10.0

### ocp-v4.0-art-dev Image ###
id=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a07d70ac656e482824b28a4467a891ed791cf85d5866f09481ef57ede29dbbd9

### ocp-v4.0-art-dev Image ###
id=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d13145aee9f58cf00d184b3359cec5506292084aa730e2bb59eed5572691ac8a

### rhacm2-tech-preview-submariner-operator-bundle Image ###
id=registry-proxy.engineering.redhat.com/rh-osbs/rhacm2-tech-preview-submariner-operator-bundle@sha256:8acfaeb40cdb26aa13b5c5a7ed61516f0a18b62f1a5c1802e96d7293d57c90a4
name=rhacm2-tech-preview/submariner-operator-bundle
release=2
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-operator-bundle/images/v0.11.1-2 
version=v0.11.1

### lighthouse-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-agent-rhel8@sha256:406ee7a116a34eaf64314984d5f37e77479221b0103b7ac91a614ade453a93f7

### lighthouse-coredns-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-coredns-rhel8@sha256:7c6572f157ca92f229cb0b33e2239e3e96ff5afc832c8e22e620bb9290109ac8

### submariner-gateway-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8@sha256:51be5ffd22c895df73b2d89ef8583351f47a0891fab63ff0d8219fb7a47da82f

### submariner-rhel8-operator Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-rhel8-operator@sha256:87c1929884c012dc2879dd0cbca1de4a335bea1c6891ded4f1718733d23928c8
name=rhacm2-tech-preview/submariner-rhel8-operator
release=4
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-rhel8-operator/images/v0.11.1-4 
version=v0.11.1

### submariner-route-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-route-agent-rhel8@sha256:3a753dbd44b32ca7278866cdbdab1ac52b322c09e1854436827587bfb65b5662

### submariner-addon-rhel8 Image ###
id=registry.redhat.io/rhacm2/submariner-addon-rhel8@sha256:60f070aa8bccd27deac943dbdf97796946432652feb3a6694da6a68ebc22441c
rhacm2-tech-preview-submariner-rhel8-operator@sha256:c71a317818701a51a837d83932cb3dec5a5f638e8a377c347f435109207e6710
rhacm2-tech-preview-submariner-gateway-rhel8@sha256:3cb7146150d84b44d425c75de66926180eec392517a91215c3f9574ce03ca6fb
rhacm2-tech-preview-submariner-route-agent-rhel8@sha256:fde5f24bfe0386d204fb8db6182bbcd360cd0cbc728caa334c654af08d3c5559
rhacm2-tech-preview-submariner-globalnet-rhel8@sha256:3010c7c10732921ead8ba290e5e38f00f0c03fc16e5d65f1cb4e57a78a36a525
rhacm2-tech-preview-submariner-networkplugin-syncer-rhel8@sha256:f4324b3ce0c1fca83d4745e18f3df84ab78f4f8c3be694ab50a066f80a69ce68
rhacm2-tech-preview-lighthouse-agent-rhel8@sha256:922529a4307f706c52b4e9b3badb440db8c228a09f93350c1332793c82665bea
rhacm2-tech-preview-lighthouse-coredns-rhel8@sha256:0e86bd4b5a0871c6b1b767c9116b295b33948e159b69c16e3252a1ca53836383

### ImageStream Tags (in namespace submariner-operator) ###

### lighthouse-agent-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-agent-rhel8/images/v0.11.1-1 
version=v0.11.1

### lighthouse-coredns-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-coredns-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-coredns-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-gateway-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-gateway-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-gateway-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-globalnet-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-globalnet-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-globalnet-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-networkplugin-syncer-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-operator-bundle:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-operator-bundle
release=2
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-operator-bundle/images/v0.11.1-2 
version=v0.11.1

### submariner-operator-bundle-index:v0.11.1 Image-Stream tag ###
name=openshift/ose-operator-registry
release=202201210948.p0.g9379df5.assembly.art3171
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.10.0-202201210948.p0.g9379df5.assembly.art3171 
version=v4.10.0

### submariner-rhel8-operator:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-rhel8-operator
release=4
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-rhel8-operator/images/v0.11.1-4 
version=v0.11.1

### submariner-route-agent-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-route-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-route-agent-rhel8/images/v0.11.1-1 
version=v0.11.1

Comment 1 Stephen Kitt 2022-01-27 13:04:40 UTC

Fixed by https://github.com/stolostron/submariner-addon/pull/262

Comment 2 Noam Manos 2022-01-31 17:49:51 UTC

In today Test run I see that the Image stream and pod images are only from 0.11.1:
https://qe-jenkins-csb-skynet.apps.ocp4.prod.psi.redhat.com/view/acm-2.4.2/job/ACM-2.4.2-Submariner-0.11.1-OSP-AWS-SDN/Test-Report/

However, "subctl show versions" still prints 0.11.0.


###############################################################
#       OCP cluster nmanos-devcluster-a2-aws information      #
###############################################################

# Cloud platform: Amazon

# OCP version: 4.10.0-0.nightly-2022-01-31-012936

### Submariner components ###

subctl version: v0.11.1

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         
 ✓ Showing versions

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 ✓ Showing versions
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 ✓ Showing versions
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         
 ✓ Showing versions

Cluster "nmanos-devcluster-a2-aws"
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         
 ✓ Showing versions

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.0         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                c71a317818701a5 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.0         

 ✓ Showing versions


ip-10-0-81-125                  active          All connections (1) are established
ip-10-0-81-125                  active          All connections (1) are established
ip-10-0-81-125                  active          All connections (1) are established
ip-10-0-81-125                  active          All connections (1) are established
ip-10-0-81-125                  active          All connections (1) are established
ip-10-0-81-125                  active          All connections (1) are established


### Images of Pods (in namespace submariner-operator) ###

### submariner-operator-bundle-index Image ###
id=image-registry.openshift-image-registry.svc:5000/submariner-operator/submariner-operator-bundle-index@sha256:586a61757a1fd2c5f76ea581e7a75f3d64508ceba80c76a08d3e8acada7bbf33
name=openshift/ose-operator-registry
release=202201210948.p0.g9379df5.assembly.art3171
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.10.0-202201210948.p0.g9379df5.assembly.art3171 
version=v4.10.0

### ocp-v4.0-art-dev Image ###
id=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7cb0ba83ed800c8ca56e5263ea9584e74c5a21b3278e082b5ead207964d08552

### ocp-v4.0-art-dev Image ###
id=quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f70cd9066dd7b9628c739641f257957b85329339623847b55b131ef7eaf5df6

### rhacm2-tech-preview-submariner-operator-bundle Image ###
id=registry-proxy.engineering.redhat.com/rh-osbs/rhacm2-tech-preview-submariner-operator-bundle@sha256:8acfaeb40cdb26aa13b5c5a7ed61516f0a18b62f1a5c1802e96d7293d57c90a4
name=rhacm2-tech-preview/submariner-operator-bundle
release=2
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-operator-bundle/images/v0.11.1-2 
version=v0.11.1

### lighthouse-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-agent-rhel8@sha256:406ee7a116a34eaf64314984d5f37e77479221b0103b7ac91a614ade453a93f7

### lighthouse-coredns-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-coredns-rhel8@sha256:7c6572f157ca92f229cb0b33e2239e3e96ff5afc832c8e22e620bb9290109ac8

### submariner-gateway-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8@sha256:51be5ffd22c895df73b2d89ef8583351f47a0891fab63ff0d8219fb7a47da82f

### submariner-rhel8-operator Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-rhel8-operator@sha256:87c1929884c012dc2879dd0cbca1de4a335bea1c6891ded4f1718733d23928c8
name=rhacm2-tech-preview/submariner-rhel8-operator
release=4
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-rhel8-operator/images/v0.11.1-4 
version=v0.11.1

### submariner-route-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-route-agent-rhel8@sha256:3a753dbd44b32ca7278866cdbdab1ac52b322c09e1854436827587bfb65b5662

### submariner-addon-rhel8 Image ###
id=registry.redhat.io/rhacm2/submariner-addon-rhel8@sha256:df105a4a95160eff8d170b2228299bc387c007fe0aff97da6f3f783511d393c0
rhacm2-tech-preview-submariner-rhel8-operator@sha256:c71a317818701a51a837d83932cb3dec5a5f638e8a377c347f435109207e6710
rhacm2-tech-preview-submariner-gateway-rhel8@sha256:3cb7146150d84b44d425c75de66926180eec392517a91215c3f9574ce03ca6fb
rhacm2-tech-preview-submariner-route-agent-rhel8@sha256:fde5f24bfe0386d204fb8db6182bbcd360cd0cbc728caa334c654af08d3c5559
rhacm2-tech-preview-submariner-globalnet-rhel8@sha256:3010c7c10732921ead8ba290e5e38f00f0c03fc16e5d65f1cb4e57a78a36a525
rhacm2-tech-preview-submariner-networkplugin-syncer-rhel8@sha256:f4324b3ce0c1fca83d4745e18f3df84ab78f4f8c3be694ab50a066f80a69ce68
rhacm2-tech-preview-lighthouse-agent-rhel8@sha256:922529a4307f706c52b4e9b3badb440db8c228a09f93350c1332793c82665bea
rhacm2-tech-preview-lighthouse-coredns-rhel8@sha256:0e86bd4b5a0871c6b1b767c9116b295b33948e159b69c16e3252a1ca53836383

### ImageStream Tags (in namespace submariner-operator) ###

### lighthouse-agent-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-agent-rhel8/images/v0.11.1-1 
version=v0.11.1

### lighthouse-coredns-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-coredns-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-coredns-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-gateway-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-gateway-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-gateway-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-globalnet-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-globalnet-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-globalnet-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-networkplugin-syncer-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8/images/v0.11.1-1 
version=v0.11.1

### submariner-operator-bundle:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-operator-bundle
release=2
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-operator-bundle/images/v0.11.1-2 
version=v0.11.1

### submariner-operator-bundle-index:v0.11.1 Image-Stream tag ###
name=openshift/ose-operator-registry
release=202201210948.p0.g9379df5.assembly.art3171
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.10.0-202201210948.p0.g9379df5.assembly.art3171 
version=v4.10.0

### submariner-rhel8-operator:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-rhel8-operator
release=4
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-rhel8-operator/images/v0.11.1-4 
version=v0.11.1

### submariner-route-agent-rhel8:v0.11.1 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-route-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-route-agent-rhel8/images/v0.11.1-1 
version=v0.11.1

Comment 7 Noam Manos 2022-02-14 17:30:27 UTC

Verified:
https://qe-jenkins-csb-skynet.apps.ocp4.prod.psi.redhat.com/view/ACM%202.4.2/job/ACM-2.4.2-Submariner-0.11.2-AWS-OSP/15/Test-Report/

# OCP version: 4.11.0-0.nightly-2022-02-12-075213

### Submariner components ###

subctl version: v0.11.2

Cluster "api-nmanos-devcluster-a2-aws-devcluster-openshift-com:6443"
 • Showing versions  ...
COMPONENT                       REPOSITORY                                            VERSION         
submariner                      registry.redhat.io/rhacm2-tech-preview                v0.11.2         
submariner-operator             registry.redhat.io/rhacm2-tech-preview                1ec21de673a2659 
service-discovery               registry.redhat.io/rhacm2-tech-preview                v0.11.2         


### LibreSwan version on the running Gateway pod: submariner-gateway-w6hdk ###
libreswan-4.4-4.el8_5.x86_64


### Images of Pods (in namespace submariner-operator) ###

### submariner-operator-bundle-index Image ###
id=image-registry.openshift-image-registry.svc:5000/submariner-operator/submariner-operator-bundle-index@sha256:39e82dfbe601452c1629b22e505548a823ec4b0b11a8c4ee98ab627828681b35
name=openshift/ose-operator-registry
release=202201310820.p0.g5863540.assembly.stream
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.10.0-202201310820.p0.g5863540.assembly.stream 
version=v4.10.0

### lighthouse-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-agent-rhel8@sha256:c5eeddb2249f2d0398a270dcc2c8b08e68539048eb4c2d09ba6203c65cb50dca
name=rhacm2-tech-preview/lighthouse-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-agent-rhel8/images/v0.11.2-1 
version=v0.11.2

### lighthouse-coredns-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/lighthouse-coredns-rhel8@sha256:69e0c6c499179c067bc5d40517b1cb6b60c4b9d8f36cbbe25ad4a8d807366180
name=rhacm2-tech-preview/lighthouse-coredns-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-coredns-rhel8/images/v0.11.2-1 
version=v0.11.2

### submariner-gateway-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-gateway-rhel8@sha256:5f766aaeddd7b46e27d302ed652354e823aa3590c3cb614e0a47e16427a63f42
name=rhacm2-tech-preview/submariner-gateway-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-gateway-rhel8/images/v0.11.2-1 
version=v0.11.2

### submariner-rhel8-operator Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-rhel8-operator@sha256:1ec21de673a2659b427c4d63719dabc272cc772a4b6924ad3cc23e7aec886a16

### submariner-route-agent-rhel8 Image ###
id=registry.redhat.io/rhacm2-tech-preview/submariner-route-agent-rhel8@sha256:d85597491e67f2a175f433533b2f8bf319dcc9d144eb6241cb226540d3fef676
name=rhacm2-tech-preview/submariner-route-agent-rhel8
release=1
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-route-agent-rhel8/images/v0.11.2-1 
version=v0.11.2

### submariner-addon-rhel8 Image ###
id=registry.redhat.io/rhacm2/submariner-addon-rhel8@sha256:0e6a737f6a3dc9ad3cc36ee9d57c881b7733dcfcf4ba27b04a5620de9210a0fb
rhacm2-tech-preview-submariner-rhel8-operator@sha256:1ec21de673a2659b427c4d63719dabc272cc772a4b6924ad3cc23e7aec886a16
rhacm2-tech-preview-submariner-gateway-rhel8@sha256:6050a07d2f1ddcb6d3fdbadfd1bf861f7694cf93ef880a1b720279ad4ce3e32d
rhacm2-tech-preview-submariner-route-agent-rhel8@sha256:430b212dc93dd5438340202552ad8aa623f9cc3eead8c3fb7c474d29871fa5c7
rhacm2-tech-preview-submariner-globalnet-rhel8@sha256:3dece88358e4eb74a9ee60f5d86c3ef8b499fe304cd51873c39831ae20b14f18
rhacm2-tech-preview-submariner-networkplugin-syncer-rhel8@sha256:e560ea10538be13995df6e0efe78f134125cd80136d0defdf2eba0c9ed46b5e7
rhacm2-tech-preview-lighthouse-agent-rhel8@sha256:44995ebdabe954d87eee1df63ac6d0d4c8ac72c5b7f28d4fe7fc39984cc5bb71
rhacm2-tech-preview-lighthouse-coredns-rhel8@sha256:9785867448400e3aeeb73b534a59319ff70a089301b279c40fa2c98fd404ad6d

Comment 11 errata-xmlrpc 2022-02-28 18:36:55 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHBA: Submariner 0.11.2 - bug fixes and container updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0707