2048741 – ACM 2.4.1 + Submariner 0.11.0: SubmarinerAgentDegraded since 0.11 channel ships only latest version 0.11.z

Bug 2048741 - ACM 2.4.1 + Submariner 0.11.0: SubmarinerAgentDegraded since 0.11 channel ships only latest version 0.11.z

Summary: ACM 2.4.1 + Submariner 0.11.0: SubmarinerAgentDegraded since 0.11 channel shi...

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	Red Hat Advanced Cluster Management for Kubernetes
Classification:	Red Hat
Component:	Submariner
Sub Component:
Version:	rhacm-2.4.z
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	medium
Target Milestone:	---
Target Release:	---
Assignee:	Stephen Kitt
QA Contact:	Noam Manos
Docs Contact:	Christopher Dawson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2022-01-31 18:12 UTC by Noam Manos
Modified:	2022-03-29 10:22 UTC (History)
CC List:	2 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2022-03-29 10:22:34 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Github	stolostron backlog issues 19575	None	None	None	2022-01-31 21:26:55 UTC
Github	stolostron submariner-addon issues 270	None	closed	Add InstallPlan variable to SubmarinerConfig	2022-02-15 14:28:05 UTC
Github	stolostron submariner-addon pull 284	None	Merged	Add InstallPlan configuration to SubmarinerConfig	2022-02-16 15:24:19 UTC

Description Noam Manos 2022-01-31 18:12:38 UTC

**What happened**:
Lately the Submariner addon deployment for 0.11.0 has started to fail due to SubmarinerAgentDegraded: 
The submariner-operator CSV (submariner.v0.11.0) is not installed from channel (alpha-0.11) in catalog source (submariner-operator/my-submariner-catalog).

It happens since Submariner "alpha-0.11" channel ships just the latest 0.11.1 version.


This is the SubmarinerConfig we apply:

apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: acm-nmanos-devcluster-a-aws
spec:
  IPSecIKEPort: 501
  IPSecNATTPort: 4501
  cableDriver: libreswan
  credentialsSecret:
    name: acm-nmanos-devcluster-a-aws-aws-creds
  gatewayConfig:
    aws:
      instanceType: c5d.large
    gateways: 1
  imagePullSpecs:
    lighthouseAgentImagePullSpec: ''
    lighthouseCoreDNSImagePullSpec: ''
    submarinerImagePullSpec: ''
    submarinerRouteAgentImagePullSpec: ''
  subscriptionConfig:
    channel: alpha-0.11
    source: my-submariner-catalog
    sourceNamespace: submariner-operator
    startingCSV: submariner.v0.11.0

    
# Looking at Submariner addon, we see that it could not find 0.11.0 version in "alpha-0.11" channel:

$ oc  describe managedclusteraddons submariner -n acm-nmanos-devcluster-a-aws

Name:         submariner
Namespace:    acm-nmanos-devcluster-a-aws
Labels:       <none>
Annotations:  <none>
API Version:  addon.open-cluster-management.io/v1alpha1
Kind:         ManagedClusterAddOn
Metadata:
  Creation Timestamp:  2022-01-31T07:43:51Z
  Finalizers:
    submarineraddon.open-cluster-management.io/submariner-addon-cleanup
  Generation:  1
  Managed Fields:
    API Version:  addon.open-cluster-management.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:installNamespace:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2022-01-31T07:43:51Z
    API Version:  addon.open-cluster-management.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"submarineraddon.open-cluster-management.io/submariner-addon-cleanup":
      f:status:
        .:
        f:addOnConfiguration:
          .:
          f:crName:
          f:crdName:
        f:addOnMeta:
          .:
          f:description:
          f:displayName:
        f:registrations:
    Manager:      submariner
    Operation:    Update
    Time:         2022-01-31T07:43:56Z
    API Version:  addon.open-cluster-management.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
    Manager:         submariner
    Operation:       Update
    Subresource:     status
    Time:            2022-01-31T15:02:06Z
  Resource Version:  791324
  UID:               68fc1d4b-b2e5-4b0d-97df-af0749aeeb77
Spec:
  Install Namespace:  submariner-operator
Status:
  Add On Configuration:
    Cr Name:   
    Crd Name:  
  Add On Meta:
    Description:   
    Display Name:  
  Conditions:
    Last Transition Time:  2022-01-31T07:43:52Z
    Message:               Registration of the addon agent is configured
    Reason:                RegistrationConfigured
    Status:                True
    Type:                  RegistrationApplied
    Last Transition Time:  2022-01-31T15:01:12Z
    Message:               manifest of addon applied successfully
    Reason:                AddonManifestApplied
    Status:                True
    Type:                  ManifestApplied
    Last Transition Time:  2022-01-31T07:44:03Z
    Message:               The submariner-operator CSV (submariner.v0.11.0) is not installed from channel (alpha-0.11) in catalog source (submariner-operator/my-submariner-catalog)
The submariner operator deployment does not exist
The gateway daemon set does not exist
The route agents are not found
    Reason:                CSVNotInstalled,NoOperatorDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet
    Status:                True
    Type:                  SubmarinerAgentDegraded
    Last Transition Time:  2022-01-31T15:02:07Z
    Message:               The nodes "ip-10-8-3-78.us-east-2.compute.internal" are labeled with "submariner.io/gateway"
    Reason:                SubmarinerGatewayNodesLabeled
    Status:                True
    Type:                  SubmarinerGatewayNodesLabeled
    Last Transition Time:  2022-01-31T15:01:38Z
    Message:               submariner add-on is available.
    Reason:                ManagedClusterAddOnLeaseUpdated
    Status:                True
    Type:                  Available
  Registrations:
    Signer Name:  kubernetes.io/kube-apiserver-client
    Subject:
      Groups:
        system:open-cluster-management:cluster:acm-nmanos-devcluster-a-aws:addon:submariner
        system:open-cluster-management:addon:submariner
        system:authenticated
      User:  system:open-cluster-management:cluster:acm-nmanos-devcluster-a-aws:addon:submariner:agent:submariner-addon-agent
Events:      <none>


**What you expected to happen**:
We need to support Submariner upgrade of Z streams (e.g. 0.11.0 --> 0.11.1).

**How to reproduce it (as minimally and precisely as possible)**:

https://qe-jenkins-csb-skynet.apps.ocp4.prod.psi.redhat.com/job/ACM-2.4.1-Submariner-0.11.0-AWSx2-SDN/94/Test-Report/

1) Deploy ACM 2.4.1 with Submariner 0.11.0

2) Verify that the Submariner agent is connected.


**Anything else we need to know?**:

 

**Environment**:

### ImageStream Tags (in namespace submariner-operator) ###

### lighthouse-agent-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-agent-rhel8
release=22
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-agent-rhel8/images/v0.11.0-22 
version=v0.11.0

### lighthouse-coredns-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/lighthouse-coredns-rhel8
release=22
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/lighthouse-coredns-rhel8/images/v0.11.0-22 
version=v0.11.0

### submariner-gateway-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-gateway-rhel8
release=21
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-gateway-rhel8/images/v0.11.0-21 
version=v0.11.0

### submariner-globalnet-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-globalnet-rhel8
release=21
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-globalnet-rhel8/images/v0.11.0-21 
version=v0.11.0

### submariner-networkplugin-syncer-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8
release=20
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-networkplugin-syncer-rhel8/images/v0.11.0-20 
version=v0.11.0

### submariner-operator-bundle:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-operator-bundle
release=54
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-operator-bundle/images/v0.11.0-54 
version=v0.11.0

### submariner-operator-bundle-index:v0.11.0 Image-Stream tag ###
name=openshift/ose-operator-registry
release=202112142229.p0.g5fc4c78.assembly.stream
url =https://access.redhat.com/containers/#/registry.access.redhat.com/openshift/ose-operator-registry/images/v4.9.0-202112142229.p0.g5fc4c78.assembly.stream 
version=v4.9.0

### submariner-rhel8-operator:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-rhel8-operator
release=51
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-rhel8-operator/images/v0.11.0-51 
version=v0.11.0

### submariner-route-agent-rhel8:v0.11.0 Image-Stream tag ###
name=rhacm2-tech-preview/submariner-route-agent-rhel8
release=21
url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2-tech-preview/submariner-route-agent-rhel8/images/v0.11.0-21 
version=v0.11.0

Comment 1 bot-tracker-sync 2022-03-16 22:40:00 UTC

G2Bsync 1066062805 comment 
 nyechiel Sun, 13 Mar 2022 09:43:03 UTC 
 G2Bsync This is not relevant anymore. ACM 2.4 now ships Submariner v0.11.2.

Note You need to log in before you can comment on or make changes to this bug.