Description of problem: $ subctl join ./broker-info.subm --cable-driver libreswan --ikeport 501 --nattport 4501 --version devel --health-check --pod-debug --ipsec-debug https://api.nmanos-cluster-a.devcluster.openshift.com:6443 * There are 1 labeled nodes in the cluster: • Discovering network details ... - ip-10-166-56-232.us-west-1.compute.internal ✓ Discovering network details Discovered network details: Network plugin: OpenShiftSDN Service CIDRs: [100.96.0.0/16] Cluster CIDRs: [10.252.0.0/14] • Discovering multi cluster details ... • Validating Globalnet configurations ... ✓ Validating Globalnet configurations • Assigning Globalnet IPs ... ✓ Assigning Globalnet IPs ✓ Allocated GlobalCIDR: 169.254.0.0/19 ✓ Discovering multi cluster details • Deploying the Submariner operator ... ✓ Deploying the Submariner operator ✓ Created operator CRDs ✓ Created operator namespace: submariner-operator ✓ Created operator service account and role ✓ Updated the privileged SCC ✓ Created lighthouse service account and role ✓ Updated the privileged SCC ✓ Created Lighthouse service accounts and roles ✓ Deployed the operator successfully • Creating SA for cluster ... ✓ Creating SA for cluster • Deploying Submariner ... ✓ Deploying Submariner ✓ Submariner is up and running $ oc get pods -n submariner-operator NAME READY STATUS RESTARTS AGE submariner-gateway-hgqrf 1/1 Running 0 72m submariner-globalnet-sjmff 1/1 Running 0 72m submariner-operator-847667df9b-5m6v2 1/1 Running 0 73m submariner-routeagent-2sk7f 1/1 Running 0 72m submariner-routeagent-2xz4v 1/1 Running 0 72m submariner-routeagent-79sg8 1/1 Running 0 72m submariner-routeagent-gqswg 1/1 Running 0 72m submariner-routeagent-ns9ld 1/1 Running 0 72m submariner-routeagent-vmxv4 1/1 Running 0 72m submariner-routeagent-xrgpw 1/1 Running 0 72m Version-Release number of selected component (if applicable): OCP cluster A (AWS): Client Version: 4.6.15 Server Version: 4.6.15 Kubernetes Version: v1.19.0+1833054 OCP cluster B (OSP): Client Version: 4.6.15 Server Version: 4.4.3 Kubernetes Version: v1.17.1 Submariner: subctl version: subctl-devel-1-gaad9f26 Showing information for cluster "default-cl1": COMPONENT REPOSITORY VERSION submariner devel submariner-operator quay.io/submariner devel How reproducible: Sometimes Steps to Reproduce: https://qe-jenkins-csb-skynet.cloud.paas.psi.redhat.com/job/Submariner-OSP-AWS-Globalnet/89/Test-Report/ (pod log collection at last step) Actual results: subctl show all Showing information for cluster "nmanos-cluster-a": Discovered network details: Network plugin: OpenShiftSDN Service CIDRs: [100.96.0.0/16] Cluster CIDRs: [10.252.0.0/14] Global CIDR: 169.254.0.0/19 No endpoints found subctl version: subctl-devel-1-gaad9f26 #In Pod logs I see: ERROR controller-runtime.controller Reconciler error {"controller": "submariner", "name": "submariner", "namespace": "submariner-operator", "error": "Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again"} 2021-02-02T06:44:39.500271665Z github.com/go-logr/zapr.(*zapLogger).Error # And: 2021-02-02T06:42:08.049198401Z E0202 06:42:08.049151 1 ipam.go:243] error retrieving submariner-ipam-controller object openshift-marketplace/certified-operators-5zzk5: pods "certified-operators-5zzk5" not found 2021-02-02T06:42:08.246907407Z E0202 06:42:08.246864 1 ipam.go:243] error retrieving submariner-ipam-controller object openshift-marketplace/redhat-operators-vtxlm: pods "redhat-operators-vtxlm" not found
The problem is reproduced: https://qe-jenkins-csb-skynet.cloud.paas.psi.redhat.com/job/Submariner-OSP-AWS-Globalnet/90/Test-Report/ On the submariner-operator pod I see: 2021-02-03T06:41:44.114477663Z 2021-02-03T06:41:44.114Z INFO controller_submariner Reconciling Submariner {"Request.Namespace": "submariner-operator", "Request.Name": "submariner"} 2021-02-03T06:41:44.123058226Z 2021-02-03T06:41:44.122Z INFO controller_submariner Using detected CIDR {"type": "Cluster", "CIDR": "10.252.0.0/14"} 2021-02-03T06:41:44.123058226Z 2021-02-03T06:41:44.122Z INFO controller_submariner Using detected CIDR {"type": "Service", "CIDR": "100.96.0.0/16"} 2021-02-03T06:41:44.130022833Z 2021-02-03T06:41:44.129Z INFO controller_submariner Updated existing DaemonSet {"Request.Namespace": "submariner-operator", "Request.Name": "submariner", "DaemonSet.Namespace": "submariner-operator", "DaemonSet.Name": "submariner-gateway"} 2021-02-03T06:41:44.136710234Z 2021-02-03T06:41:44.136Z INFO controller_submariner Updated existing Service {"Request.Namespace": "submariner-operator", "Request.Name": "submariner", "Service.Namespace": "submariner-operator", "Service.Name": "submariner-engine-metrics"} 2021-02-03T06:41:45.361800195Z 2021-02-03T06:41:45.361Z INFO controller_submariner Could not create ServiceMonitor object {"error": "servicemonitors.monitoring.coreos.com \"submariner-globalnet-metrics\" already exists"} 2021-02-03T06:41:45.372504317Z 2021-02-03T06:41:45.372Z ERROR controller_submariner failed to update the Submariner status {"Request.Namespace": "submariner-operator", "Request.Name": "submariner", "error": "Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again"} 2021-02-03T06:41:45.372504317Z github.com/go-logr/zapr.(*zapLogger).Error 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128 2021-02-03T06:41:45.372504317Z github.com/submariner-io/submariner-operator/controllers/submariner.(*SubmarinerReconciler).Reconcile 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/controllers/submariner/submariner_controller.go:218 2021-02-03T06:41:45.372504317Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:246 2021-02-03T06:41:45.372504317Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222 2021-02-03T06:41:45.372504317Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201 2021-02-03T06:41:45.372504317Z k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 2021-02-03T06:41:45.372504317Z k8s.io/apimachinery/pkg/util/wait.JitterUntil 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 2021-02-03T06:41:45.372504317Z k8s.io/apimachinery/pkg/util/wait.Until 2021-02-03T06:41:45.372504317Z /go/src/github.com/submariner-io/submariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 2021-02-03T06:41:45.372504317Z 2021-02-03T06:41:45.372Z ERROR controller-runtime.controller Reconciler error {"controller": "submariner-controller", "name": "submariner", "namespace": "submariner-operator", "error": "Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again"}
https://github.com/submariner-io/submariner-operator/pull/1058 https://github.com/submariner-io/submariner-operator/pull/1062
YTMP3's cross-platform flexibility is one of its best qualities; https://ytmp3.ad/ it requires no installation or registration and functions flawlessly on Windows, macOS, Android, and iOS devices.