Description of problem: - Logging upgrade path 4.3 -> 4.4. - Elasticsearch Operator is not upgraded even though the subscription has been set to the 4.4 channel: ~~~ $ oc get subscriptions NAME PACKAGE SOURCE CHANNEL cluster-logging cluster-logging redhat-operators 4.4 [nnosenzo@localhost OCP4]$ oc get csv NAME DISPLAY VERSION REPLACES PHASE clusterlogging.4.4.0-202005121717 Cluster Logging 4.4.0-202005121717 Succeeded elasticsearch-operator.4.3.19-202005041055 Elasticsearch Operator 4.3.19-202005041055 Succeeded [...] ~~~ Version-Release number of selected component (if applicable): 4.4 How reproducible: 100% Steps to Reproduce: 1. Deploy logging in 4.3 2. Upgrade cluster to latest 4.4 3. Follow the steps in [0] to upgrade Logging [0] https://docs.openshift.com/container-platform/4.4/logging/cluster-logging-upgrading.html Actual results: EO remains in version elasticsearch-operator.4.3.19 Expected results: EO should upgrade to 4.4 Additional info:
Could you provide the output. #oc version #oc get packagemanifest elasticsearch-operator -o json #oc get sub elasticsearch-operator -o json -n openshift-operators-redhat
Created attachment 1692962 [details] deployment info Just fyi, service mesh was already deployed on this cluster, so the EO Subscription resides within the openshift-operators namespace.
Moving to OLM team to investigate as far as I understand the logging csv et al is correct
Could you please provide more information about this cluster and the upgrade? - What is current OCP version? (z-stream) - Logs of the catalog-operator pod in openshift-operator-lifecycle-manager - Logs of the olm-operator pod in openshift-operator-lifecycle-manager - Status of the CSV objects listed (elasticsearch and cluster-logging)
Created attachment 1694345 [details] olm-operator logs - What is current OCP version? (z-stream) $ oc version Client Version: 4.5.0-0.nightly-2020-05-13-092431 Server Version: 4.4.4 Kubernetes Version: v1.17.1 - Logs of the catalog-operator pod in openshift-operator-lifecycle-manager Attached. - Logs of the olm-operator pod in openshift-operator-lifecycle-manager Attached. - Status of the CSV objects listed (elasticsearch and cluster-logging) Attached, as well as the objects manifest.
Created attachment 1694346 [details] catalog operator logs
Created attachment 1694347 [details] csv info
Hi Nicolas, Could you also grab the CSVs and InstallPlans from the openshift-operators namespace? Thanks
Created attachment 1694469 [details] CSV and InstallPlans list Hi Nick, Info attached.
The logs and objects provided do not indicate any errors. From the first report: $ oc get subscriptions NAME PACKAGE SOURCE CHANNEL cluster-logging cluster-logging redhat-operators 4.4 There should also be a subscription for elasticsearch. Can you find that subscription and ensure that the channel has been set to 4.4 there as well? If that does not appear to be the issue, can you share the full contents of all subscriptions on the cluster?
Evan, as mentioned in the issue description the EO subscription was set to the correct channel, even though the Operator was not upgraded: $ oc get subscriptions --all-namespaces NAMESPACE NAME PACKAGE SOURCE CHANNEL openshift-logging cluster-logging cluster-logging redhat-operators 4.4 openshift-operators elasticsearch-operator elasticsearch-operator redhat-operators 4.4 openshift-operators jaeger-product jaeger-product redhat-operators stable openshift-operators kiali-ossm kiali-ossm redhat-operators stable openshift-operators servicemeshoperator servicemeshoperator redhat-operators stable Attaching all the subscription manifests.
Created attachment 1696673 [details] subscriptions manifests May the below be the culprit ? conditions: - lastTransitionTime: "2020-05-21T10:59:07Z" message: all available catalogsources are healthy reason: AllCatalogSourcesHealthy status: "False" type: CatalogSourcesUnhealthy