Bug 1822213 - local-storage-local-diskmaker and local-storage-local-provisioner daemonsets do not upgrade after the CSV is upgraded to a new version
Summary: local-storage-local-diskmaker and local-storage-local-provisioner daemonsets ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.3.z
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.5.0
Assignee: Christian Huffman
QA Contact: Qin Ping
URL:
Whiteboard:
Depends On:
Blocks: 1845148 1845155
TreeView+ depends on / blocked
 
Reported: 2020-04-08 13:58 UTC by Qin Ping
Modified: 2020-07-13 17:26 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: When upgrading the Local Storage Operator the associated diskmaker and provisioner pods are not updated unless the LocalVolume resource is also modified. Consequence: The diskmaker and provisioner pods could be outdated, despite the Local Storage Operator being updated. Fix: Included the DaemonSet's hash in an annotation that is examined. If the hash does not match, then the pods are deployed. Result: The diskmaker and provisioner pods are now successfully updated when the Local Storage Operator is updated.
Clone Of:
: 1845148 (view as bug list)
Environment:
Last Closed: 2020-07-13 17:26:16 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift local-storage-operator pull 105 0 None closed Bug 1822213: Include version in diskmaker and provisioner configmap 2020-10-27 05:31:17 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:26:47 UTC

Description Qin Ping 2020-04-08 13:58:21 UTC 
Description of problem:
local-storage-local-diskmaker and local-storage-local-provisioner daemonsets do not upgrade after the CSV is upgraded to a new version

Version-Release number of selected component (if applicable):
upgrade from 4.3.10 to 4.4.0-0.nightly-2020-04-07-130324

How reproducible:
always

Steps to Reproduce:
1. Before OCP upgrade, install the local storage operator from channel 4.3
2. After the OCP upgraded to 4.4.0-0.nightly-2020-04-07-130324, update the local storage operator subscription's channel to 4.4
3. check the local storage operator

Actual results:
csv is upgraded to the 4.4 version, but the local-storage-local-diskmaker and local-storage-local-provisioner daemonsets do not upgrade.

Expected results:
local-storage-local-diskmaker and local-storage-local-provisioner daemonsets upgrade to the 4.4 too.

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

$ oc get csv -n local-storage|grep local-storage
local-storage-operator.4.4.0-202004040654   Local Storage            4.4.0-202004040654   local-storage-operator.4.3.10-202003311428   Succeeded
$ oc get pod -n local-storage
NAME                                      READY   STATUS    RESTARTS   AGE
hello-openshift-1-jm2vb                   1/1     Running   0          155m
local-storage-local-diskmaker-99kj6       1/1     Running   0          3h57m
local-storage-local-diskmaker-gdmsq       1/1     Running   0          3h57m
local-storage-local-diskmaker-nwrpf       1/1     Running   0          3h57m
local-storage-local-diskmaker-xqzbt       1/1     Running   0          3h57m
local-storage-local-provisioner-6bztx     1/1     Running   0          3h57m
local-storage-local-provisioner-985p7     1/1     Running   0          3h57m
local-storage-local-provisioner-fftx7     1/1     Running   0          3h57m
local-storage-local-provisioner-vv2cf     1/1     Running   0          3h57m
local-storage-operator-6c64fff467-97jrg   1/1     Running   0          152m
$ oc get pod local-storage-operator-6c64fff467-97jrg -n local-storage -oyaml|grep "image:"
    image: quay.io/openshift-qe-optional-operators/ose-local-storage-operator@sha256:07163b0e862bd6ee36809c0d43b61c1c494c9111e8cbcbe0708e19e8e8a1293f
    image: quay.io/openshift-qe-optional-operators/ose-local-storage-operator@sha256:07163b0e862bd6ee36809c0d43b61c1c494c9111e8cbcbe0708e19e8e8a1293f
$ cat manifests/4.4/local-storage-operator.v4.4.0.clusterserviceversion.yaml |grep diskmaker
                        value: image-registry.openshift-image-registry.svc:5000/openshift/ose-local-storage-diskmaker:v4.4.0-202004040654
$ oc get ds local-storage-local-diskmaker -n local-storage -oyaml|grep image
        image: registry.redhat.io/openshift4/ose-local-storage-diskmaker@sha256:2881947da90a6d33c0f7d5c0c5377a6a58dd025e0fb5ce3a31bb15da34047d69
Comment 1 Christian Huffman 2020-04-13 20:33:04 UTC 
This issue is because the ConfigMaps for the diskmaker and provisioner are not modified when the LSO is updated. They currently are only modified when the LocalVolume resource changes, which does not occur during an operator update.

I've submitted a PR [1] which includes the controller version in these ConfigMaps, thereby ensuring that they will be redeployed whenever the Operator is updated.

[1] https://github.com/openshift/local-storage-operator/pull/105
Comment 4 Qin Ping 2020-05-25 13:59:56 UTC 
Verified with: local-storage-operator.4.5.0-202005221637

$ oc get csv -n local-storage
NAME                                        DISPLAY         VERSION              REPLACES                                     PHASE
local-storage-operator.4.5.0-202005221637   Local Storage   4.5.0-202005221637   local-storage-operator.4.3.22-202005201238   Succeeded
Comment 5 errata-xmlrpc 2020-07-13 17:26:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409
Note You need to log in before you can comment on or make changes to this bug.