Bug 1988982

Summary: [IBM Z]: Reinstallation of OCS (4.8.0-175.ci) seems to have leftover data from old install leaving dangling noobaa pods after ocs-operator installation
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Sravika <sbalusu>
Component: Multi-Cloud Object GatewayAssignee: Nimrod Becker <nbecker>
Status: CLOSED WORKSFORME QA Contact: Elad <ebenahar>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.8CC: etamir, kramdoss, muagarwa, nbecker, ocs-bugs, odf-bz-bot
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-31 10:02:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
namespacestore_ocsoperator
none
backingstore_ocsoperator none

Description Sravika 2021-08-02 08:29:31 UTC 
Description of problem:

Uninstall OCS completely and install OCS operator 4.8.0-175.ci. After the installation of ocs-operator and before the storage cluster creation, there are 2 pending noobaa pods in openshift-storage namespace. Those 2 pods status changed to "Running" after the storage cluster creation.

# oc get po -n openshift-storage
NAME                                    READY   STATUS    RESTARTS   AGE
noobaa-core-0                           0/1     Pending   0          24m
noobaa-db-pg-0                          0/1     Pending   0          24m
noobaa-operator-5bd86b4d59-z2bjr        1/1     Running   0          24m
ocs-metrics-exporter-7dd5666b6b-pxk65   1/1     Running   0          24m
ocs-operator-777b6f46dc-jqs9z           1/1     Running   0          24m
rook-ceph-operator-7dd45bd578-qbrwm     1/1     Running   0          24m
#

After the ocs-operator installation, in the openshift-console the ocs-operator NamespaceStore showed data from the old installation hanging in deleting state, please find screenshot attached.

After the installation of storage cluster, a backingstore(pvbackebsyaml) in "Rejected" state is still visible from the old installation, attached screenshot.   

I have followed the uninstall guide to uninstall OCS which seems to have not entirely cleaned up the noobaa resources as the reinstallation still shows the old data.


Version-Release number of selected component (if applicable):

OCP: 4.8.0-rc.3
LSO: 4.8.0-202106291913
Ocs-operator: 4.8.0-175.ci

How reproducible:


Steps to Reproduce:
1. Install OCP, OCS
2. Add storage resources for hybrid or multicloud
3. Uninstall OCS following the instruction guide
4. Install OCS on the same OCP cluster

Actual results:

Left over data from the old installation found during reinstallation of OCS operator


Expected results:

No leftover data from noobaa 


Additional info:

https://drive.google.com/file/d/1u7UbwyJuKjeZZVkS1j3DeaSSr4wH6MdQ/view?usp=sharing
Comment 2 Sravika 2021-08-02 08:30:08 UTC 
Created attachment 1810017 [details]
namespacestore_ocsoperator
Comment 3 Sravika 2021-08-02 08:30:41 UTC 
Created attachment 1810018 [details]
backingstore_ocsoperator
Comment 4 Nimrod Becker 2021-08-02 11:30:48 UTC 
When the backingstore is being deleted, it removes the data that was written on it.
How much time passed between the uninstall and the 2nd installation?
Comment 5 Sravika 2021-08-02 13:25:18 UTC 
@nbecker : I can see a needinfo flag set on me, however I cannot see any comment. Could you please make the comment public ?
Comment 6 Mudit Agarwal 2021-08-03 06:34:36 UTC 
(In reply to Sravika from comment #5)
> @nbecker : I can see a needinfo flag set on me, however I cannot
> see any comment. Could you please make the comment public ?

Done, please check.

Moving out of 4.8 as it is not a blocker.
Comment 7 Sravika 2021-08-03 11:19:11 UTC 
@nbecker: I did not keep track of the exact time between uninstall and second install, but I think it should more than 15 minutes.
Comment 11 Sravika 2022-03-28 09:29:34 UTC 
@nbecker : There is a "need info" flag on me but I cannot see your comment. Could you please make it public?
Comment 12 Nimrod Becker 2022-03-28 10:17:20 UTC 
Does the issue still happening ?
Can we get reopro details
Comment 13 Nimrod Becker 2022-05-25 17:48:31 UTC 
Comment 12
Comment 14 Yaniv Kaul 2022-05-31 10:02:11 UTC 
Closing, please re-open if it still happens on latest version.
Comment 15 Sravika 2022-10-10 08:04:12 UTC 
works fine on latest version