Bug 2125204

Summary: Must-Gather, ocs-storagecluster-cephnfs-builtin-pool, command terminated with exit code 2
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Oded <oviner>
Component: must-gatherAssignee: yati padia <ypadia>
Status: ASSIGNED --- QA Contact: Prasad Desala <tdesala>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.11CC: muagarwa, odf-bz-bot, ypadia
Target Milestone: ---Flags: muagarwa: needinfo? (oviner)
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Oded 2022-09-08 10:33:57 UTC 
Description of problem (please be detailed as possible and provide log
snippests):

The ocs-storagecluster-cephnfs-builtin-pool does not exist

rbd: error opening pool 'ocs-storagecluster-cephnfs-builtin-pool': (2) No such file or directory
command terminated with exit code 2

$ oc get cephblockpools.ceph.rook.io -n openshift-storage
NAME                               PHASE
ocs-storagecluster-cephblockpool   Ready

$ oc get cephfilesystems.ceph.rook.io 
NAME                                ACTIVEMDS   AGE     PHASE
ocs-storagecluster-cephfilesystem   1           2d14h   Ready


Version of all relevant components (if applicable):
ODF Version: 4.11.1-4
OCP version: 4.11.0-0.nightly-2022-08-26-162248
Provider: Vmware


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?


Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
ceph/logs/gather-rbd-mirror-pool-info-ocs-storagecluster-cephnfs-builtin-pool-debug.log:
rbd: error opening pool 'ocs-storagecluster-cephnfs-builtin-pool': (2) No such file or directory
command terminated with exit code 2

ceph/logs/gather-rbd-mirror-pool-status-ocs-storagecluster-cephnfs-builtin-pool-debug.log:
rbd: error opening pool 'ocs-storagecluster-cephnfs-builtin-pool': (2) No such file or directory
command terminated with exit code 2


ceph/logs/gather-rbd-trash-ls-ocs-storagecluster-cephnfs-builtin-pool-json-debug.log:
rbd: error opening pool 'ocs-storagecluster-cephnfs-builtin-pool': (2) No such file or directory
command terminated with exit code 2

http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/j-035vukv11cs33-t1/j-035vukv11cs33-t1_20220830T073853/logs/failed_testcase_ocs_logs_1661847973/test_encrypted_rbd_block_pvc_snapshot%5bv2-vault-False%5d_ocs_logs/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-2126b45f5b577efcf041c57d94d368fd173161ba9d27ea8cf8e82d5c776d39ee/ceph/logs/gather-rbd-trash-ls-ocs-storagecluster-cephnfs-builtin-pool-json-debug.log

MG:
http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/j-035vukv11cs33-t1/j-035vukv11cs33-t1_20220830T073853/logs/failed_testcase_ocs_logs_1661847973/test_encrypted_rbd_block_pvc_snapshot%5bv2-vault-False%5d_ocs_logs/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-2126b45f5b577efcf041c57d94d368fd173161ba9d27ea8cf8e82d5c776d39ee/

Job link:
https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster-prod/5202/


Actual results:


Expected results:


Additional info:
Comment 3 Oded 2023-01-05 15:36:24 UTC 
This issue reconstructed on ODF4.12.

/ceph/logs/gather-rbd-mirror-pool-info-ocs-storagecluster-cephnfs-builtin-pool-debug.log 

rbd: error opening pool 'ocs-storagecluster-cephnfs-builtin-pool': (2) No such file or directory
command terminated with exit code 2

https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster-prod/6466/

SetUp:
OCP Version:4.12.0-0.nightly-2022-12-27-111646
ODF Version:4.12.0-152
Comment 5 Oded 2023-01-06 07:04:12 UTC 
http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/j-222ai3c333-t1/j-222ai3c333-t1_20230102T074505/logs/failed_testcase_ocs_logs_1672649331/test_validate_ceph_config_values_in_rook_config_override_ocs_logs/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-b9dc9da44e281efdb293d80324c822377d06dc5b29289ebb31c8b6db37305363/ceph/logs/gather-rbd-mirror-pool-info-ocs-storagecluster-cephnfs-builtin-pool-debug.log
Comment 8 Mudit Agarwal 2023-08-16 10:03:54 UTC 
This is the blockpool for NFS, looks expected. Did you enable NFS anytime on these clusters?

https://github.com/red-hat-storage/ocs-operator/blob/290d0fc92d084a7ebe89248f95a7e8a7499f4133/controllers/storagecluster/cephblockpools.go#L106