Description of problem (please be detailed as possible and provide log
snippests):
-----------------------------------------------------------------------
Debugging of a failed NSFS regression test on OCS-CI reveals that when we create a NSFS Namespacestore, it remains stuck in Rejected with the following error at the YAML:
```
...
...
status:
  conditions:
  - lastHeartbeatTime: "2024-06-03T12:17:14Z"
    lastTransitionTime: "2024-06-03T12:17:14Z"
    message: NamespaceStorePhaseRejected
    reason: 'Namespace store mode: STORAGE_NOT_EXIST'
    status: Unknown
    type: Available
```

However, after pausing the test at this phase I was still able to create the required MCG account that uses the namespacestore for its default buckets, and then create and write to/from the said bucket. It appears the Namespacestore is still functional despite the error message.


Version of all relevant components (if applicable):
-----------------------------------------------------------------------
OCP: 4.16.0-0.nightly-2024-06-02-000851
ODF: 4.16.0-113
ceph: 18.2.1-188.el9cp (b1ae9c989e2f41dcfec0e680c11d1d9465b1db0e) reef (stable)
rook: v4.16.0-0.a2396a5186cc038b22154e857e0f7865e709d06a
noobaa core: 5.16.0-03db21f
noobaa operator: 5.16.0-705652b55ddaabc6bbdf16cb648c4f9a72345cf1


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
-----------------------------------------------------------------------
It fails the regression test, but otherwise the feature still seems functional.


Is there any workaround available to the best of your knowledge?
-----------------------------------------------------------------------
No


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
-----------------------------------------------------------------------------
3


Can this issue reproducible?
-----------------------------------------------------------------------------
Yep


Can this issue reproduce from the UI?
-----------------------------------------------------------------------------
N/A


If this is a regression, please provide more details to justify this:
-----------------------------------------------------------------------------
The OCS-CI tests in https://github.com/red-hat-storage/ocs-ci/blob/master/tests/functional/object/mcg/test_nsfs.py have been passing before 4.16, and are now failing because the Namespacestore appears unfunctional.


Steps to Reproduce:
------------------------------------------------------------------------------
1. Create an RWX CephFS PVC in the openshift-storage project:


2. Create a deployment that mounts PVC at mountpath "/nsfs":
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: my-pvc
 namespace: openshift-storage
spec:
 accessModes:
 - ReadWriteMany
 resources:
   requests:
     storage: 25Gi
 storageClassName: ocs-storagecluster-cephfs
``` 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nsfs-interface
  namespace: openshift-storage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nsfs-interface
  template:
    metadata:
      labels:
        app: nsfs-interface
    spec:
      containers:
      - command:
        - /bin/sh
        image: registry.access.redhat.com/ubi8/ubi:8.5-214
        imagePullPolicy: IfNotPresent
        name: ubi8
        stdin: true
        tty: true
        volumeMounts:
        - mountPath: /nsfs
          name: my-pvc
      securityContext:
        runAsUser: 1000620000
      volumes:
      - name: my-pvc
        persistentVolumeClaim:
          claimName: my-pvc
```
3. Create an NSFS Namespacestore:
```
apiVersion: noobaa.io/v1alpha1
kind: NamespaceStore
metadata:
  name: nsfs-nss
  namespace: openshift-storage
  labels:
    app: noobaa
  finalizers:
    - noobaa.io/finalizer
spec:
  type: nsfs
  nsfs:
    pvcName: my-pvc
    subpath: "nsfs"
```
4. Create a new MCG account that uses the NSFS NSS by default:
```
noobaa account create nsfs-account --allow_bucket_create=True --default_resource nsfs-nss --gid 1234 --new_buckets_path / --nsfs_account_config=True --nsfs_only=False --uid 5678  -n openshift-storage
```

5. Use its credentials to create a new NSFS bucket on the NSS via S3 and make some I.O against it:
```
ACC_NAME=nsfs-account
S3_ENDPOINT=https://$(oc get route s3 -n openshift-storage -o json | jq -r '.status.ingress[0].host')
S3_ACCESS_KEY=$(kubectl get secret noobaa-account-$ACC_NAME -n openshift-storage -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d')
S3_SECRET_KEY=$(kubectl get secret noobaa-account-$ACC_NAME -n openshift-storage -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d')

alias my_s3="AWS_ACCESS_KEY_ID=$S3_ACCESS_KEY AWS_SECRET_ACCESS_KEY=$S3_SECRET_KEY aws --endpoint $S3_ENDPOINT --no-verify-ssl s3 

my_s3 mb s3://nsfs-bucket --region=us-east-2

my_s3 sync test_objects/ s3://nsfs-bucket/

my_s3 ls s3://nsfs-bucket/

```


Actual results:
---------------------------------------------------------------
The Namespacestore appears Rejected after its creation, but the bucket creation at the last step and the I.O against it works in spite of the status of the NSS.


Expected results:
---------------------------------------------------------------
The NSFS Namespacestore should remain in the Ready phase shortly after its creation, and the bucket creation and I.O against it at the last step should work.


Additional info:
---------------------------------------------------------------
- A live cluster with the issue should be available here until the 5th of June: https://url.corp.redhat.com/c406c3c

- Due to an automation issue, the OCS-CI test requires the following fix that hasn't been merged yet in order to reach the step at the test in which we see the NSS remains Rejected: https://github.com/red-hat-storage/ocs-ci/pull/9892