Describe the issue: When running windows VMs with krbd the user may get bad crc/signature errors, see more: https://bugzilla.redhat.com/show_bug.cgi?id=1973317 We introduced a new flag for krbd rxbounce to solve this issue. The users will need to update the StorageClass they are using to enable it. Describe the task you were trying to accomplish: Document the new rxbounce options and how to enable it in ODF Suggestions for improvement: Document URL: Chapter/Section Number and Title: Product Version: Environment Details: Any other versions of this document that also needs this update: Additional information:
An example of a new StorageClass that has the "rxbounce" option set: $ oc get sc/windows-vms -oyaml allowVolumeExpansion: true apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: windows-vms parameters: clusterID: openshift-storage csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner csi.storage.k8s.io/controller-expand-secret-namespace: openshift-storage csi.storage.k8s.io/fstype: ext4 csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node csi.storage.k8s.io/node-stage-secret-namespace: openshift-storage csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner csi.storage.k8s.io/provisioner-secret-namespace: openshift-storage imageFeatures: layering imageFormat: "2" pool: ocs-storagecluster-cephblockpool mapOptions: "krbd:rxbounce" provisioner: openshift-storage.rbd.csi.ceph.com reclaimPolicy: Delete volumeBindingMode: Immediate Note that some options were stripped. The important part is the added "mapOptions:" value. The rest of the StorageClass is just a copy of the default "ocs-storagecluster-ceph-rbd" one.
Correcting myself, as the StorageClass in comment #2 is not functional for ODF-4.10, it needs a "mounter: rbd" option as well. The following StorageClass has successfully been tested: --- kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: windows-vms parameters: clusterID: openshift-storage csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner csi.storage.k8s.io/controller-expand-secret-namespace: openshift-storage csi.storage.k8s.io/fstype: ext4 csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node csi.storage.k8s.io/node-stage-secret-namespace: openshift-storage csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner csi.storage.k8s.io/provisioner-secret-namespace: openshift-storage imageFeatures: layering imageFormat: "2" pool: ocs-storagecluster-cephblockpool mounter: rbd mapOptions: "krbd:rxbounce" provisioner: openshift-storage.rbd.csi.ceph.com reclaimPolicy: Delete volumeBindingMode: Immediate allowVolumeExpansion: true --- Validation can be done by checking the logs from the csi-rbdplugin pod that runs on the node where a pod uses the PVC: $ oc -n openshift-storage logs -c csi-rbdplugin csi-rbdplugin-b4zh6 ... I0722 10:01:56.798742 198188 utils.go:191] ID: 10 Req-ID: 0001-0011-openshift-storage-0000000000000001-4cd4fad2-09a5-11ed-b174-0a580a800216 GRPC call: /csi.v1.Node/NodeStageVolume ... I0722 10:01:57.060675 198188 cephcmds.go:63] ID: 10 Req-ID: 0001-0011-openshift-storage-0000000000000001-4cd4fad2-09a5-11ed-b174-0a580a800216 command succeeded: rbd [--id csi-rbd-node -m 172.30.83.204:6789,172.30.129.151:6789,172.30.134.46:6789 --keyfile=***stripped*** map ocs-storagecluster-cephblockpool/csi-vol-4cd4fad2-09a5-11ed-b174-0a580a800216 --device-type krbd --options noudev --options rxbounce] I0722 10:01:57.060749 198188 nodeserver.go:397] ID: 10 Req-ID: 0001-0011-openshift-storage-0000000000000001-4cd4fad2-09a5-11ed-b174-0a580a800216 rbd image: ocs-storagecluster-cephblockpool/csi-vol-4cd4fad2-09a5-11ed-b174-0a580a800216 was successfully mapped at /dev/rbd0 ... This shows that the node has /dev/rbd0 mapped. It should also have the "rxbounce" option set: $ oc -n openshift-storage exec -c csi-rbdplugin csi-rbdplugin-b4zh6 -- cat /sys/block/rbd0/device/config_info 172.30.129.151:6789,172.30.134.46:6789,172.30.83.204:6789 name=csi-rbd-node,key=client.csi-rbd-node,rxbounce ocs-storagecluster-cephblockpool csi-vol-4cd4fad2-09a5-11ed-b174-0a580a800216 -
to further enhance this, the validation steps in comment 3, to make sure you've turned this on correctly, is a horrible and error prone user experience.