Description of problem: Currently the RBDs created with the RHS-C look like this: [root@rhcs1 ceph]# rbd info testRBD rbd image 'testRBD': size 10240 MB in 2560 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.105b74b0dc51 format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten <---- flags: Because of https://access.redhat.com/solutions/2258301 this will prevent rbds to be mapped with these features activated until the GA of RHEL7.3 (in Autumn) Suggestion: Create new RBDs without those features to make them mappable in RHEL7.2. In a non-RHS-C setup the default RBD looks like this: [root@mon ceph]# rbd create testRBD1 --size 10 [root@mon ceph]# rbd info testRBD1 rbd image 'testRBD1': size 10240 kB in 3 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.101f74b0dc51 format: 2 features: layering flags: [root@mon ceph]# ceph --version ceph version 0.94.5-9.el7cp (deef183a81111fa5e128ec88c90a32c9587c615d) Version-Release number of selected component (if applicable): Recent
Same thing tested n Jewel Ceph: [root@mon vagrant]# rbd create testRBD1 --size 10 [root@mon vagrant]# rbd info testRBD1 rbd image 'testRBD1': size 10240 kB in 3 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.101f2ae8944a format: 2 features: layering flags: [root@mon vagrant]# ceph --version ceph version 10.2.0-1.el7cp (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
Calamari provides no options to unset this hile pool is created. so moving to calamari
ceph-installer creates the cluster in such a way that these features enabled by default
These are set using the key rbd_default_features in ceph.conf The list of features that don't work till RHEL7.3 in kRBD are: object map, deep flatten, journaling, exclusive lock, fast diff These work just fine in userspace RBD. My take on this is we should include docs that explain what action to take if you are a kRBD user. Chris what do you think about that?
kRBD users are few. We're not going to make code changes in 2.0 for this. Ken would you please add an entry in the 2.0 docs/release notes based on Comment 7?
I would try to design it in a way that it 'just works' for the regular user - meaning that I would deactivate the features unless there is a good reason to keep them active. Once 7.3 is released, we could bring the features back as a small update? This is all based on the assumption that these features are not essential for something important... If the features will stay activated, I would vote for a better help message that clearly tells the user that kRBD can't currently map this RBD (if that is somehow possible?)
This doc text should really mention that these features work fine with userspace RBD and this is only a limitation in krbd (kernel rbd)
Jason, Do we have an updated kRBD module shipped in RHEL7.3 that supports object map, deep flatten, journaling, exclusive lock, fast diff? i.e. can we close this BZ as works in current-release? Also in future is there something sane we can do with the value of rbd_default_features? cheers, G
The RHEL 7.3 krbd module supports layering and exclusive-lock, but still doesn't support object-map, fast-diff, deep-flatten, and journaling (not enabled by default). For the users of krbd, they can set "rbd default features = 1" in the ceph.conf file where they create images. Alternatively, a user can also run "rbd feature disable <image-spec> exclusive-lock,object-map,fast-diff,deep-flatten".
@Bara: lgtm
Bara, Looks good except one change I suggest: In that section it says "The ceph-installer utility enables the following features by default when creating a Ceph Block Device:" It's true that ceph-installer configures the cluster that way e.g. the defaults are "object-map, fast-diff, deep-flatten" It is misleading to say that ceph-installer creates the block device. For clarity I would say this instead "The default configuration of Ceph includes the following RBD features" cheers, G
Jason, What I'm hearing is that we'd need to take a lowest-common-denominator approach here and just set "rbd default features = 1" in the ceph.conf since ceph-installer(ansible) has no way of knowing what clients will be used or what RBDs will be created My concern is that my understanding is kRBD users are a minority when compared to user-space RBD. That means that this change would hurt the 80%(made up number) use case. What other options do we have here? Could the kRBD client warn that certain block devices have unsupported features? Can they be changed after the block-device is created? thanks, G
Greg, I believe the verbiage added by Bara in the doc answers your question re: can they be changed after it's created (yup). Since this config suggestion is only in the section related to krbd block devices, I would also think it would appear to be applicable to those users only.