Document URL: https://access.redhat.com/documentation/en/openshift-enterprise/version-3.1/architecture/#persistent-volume-claims Section Number and Name: 4.5.4. Persistent Volume Claims Describe the issue: Several templates provided with OSE have multiple AccessModes specified in the PVC. This is fine as this simply allows the claim to bind to a PV of either mode. The problem is that the documentation does not describe what happens when PVs of BOTH AccessModes are present. Does it select the PV that has the AccessMode that is listed first in the claim? Or does it go by something else? Suggestions for improvement: Determine how the claim determines which PV to bind to.
@Mark-Do you have any insight into this? Thanks!
update doc here: https://github.com/openshift/openshift-docs/pull/1665
follow-up PR with minor edits: https://github.com/openshift/openshift-docs/pull/1683
Commits pushed to master at https://github.com/openshift/openshift-docs https://github.com/openshift/openshift-docs/commit/55dccec546adeb2c3b6591440fd586a3b761f22a Bug 1304489, edited AccessModes content https://github.com/openshift/openshift-docs/commit/cd0545c4a43c13e4dd9d8b3f654613012011da60 Merge pull request #1683 from ahardin-rh/access-modes Bug 1304489, edited AccessModes content
Content is published: https://access.redhat.com/documentation/en/openshift-enterprise/3.1/architecture/chapter-4-additional-concepts#pv-access-modes https://docs.openshift.com/enterprise/3.1/architecture/additional_concepts/storage.html#pv-access-modes
The information that you were able to provide is very helpful, however, there is another bit of clarification that would be beneficial and the main reason I opened the bug was that there was no information on how a PVC determines what PV to select and this is still true. In the example you provide, "if a claim requests RWO, but the only volume available was an NFS PV (RWO+ROX+RWX)" what if there were two NFS PVs and an iSCSI PV, NFSPV1 had RWO+ROX+RWX, for some reason NFSPV2 only had RWX+RWO, and ISCSIPV had RWO+ROX which of those would the PVC bind to? Sorry to re-open the bug, I suppose I should've clarified that detail in the beggining, I'm sorry.
@Eric, No problem! @Mark, Can you please help me fill in the gap? Thanks!
@Eric, The only 2 matching criteria are access modes and size. The modes must match or contain more modes than you requested. The size must be greater than or equal to what is expected. If 2 types of volumes (NFS and iscsci) both have the same set of access modes, then *either* of them will match a claim w/ those modes. There is no ordering between types of volumes. In your example, a claim w/ all 3 modes would only match the 2 volumes with all 3 modes. If the claim had only RWX, then it could match *any* of the volumes, since all have RWX. Overall, though, if you've got 2 types of volumes with the same access modes, there is no way to choose one type over another. That'll be achievable with the PersistentVolumeSelector feature currently in development. Please let me know if this is helpful or if you'd like more info. I can appreciate that it's complicated and that an explanation doesn't necessarily make it any clearer ...
I followed up with Mark in IRC to the following effect: Eric: I do have one last question. I understand that we cannot specify which one to choose, in the situation with multiple PV's that all meet the criteria for the PVC, but how does the system decide? There has to be something that OSE uses to decide which one to choose, even if we can't impact that decision at the moment. Mark: All volumes with the same modes are grouped and then sorted by size (smallest to largest). The binder gets the group with matching modes and simply iterates over each (in size order) until one size matches. This answers my questions @Ashley!
New PR created: https://github.com/openshift/openshift-docs/pull/1750 @ Mark & Eric, please review. Thanks!
Things look good to me. Unfortunately I realized I have one last question for @Mark, and I genuinely believe this is the last one. If there are two PV's that match in size and AccessModes, what does the PVC make the decision based on? To word it differently, if size and AccessModes are the same for two PVs, when they are sorted along with the other PVs, are they going to be sorted by name at that point or some other feature? Clearly one has to go "higher" on the priority of selection.
There is no sorting beyond size/capacity. However the sort algorithm works in Go is how the PVs are sorted in the list. All in the list will have the same modes. There is no deterministic way to know how NFS and ISCSI sorted in the list. If both volumes are 5gb, there is no telling which one is "up next" to match the 5gb claim.
Good to know. Thank you for the information @Mark, and the docs @Ashley!
Thanks, guys! Should anything else be added to the PR?
I think that is everything!
Commits pushed to master at https://github.com/openshift/openshift-docs https://github.com/openshift/openshift-docs/commit/c4969a5deaa9b037effc1ec2fae8b9e9e2399e60 Bug 1304489, clarified how a PVC determines what PV to select in Access Modes section https://github.com/openshift/openshift-docs/commit/dadb815d936f5e3189495c30cd838c09551ea9cd Merge pull request #1750 from ahardin-rh/access-modes Bug 1304489, clarified how a PVC determines what PV to select
Content is published: https://access.redhat.com/documentation/en/openshift-enterprise/version-3.1/architecture/#pv-access-modes https://docs.openshift.com/enterprise/3.1/architecture/additional_concepts/storage.html#pv-access-modes