Bug 1304489 - What PV is chosen when multiple are provided with different AccessModes
What PV is chosen when multiple are provided with different AccessModes
Status: CLOSED CURRENTRELEASE
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation (Show other bugs)
3.1.0
Unspecified Unspecified
high Severity low
: ---
: ---
Assigned To: Ashley Hardin
Eric Jones
Vikram Goyal
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-03 14:36 EST by Eric Jones
Modified: 2016-03-18 11:12 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-18 11:12:17 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Eric Jones 2016-02-03 14:36:52 EST
Document URL: 
https://access.redhat.com/documentation/en/openshift-enterprise/version-3.1/architecture/#persistent-volume-claims

Section Number and Name: 
4.5.4. Persistent Volume Claims

Describe the issue: 
Several templates provided with OSE have multiple AccessModes specified in the PVC. This is fine as this simply allows the claim to bind to a PV of either mode. The problem is that the documentation does not describe what happens when PVs of BOTH AccessModes are present. Does it select the PV that has the AccessMode that is listed first in the claim? Or does it go by something else?

Suggestions for improvement: 
Determine how the claim determines which PV to bind to.
Comment 3 Ashley Hardin 2016-02-16 11:52:49 EST
@Mark-Do you have any insight into this? Thanks!
Comment 4 Mark Turansky 2016-02-29 12:20:57 EST
update doc here:  https://github.com/openshift/openshift-docs/pull/1665
Comment 5 Ashley Hardin 2016-03-02 16:53:21 EST
follow-up PR with minor edits: https://github.com/openshift/openshift-docs/pull/1683
Comment 6 openshift-github-bot 2016-03-04 16:59:08 EST
Commits pushed to master at https://github.com/openshift/openshift-docs

https://github.com/openshift/openshift-docs/commit/55dccec546adeb2c3b6591440fd586a3b761f22a
Bug 1304489, edited AccessModes content

https://github.com/openshift/openshift-docs/commit/cd0545c4a43c13e4dd9d8b3f654613012011da60
Merge pull request #1683 from ahardin-rh/access-modes

Bug 1304489, edited AccessModes content
Comment 8 Eric Jones 2016-03-10 17:27:37 EST
The information that you were able to provide is very helpful, however, there is another bit of clarification that would be beneficial and the main reason I opened the bug was that there was no information on how a PVC determines what PV to select and this is still true.

In the example you provide, "if a claim requests RWO, but the only volume available was an NFS PV (RWO+ROX+RWX)" what if there were two NFS PVs and an iSCSI PV, NFSPV1 had RWO+ROX+RWX, for some reason NFSPV2 only had RWX+RWO, and ISCSIPV had RWO+ROX which of those would the PVC bind to?

Sorry to re-open the bug, I suppose I should've clarified that detail in the beggining, I'm sorry.
Comment 9 Ashley Hardin 2016-03-11 10:01:58 EST
@Eric, No problem!
@Mark, Can you please help me fill in the gap? Thanks!
Comment 10 Mark Turansky 2016-03-16 11:55:37 EDT
@Eric,

The only 2 matching criteria are access modes and size.  The modes must match or contain more modes than you requested.  The size must be greater than or equal to what is expected.   If 2 types of volumes (NFS and iscsci) both have the same set of access modes, then *either* of them will match a claim w/ those modes.  There is no ordering between types of volumes.

In your example, a claim w/ all 3 modes would only match the 2 volumes with all 3 modes.   If the claim had only RWX, then it could match *any* of the volumes, since all have RWX.  

Overall, though, if you've got 2 types of volumes with the same access modes, there is no way to choose one type over another.  That'll be achievable with the PersistentVolumeSelector feature currently in development.

Please let me know if this is helpful or if you'd like more info.  I can appreciate that it's complicated and that an explanation doesn't necessarily make it any clearer ...
Comment 11 Eric Jones 2016-03-16 12:20:30 EDT
I followed up with Mark in IRC to the following effect:

Eric: I do have one last question. I understand that we cannot specify which one to choose, in the situation with multiple PV's that all meet the criteria for the PVC, but how does the system decide? There has to be something that OSE uses to decide which one to choose, even if we can't impact that decision at the moment.

Mark: All volumes with the same modes are grouped and then sorted by size (smallest to largest). The binder gets the group with matching modes and simply iterates over each (in size order) until one size matches.


This answers my questions @Ashley!
Comment 12 Ashley Hardin 2016-03-16 13:10:33 EDT
New PR created: https://github.com/openshift/openshift-docs/pull/1750

@ Mark & Eric, please review. Thanks!
Comment 13 Eric Jones 2016-03-16 14:21:30 EDT
Things look good to me. 

Unfortunately I realized I have one last question for @Mark, and I genuinely believe this is the last one. 

If there are two PV's that match in size and AccessModes, what does the PVC make the decision based on? To word it differently, if size and AccessModes are the same for two PVs, when they are sorted along with the other PVs, are they going to be sorted by name at that point or some other feature? Clearly one has to go "higher" on the priority of selection.
Comment 14 Mark Turansky 2016-03-16 14:29:57 EDT
There is no sorting beyond size/capacity.  However the sort algorithm works in Go is how the PVs are sorted in the list.  All in the list will have the same modes.

There is no deterministic way to know how NFS and ISCSI sorted in the list.   If both volumes are 5gb, there is no telling which one is "up next" to match the 5gb claim.
Comment 15 Eric Jones 2016-03-16 14:33:43 EDT
Good to know. Thank you for the information @Mark, and the docs @Ashley!
Comment 16 Ashley Hardin 2016-03-16 14:42:59 EDT
Thanks, guys! Should anything else be added to the PR?
Comment 17 Eric Jones 2016-03-16 14:46:46 EDT
I think that is everything!
Comment 18 openshift-github-bot 2016-03-16 14:57:35 EDT
Commits pushed to master at https://github.com/openshift/openshift-docs

https://github.com/openshift/openshift-docs/commit/c4969a5deaa9b037effc1ec2fae8b9e9e2399e60
Bug 1304489, clarified how a PVC determines what PV to select in Access Modes section

https://github.com/openshift/openshift-docs/commit/dadb815d936f5e3189495c30cd838c09551ea9cd
Merge pull request #1750 from ahardin-rh/access-modes

Bug 1304489, clarified how a PVC determines what PV to select

Note You need to log in before you can comment on or make changes to this bug.