Bug 1392383

This is an RFE and not a BUG, hopefully we will have it in near future.

Additional info#  https://github.com/humblec/kubernetes/commit/c105ba15c2e19d5399adabdef19682cc65f02877

(In reply to Humble Chirammal from comment #2)
> This is an RFE and not a BUG, hopefully we will have it in near future.
> 
> Additional info# 
> https://github.com/humblec/kubernetes/commit/
> c105ba15c2e19d5399adabdef19682cc65f02877

This `storage class` can be of many types. The QOS based storage class is *just* one of them. So, we can not say it doesnt make sense to define this way or other. As I said in the previous comment, if we really think `clusterid` can be an optional parameter, its fine. However let me emphasize that, it is by design we give full control to Heketi to decide where to place the volume. We never wanted to take that control from Heketi and give it to someone because the intelligence to decide where the brick should go ...etc are one of the main advantage of Heketi. But to trigger a discussion whether this optional parameter add any value or not, I have converted my PR to k8s PR as you can see here https://github.com/kubernetes/kubernetes/pull/36437. Please note that this PR is filed *just for triggering the discussion or to get opinion from others*. Its never guaranteed that we will be solving it this way. At any case, this is not tracked for CNS release now.

Thanks for your comments, Humble!

To enforce your point:

It is deliberate that we don't expose the clusterID to creating volumes. We don't want that and we will never do that (as far as I am concerned). It's heketi's task to hide that from the end user.

What I can very well imagine is having, instead of a clusterId, a volume quality, like 'slow', and 'fast', as an optional parameter, which will, when handed to the volume creation, have the effect of reducing the clusters that heketi consults for creating the volume to those who can satisfy this class.

So:

If we can rephrase the RFE in these directions, I am fine with it. But not for 3.4, but rather for a future release.

If instead you insist on clusterID exposed, I will refuse and close as wontfix.

@Humble
Well, with storage the idea of putting a TSP with SSD's will be be get a better quality of service (and storage class will enable be to do that). However, if heketi is going to override that and put the volume on a SATA based TSP, it may just defeat the purpose. 

Yes, we can put it as an RFE, but I see it as a limitation for now.

@Anoop:

Please see comment #4 : 

Gluster-Cluster-level (is that what you mean by TSP?) is just the wrong granularity. We will not support that because it seems wrong layering and contradicting our appoaches. As I said, giving a storage quality (however this will be called - slow vs fast or ssd vs spinning rust or ...) is a reasonable request, and we'd need to make heketi aware of the concepts and then properly hand it through the layers so that heketi can honor this attribute in the volume create request. But clusterID-level is plain wrong, imho.

By TSP I mean Trusted storage pool. Today heketi takes the decision on finding an appropriate TSP for the requested volume. If we are explicitly defining a storage class (say "fast" on SSD's), then as a user I would expect the same is passed to Heketi and I get volume from the SSD pool. Without this, how does defining a storage class help the user?

(In reply to Anoop from comment #7)
> By TSP I mean Trusted storage pool. Today heketi takes the decision on
> finding an appropriate TSP for the requested volume.

Yes, and this is how it should be.

> If we are explicitly
> defining a storage class (say "fast" on SSD's), then as a user I would
> expect the same is passed to Heketi and I get volume from the SSD pool.
> Without this, how does defining a storage class help the user?

I fully agree. May I quote myself from comment #6:

>> As I said, giving a storage quality (however
>> this will be called - slow vs fast or ssd vs spinning rust or ...) is a
>> reasonable request, and we'd need to make heketi aware of the concepts and
>> then properly hand it through the layers so that heketi can honor this
>> attribute in the volume create request.

So as I said in comment #4:

>>> What I can very well imagine is having, instead of a clusterId,
>>> a volume quality, like 'slow', and 'fast', as an optional parameter,
>>> which will, when handed to the volume creation, have the effect of
>>> reducing the clusters that heketi consults for creating the volume
>>> to those who can satisfy this class.

I.e. yes, we would need to teach heketi how to create a volume from a SUBSET of all clusters available (the subset that satisfy the quality propoerty of the storage class), but heketi should still decide which exact cluster to use. 

I am only saying we won't teach heketi to create a volume from an explicitly chose clusterId because that contradicts the main (and imho good) design and usability prinicples of heketi.

So, if we can agree on a different wording of the RFE, I am all supportive... :-)

(In reply to Michael Adam from comment #8)
> (In reply to Anoop from comment #7)
> > By TSP I mean Trusted storage pool. Today heketi takes the decision on
> > finding an appropriate TSP for the requested volume.
> 
> Yes, and this is how it should be.
> 
> > If we are explicitly
> > defining a storage class (say "fast" on SSD's), then as a user I would
> > expect the same is passed to Heketi and I get volume from the SSD pool.
> > Without this, how does defining a storage class help the user?
> 
> I fully agree. May I quote myself from comment #6:
> 
> >> As I said, giving a storage quality (however
> >> this will be called - slow vs fast or ssd vs spinning rust or ...) is a
> >> reasonable request, and we'd need to make heketi aware of the concepts and
> >> then properly hand it through the layers so that heketi can honor this
> >> attribute in the volume create request.
> 
> So as I said in comment #4:
> 
> >>> What I can very well imagine is having, instead of a clusterId,
> >>> a volume quality, like 'slow', and 'fast', as an optional parameter,
> >>> which will, when handed to the volume creation, have the effect of
> >>> reducing the clusters that heketi consults for creating the volume
> >>> to those who can satisfy this class.
> 
> I.e. yes, we would need to teach heketi how to create a volume from a SUBSET
> of all clusters available (the subset that satisfy the quality propoerty of
> the storage class), but heketi should still decide which exact cluster to
> use. 
> 
> I am only saying we won't teach heketi to create a volume from an explicitly
> chose clusterId because that contradicts the main (and imho good) design and
> usability prinicples of heketi.
> 
> So, if we can agree on a different wording of the RFE, I am all
> supportive... :-)

Right, I agree and it makes sense to leave the TSP selection to heketi based on the claim request.

enhancement, for next release at earliest

Patch has made it into openshift 3.5.

==> once we have all ACKs, we only need to wait for the openshift build.

The functionality has been added to Kube and available in Origin via #https://github.com/openshift/origin/pull/12556


--snip-- from the subjected PR

 * `clusterid`: `630372ccdc720a92c681fb928f27b53f` is the ID of the cluster which will be used by Heketi when provisioning the volume. It can also be a list of clusterids, for ex:
"8452344e2becec931ece4e33c4674e4e,42982310de6c63381718ccfa6d8cf397". This is an optional parameter.

--/snip--

Now admin have the privilage to specify the Cluster which he want as shown below. With this heketi provisioner instruct heketi to create volumes from the mentioned Cluster or Group of clusters.  In short, it  gives the admin the choice to specify the cluster based on QOS.

Fixed in CNS 3.5 release.