Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: IF a gluster cluster deployed using cns-deploy is running in one namespace using 3 storage nodes, it's not possible to create another gluster cluster in another namespace using different set of 3 storage nodes. The reasoning behind this is that, when first gluster cluster is created using cns-deploy, it labels those 3 nodes as storagenode=glusterfs and entry is made to daemonset. This labeling is global and in 2nd instance when we give 3 different set of nodes in topology file, it doesn't reads from it and picks up OCP nodes based on node label (storagenode=glusterfs ) and not based on what nodes are in topology.json Version-Release number of selected component (if applicable): cns-deploy-3.1.0-14.el7rhgs.x86_64 heketi-3.1.0-14.el7rhgs.x86_64 heketi-client-3.1.0-14.el7rhgs.x86_64 How reproducible: Always Steps to Reproduce: 1. Create topology file and deploy cns using the same in a namespace. 2. Create a new namespace 3. Create a new topology file and try to create another gluster cluster in a new namespace Actual results: Waiting for GlusterFS pods to start ... Timed out waiting for pods matching 'glusterfs-node=pod'. No resources found Error from server (NotFound): deploymentconfig "heketi" not found Error from server (NotFound): services "heketi" not found Error from server (NotFound): routes "heketi" not found Error from server (NotFound): services "heketi-storage-endpoints" not found serviceaccount "heketi-service-account" deleted template "deploy-heketi" deleted template "heketi" deleted node "ip-172-31-41-103.us-west-2.compute.internal" labeled node "ip-172-31-43-103.us-west-2.compute.internal" labeled node "ip-172-31-56-4.us-west-2.compute.internal" labeled daemonset "glusterfs" deleted template "glusterfs" deleted If you see node labels after executing this, it still shows only storagenode=glusterfs on those nodes which was used to create CNS in first instance. See oc get nodes --show-labels output for that. Expected results: CNS-deploy should progress smoothly and we have two different gluster cluster on two different name spaces.
If you are planning for different cluster in different namespace, you can try different storage node labelling. To accommodate/achieve this we have to edit a new label in the template and trycns-deploy with second namespace.
Yeah As for longer plan to support this, As the node label are global, DaemonSet being Namespace specific which deploys based on node labels set. There is no validation to stop you from starting the same application on different namespace with same label. From cns-deploy point, We can add an option to do configurable label. So that the deployment of cns on another namespace do not clash on the node labels used by the first Deployment. We can have a flag in cns-deploy like --node-label with just like --namespace. If not specified we can fall to default value. Humble and I discussed about this, Please share your thoughts Jose.
The proposed solution seems all right. Given that heketi limits its pod search by namespace, I don't see a conflict. I do have a question about the initial request, however: Why do we need to run cns-deploy more than once? Couldn't the second cluster just be managed by the same heketi instance?
I am trying to create CNS cluster in a new namespace and not in the existing namespace. Hence I was trying it by running cns-deploy again. What are the steps to create a new CNS cluster in a new namespace using the existing heketi instance?
Thi is fixed in CNS 3.5 release. Precisely with an async release.
*** This bug has been marked as a duplicate of bug 1444460 ***
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days