3. What is the nature and description of the request? - implement port range configuration option for gluster: gluster will only try to allocate a port within this range and if it fails error out before creating the brick - each gluster cluster member should "scan" this portrange for gaps in the assignment of ports and if they can bind to this port; the port maps should be merged from each member to find a proper port for the brick. - another way would be to bind all ports already (basically reserving them), which would allow gluster to verify that all ports are reachable(not blocked by fw) and bindable; glusterfs can already give an error/warning about this early on and you will have guarantee of the amount of bricks you want to create. 4. Why does the customer need this? (List the business requirements here) - Recommendations result in a broken setup - Resizing the port range is a workaround but doesn't prevent these issues - Reliable and predictable number of maximum bricks - Have an atomic state: either successfully create the volume/bricks or fail out - Opening a port range that can be used by other software defeats partly the purpose of a firewall 5. How would the customer like to achieve this? (List the functional requirements here) See 3. 6. For each functional requirement listed in question 5, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented. - Open a port range of 100 - create 100 bricks/volumes - verify behavior: a) 100 volumes get created b) an error returned that no more open ports are available if other processes use a port in this range.
Ok, I have gone through the case also. The expectation here is to specify max port range for gluster in some config and then match it with firewall. In theory it is possible with adding a option like option base-port 49152 in glusterd.vol. option max-port or something. I say theoretically because it conflicts with longer road map for Gluster. I will have an update on this by this week end on feasibility.
This has been implemented in RHGS (see BZ #1474745). Is that all that is needed, or do we need to do any changes in the container itself?
The feature now exists in gluster, but in order to make use of it and enable it conveniently for the user if OCS, we need changes in the server-container (at least). This RFE is beyond the scope of 3.11.0.
The requirement is - Open a port range of 100 - create 100 bricks/volumes - verify behavior, that either of the two happens: a) 100 volumes get created b) an error returned that no more open ports are available if other processes use a port in this range. With the feature in RHGS(https://bugzilla.redhat.com/show_bug.cgi?id=1474745), this can be achieved by changing the test case above to - Open a port range of 100, say from 49152 to 49252 - Set the range in /etc/glusterfs/glusterd.vol and restart glusterd using the two lines. option base-port 49152 option max-port 49252 - create 100 bricks/volumes - verify behavior, that either of the two happens: a) 100 volumes get created b) an error returned that no more open ports are available if other processes use a port in this range. The range however isn't easy to make an estimate of. Depending on the whether the setup has brick mux enabled or not and how many volumes the setup has, the range could be anywhere from 10 to 1000. Option 1: Automate as much as possible The openshift ansible installer opens 100 ports from 49152 to 49251 and may be it would be good to match the two. To do so, the best way is to add the two options to the template/daemonset and then use the internal script to set it in glusterd.vol. Option 2: Just document the RHGS feature Michael/Jose/John, What are your preferences?
I've read the bz over a few times and I'm still confused. Is it correct to say that brick mux makes the number of volumes supported within this restricted port range unpredictable? If so, then the test case given above is not valid, but rather you need to create glusterfsd until all 100 (or whatever number) port range is consumed. If I'm wrong and this really restricts us to 100 ports what happens to all the users who already have more than 100 volumes? Honestly, I'm a little skeptical that implementing this RFE won't cause more problems than it solves, especially given that it was originally created a couple of years ago.
We already effectively restrict the number of ports available for bricks because we only open the specified range in the node firewalls via openshift-ansible. It would be trivial to change the GlusterFS DaemonSet to use these values, but non-trivial to make them configurable. That said, it doesn't seem worth our time to implement at this point, given that very few (if any?) customers have run into issues with this.
Thanks Jose and John. Agreeing with Jose and John, I am converting this bug into a doc bz. Changes required in admin guide of OCS. In the section where brick multiplexing is explained, please link the "21.2.2. Port Range Configuration" section of RHGS admin guide. Prefix to the link: If you want to configure gluster bricks to bind and listen to a limited range of ports, please refer to the section <link>.
Adding more content for the doc, another line to prefix. To edit configuration files for glusterd, you can rsh into the pod using the command oc rsh <glusterfs-pod-name>