This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1306729 - Glusterfs/Glusterd blocking root ports ( 1-1024 )
Glusterfs/Glusterd blocking root ports ( 1-1024 )
Status: CLOSED NEXTRELEASE
Product: GlusterFS
Classification: Community
Component: rpc (Show other bugs)
3.6.8
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Raghavendra G
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-11 11:40 EST by customercare
Modified: 2016-08-23 08:57 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-08-23 08:57:51 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description customercare 2016-02-11 11:40:49 EST
Description of problem:

Gluster blocks unused privileged ports between 1-1024 that are not in use on the start of Glusterd

as a result, you can't bind to ports that are shown in this "netstat -lnap" output :

tcp        0      0 0.0.0.0:24007           0.0.0.0:*               LISTEN      11550/glusterd      
tcp        0      0 83.246.80.131:985       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1007      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:980       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1003      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1006      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:24007     83.246.80.132:1023      VERBUNDEN   11550/glusterd      
tcp        0      0 127.0.0.1:1019          127.0.0.1:24007         VERBUNDEN   11573/glusterfs     
tcp        0      0 83.246.80.131:1013      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:992       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1017      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1014      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:989       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1020      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:991       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:986       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:994       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:990       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:981       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1009      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1022      83.246.80.131:24007     VERBUNDEN   2956/glusterfs      
tcp        0      0 83.246.80.131:1001      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1000      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:984       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1018      83.246.80.132:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1012      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:24007     83.246.80.131:1022      VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:998       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:996       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1002      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:997       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1023      83.246.80.132:24007     VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:983       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:988       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:987       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:978       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:979       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1016      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1005      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1004      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 127.0.0.1:24007         127.0.0.1:1019          VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:1015      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:977       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:999       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1010      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:982       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1011      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 127.0.0.1:1020          127.0.0.1:24007         TIME_WAIT   -   

i.e. if gluster starts before dovecot / exim / httpd etc. those service fail to bind itself to theire ports. 

This is a major fault, as systemds starts gluster in an unpredicable way and interrupts production servers afer a reboot.



Version-Release number of selected component (if applicable):

glusterfs-3.6.8-2.fc22.i686
glusterfs-cli-3.6.8-2.fc22.i686
glusterfs-server-3.6.8-2.fc22.i686
glusterfs-api-3.6.8-2.fc22.i686
glusterfs-fuse-3.6.8-2.fc22.i686
glusterfs-libs-3.6.8-2.fc22.i686

How reproducible:

100%

Steps to Reproduce:

starting replaction mode cluster between 2 servers 

Actual results:

a lot of halfopen connections state TIME_WAIT on ports that are not in use between 1 and 1024.

Expected results:

only connections to port 1020 from port 24007


Additional info:
Comment 1 Joe Julian 2016-06-23 13:47:17 EDT
You can extend the ordering of your systemd services with dropins, for instance dovecot comes with a dovecot.socket file which opens the tcp ports for imap and imaps. You can tell the glusterd.service file to wait for dovecot.socket to be started first with a drop-in:

/etc/systemd/system/glusterd.service.d/10-waitfor_dovecot_socket

    [Unit]
    After=dovecot.socket


IMHO, that's the best way to handle this particular problem. 

You should also be able to use net.ipv4.ip_local_reserved_ports in /etc/sysctl.conf, ie. "net.ipv4.ip_local_reserved_ports = 143, 993, 443, 25, 587" for instance.
Comment 2 Kaushal 2016-08-23 08:57:51 EDT
This bug is being closed as GlusterFS-3.6 is nearing its End-Of-Life and only important security bugs will be fixed. This bug has been fixed in more recent GlusterFS releases. If you still face this bug with the newer GlusterFS versions, please open a new bug.

Since GlusterFS-3.7.3, Gluster defaults to using insecure ports (ie. ports > 1024). This issue should no longer happen in newer GlusterFS releases.

Note You need to log in before you can comment on or make changes to this bug.