1306729 – Glusterfs/Glusterd blocking root ports ( 1-1024 )

Bug 1306729 - Glusterfs/Glusterd blocking root ports ( 1-1024 )

Summary: Glusterfs/Glusterd blocking root ports ( 1-1024 )

Keywords:
Status:	CLOSED NEXTRELEASE
Alias:	None
Product:	GlusterFS
Classification:	Community
Component:	rpc
Sub Component:
Version:	3.6.8
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	urgent
Target Milestone:	---
Assignee:	Raghavendra G
QA Contact:
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-02-11 16:40 UTC by customercare
Modified:	2016-08-23 12:57 UTC (History)
CC List:	9 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-08-23 12:57:51 UTC
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description customercare 2016-02-11 16:40:49 UTC

Description of problem:

Gluster blocks unused privileged ports between 1-1024 that are not in use on the start of Glusterd

as a result, you can't bind to ports that are shown in this "netstat -lnap" output :

tcp        0      0 0.0.0.0:24007           0.0.0.0:*               LISTEN      11550/glusterd      
tcp        0      0 83.246.80.131:985       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1007      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:980       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1003      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1006      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:24007     83.246.80.132:1023      VERBUNDEN   11550/glusterd      
tcp        0      0 127.0.0.1:1019          127.0.0.1:24007         VERBUNDEN   11573/glusterfs     
tcp        0      0 83.246.80.131:1013      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:992       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1017      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1014      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:989       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1020      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:991       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:986       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:994       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:990       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:981       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1009      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1022      83.246.80.131:24007     VERBUNDEN   2956/glusterfs      
tcp        0      0 83.246.80.131:1001      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1000      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:984       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1018      83.246.80.132:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1012      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:24007     83.246.80.131:1022      VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:998       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:996       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1002      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:997       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1023      83.246.80.132:24007     VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:983       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:988       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:987       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:978       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:979       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1016      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1005      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1004      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 127.0.0.1:24007         127.0.0.1:1019          VERBUNDEN   11550/glusterd      
tcp        0      0 83.246.80.131:1015      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:977       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:999       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1010      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:982       83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 83.246.80.131:1011      83.246.80.131:24007     TIME_WAIT   -                   
tcp        0      0 127.0.0.1:1020          127.0.0.1:24007         TIME_WAIT   -   

i.e. if gluster starts before dovecot / exim / httpd etc. those service fail to bind itself to theire ports. 

This is a major fault, as systemds starts gluster in an unpredicable way and interrupts production servers afer a reboot.



Version-Release number of selected component (if applicable):

glusterfs-3.6.8-2.fc22.i686
glusterfs-cli-3.6.8-2.fc22.i686
glusterfs-server-3.6.8-2.fc22.i686
glusterfs-api-3.6.8-2.fc22.i686
glusterfs-fuse-3.6.8-2.fc22.i686
glusterfs-libs-3.6.8-2.fc22.i686

How reproducible:

100%

Steps to Reproduce:

starting replaction mode cluster between 2 servers 

Actual results:

a lot of halfopen connections state TIME_WAIT on ports that are not in use between 1 and 1024.

Expected results:

only connections to port 1020 from port 24007


Additional info:

Comment 1 Joe Julian 2016-06-23 17:47:17 UTC

You can extend the ordering of your systemd services with dropins, for instance dovecot comes with a dovecot.socket file which opens the tcp ports for imap and imaps. You can tell the glusterd.service file to wait for dovecot.socket to be started first with a drop-in:

/etc/systemd/system/glusterd.service.d/10-waitfor_dovecot_socket

    [Unit]
    After=dovecot.socket


IMHO, that's the best way to handle this particular problem. 

You should also be able to use net.ipv4.ip_local_reserved_ports in /etc/sysctl.conf, ie. "net.ipv4.ip_local_reserved_ports = 143, 993, 443, 25, 587" for instance.

Comment 2 Kaushal 2016-08-23 12:57:51 UTC

This bug is being closed as GlusterFS-3.6 is nearing its End-Of-Life and only important security bugs will be fixed. This bug has been fixed in more recent GlusterFS releases. If you still face this bug with the newer GlusterFS versions, please open a new bug.

Since GlusterFS-3.7.3, Gluster defaults to using insecure ports (ie. ports > 1024). This issue should no longer happen in newer GlusterFS releases.

Note You need to log in before you can comment on or make changes to this bug.