Bug 1528809

Summary: glusterd blocks DNF on install
Product: [Community] GlusterFS Reporter: customercare
Component: glusterdAssignee: bugs <bugs>
Status: CLOSED EOL QA Contact:
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.10CC: bugs, customercare, humble.devassy, jonathansteffan, kkeithle, matthias, ndevos, ramkrsna
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-20 18:27:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description customercare 2017-12-24 10:23:59 UTC
Description of problem:

Again Gluster is a "pain...ss" installing it :


(1/10): glusterfs-cli-3.10.8-1.fc26.x86_64.rpm               131 kB/s | 206 kB     00:01    
(2/10): glusterfs-3.10.8-1.fc26.x86_64.rpm                   302 kB/s | 545 kB     00:01    
(3/10): glusterfs-fuse-3.10.8-1.fc26.x86_64.rpm              898 kB/s | 144 kB     00:00    
(4/10): glusterfs-server-3.10.8-1.fc26.x86_64.rpm            715 kB/s | 1.4 MB     00:01    
(5/10): pyxattr-0.5.3-10.fc26.x86_64.rpm                     223 kB/s |  33 kB     00:00    
(6/10): firewalld-filesystem-0.4.4.5-1.fc26.noarch.rpm       304 kB/s |  70 kB     00:00    
(7/10): userspace-rcu-0.9.3-2.fc26.x86_64.rpm                616 kB/s |  91 kB     00:00    
(8/10): glusterfs-api-3.10.8-1.fc26.x86_64.rpm               959 kB/s |  98 kB     00:00    
(9/10): glusterfs-client-xlators-3.10.8-1.fc26.x86_64.rpm    1.0 MB/s | 857 kB     00:00    
(10/10): glusterfs-libs-3.10.8-1.fc26.x86_64.rpm             1.1 MB/s | 388 kB     00:00    
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gesamt                                                       732 kB/s | 3.8 MB     00:05     
Transaktionsüberprüfung wird ausgeführt
Transaktionsprüfung war erfolgreich.
Transaktion wird getestet
Transaktionstest war erfolgreich.
Transaktion wird ausgeführt
  Vorbereitung läuft    :                                                                1/1 
  Installieren          : glusterfs-libs-3.10.8-1.fc26.x86_64                           1/10 
  Ausgeführtes Scriptlet: glusterfs-libs-3.10.8-1.fc26.x86_64                           1/10 
  Installieren          : glusterfs-3.10.8-1.fc26.x86_64                                2/10 
  Ausgeführtes Scriptlet: glusterfs-3.10.8-1.fc26.x86_64                                2/10 
  Installieren          : glusterfs-client-xlators-3.10.8-1.fc26.x86_64                 3/10 
  Installieren          : glusterfs-fuse-3.10.8-1.fc26.x86_64                           4/10 
  Installieren          : glusterfs-api-3.10.8-1.fc26.x86_64                            5/10 
  Ausgeführtes Scriptlet: glusterfs-api-3.10.8-1.fc26.x86_64                            5/10 
  Installieren          : glusterfs-cli-3.10.8-1.fc26.x86_64                            6/10 
  Installieren          : userspace-rcu-0.9.3-2.fc26.x86_64                             7/10 
  Ausgeführtes Scriptlet: userspace-rcu-0.9.3-2.fc26.x86_64                             7/10 
  Installieren          : pyxattr-0.5.3-10.fc26.x86_64                                  8/10 
  Installieren          : firewalld-filesystem-0.4.4.5-1.fc26.noarch                    9/10 
  Installieren          : glusterfs-server-3.10.8-1.fc26.x86_64                        10/10 
  Ausgeführtes Scriptlet: glusterfs-server-3.10.8-1.fc26.x86_64     

Here DNF output stops because of :

        ├─sshd─┬─sshd───sshd───bash───dnf───sh───glusterd───7*[{glusterd}]
        │      └─sshd───sshd───bash───pstree


aons later, i had time to get a coffee, it returned. I bet 10:1, it dit something, it should do 
when started for the firsttime, not on install or update time.


Meanwhile it did this ( partly of course ) 

[pid  8502] socket(AF_NETLINK, SOCK_RAW|SOCK_CLOEXEC, NETLINK_ROUTE) = 10
[pid  8502] bind(10, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 0
[pid  8502] getsockname(10, {sa_family=AF_NETLINK, nl_pid=8500, nl_groups=00000000}, [12]) = 0
[pid  8502] sendto(10, {{len=20, type=RTM_GETLINK, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110364, pid=0}, {ifi_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110364, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_LOOPBACK, ifi_index=if_nametoindex("lo"), ifi_flags=IFF_UP|IFF_LOOPBACK|IFF_RUNNING|0x10000, ifi_change=0}, [{{nla_len=7, nla_type=IFLA_IFNAME}, "lo"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 0}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 65536}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=12, nla_type=IFLA_QDISC}, "noqueue"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 0}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=14626062, tx_packets=14626062, rx_bytes=2376154151, tx_bytes=2376154151, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=14626062, tx_packets=14626062, rx_bytes=2376154151, tx_bytes=2376154151, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00"...}]}, {{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110364, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_ETHER, ifi_index=if_nametoindex("eth0"), ifi_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST|0x10000, ifi_change=0}, [{{nla_len=9, nla_type=IFLA_IFNAME}, "eth0"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 6}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 1500}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=8, nla_type=IFLA_QDISC}, "htb"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 1}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x7e\xdd\xa4\x4e\x06\x9b"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\xff\xff\xff\xff\xff\xff"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=41984709, tx_packets=8028725, rx_bytes=4375445663, tx_bytes=94626834776, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=41984709, tx_packets=8028725, rx_bytes=80478367, tx_bytes=137554264, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00"...}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 2536
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110364, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] sendto(10, {{len=20, type=RTM_GETADDR, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110365, pid=0}, {ifa_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=76, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110365, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 127.0.0.1}, {{nla_len=8, nla_type=IFA_LOCAL}, 127.0.0.1}, {{nla_len=7, nla_type=IFA_LABEL}, "lo"}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=1909}}]}, {{len=88, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110365, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=27, ifa_flags=0, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("eth0")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_LOCAL}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_BROADCAST}, 83.246.80.159}, {{nla_len=9, nla_type=IFA_LABEL}, "eth0"}, {{nla_len=8, nla_type=IFA_FLAGS}, 0}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=762063, ifa_valid=762063, cstamp=2290, tstamp=254192327}}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 164
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110365, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=128, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, ::1}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=144}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}, {{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110365, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("eth0")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, fe80::7cdd:a4ff:fe4e:69b}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1985, tstamp=1985}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 144
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110365, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] close(10)                   = 0
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643921, tv_nsec=15904061}) = 0
[pid  8502] sendto(6, "h,\361\335\0\0\0\0\0\0\0\2\0\1\206\240\0\0\0\2\0\0\0\2\0\0\0\0\0\0\0\0"..., 56, 0, {sa_family=AF_INET, sin_port=htons(111), sin_addr=inet_addr("127.0.0.1")}, 16) = 56
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643921, tv_nsec=16437493}) = 0
[pid  8502] poll([{fd=6, events=POLLIN}], 1, 5000 <unfinished ...>
[pid  8501] <... nanosleep resumed> NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643921, tv_nsec=891261274}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643922, tv_nsec=891712420}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643923, tv_nsec=892107967}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643924, tv_nsec=892498950}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643925, tv_nsec=892930811}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0},  <unfinished ...>
[pid  8502] <... poll resumed> )        = 0 (Timeout)
[pid  8502] socket(AF_NETLINK, SOCK_RAW|SOCK_CLOEXEC, NETLINK_ROUTE) = 10
[pid  8502] bind(10, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 0
[pid  8502] getsockname(10, {sa_family=AF_NETLINK, nl_pid=8500, nl_groups=00000000}, [12]) = 0
[pid  8502] sendto(10, {{len=20, type=RTM_GETLINK, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110369, pid=0}, {ifi_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110369, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_LOOPBACK, ifi_index=if_nametoindex("lo"), ifi_flags=IFF_UP|IFF_LOOPBACK|IFF_RUNNING|0x10000, ifi_change=0}, [{{nla_len=7, nla_type=IFLA_IFNAME}, "lo"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 0}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 65536}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=12, nla_type=IFLA_QDISC}, "noqueue"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 0}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=14626285, tx_packets=14626285, rx_bytes=2376187282, tx_bytes=2376187282, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=14626285, tx_packets=14626285, rx_bytes=2376187282, tx_bytes=2376187282, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00"...}]}, {{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110369, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_ETHER, ifi_index=if_nametoindex("eth0"), ifi_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST|0x10000, ifi_change=0}, [{{nla_len=9, nla_type=IFLA_IFNAME}, "eth0"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 6}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 1500}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=8, nla_type=IFLA_QDISC}, "htb"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 1}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x7e\xdd\xa4\x4e\x06\x9b"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\xff\xff\xff\xff\xff\xff"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=41984772, tx_packets=8028744, rx_bytes=4375449669, tx_bytes=94626848307, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=41984772, tx_packets=8028744, rx_bytes=80482373, tx_bytes=137567795, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00"...}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 2536
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110369, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] sendto(10, {{len=20, type=RTM_GETADDR, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110370, pid=0}, {ifa_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=76, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110370, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 127.0.0.1}, {{nla_len=8, nla_type=IFA_LOCAL}, 127.0.0.1}, {{nla_len=7, nla_type=IFA_LABEL}, "lo"}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=1909}}]}, {{len=88, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110370, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=27, ifa_flags=0, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("eth0")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_LOCAL}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_BROADCAST}, 83.246.80.159}, {{nla_len=9, nla_type=IFA_LABEL}, "eth0"}, {{nla_len=8, nla_type=IFA_FLAGS}, 0}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=762058, ifa_valid=762058, cstamp=2290, tstamp=254192327}}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 164
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110370, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=128, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, ::1}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=144}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}, {{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110370, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("eth0")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, fe80::7cdd:a4ff:fe4e:69b}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1985, tstamp=1985}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 144
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110370, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] close(10)                   = 0
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643926, tv_nsec=26429665}) = 0
[pid  8502] sendto(6, "h,\361\335\0\0\0\0\0\0\0\2\0\1\206\240\0\0\0\2\0\0\0\2\0\0\0\0\0\0\0\0"..., 56, 0, {sa_family=AF_INET, sin_port=htons(111), sin_addr=inet_addr("127.0.0.1")}, 16) = 56
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643926, tv_nsec=26791790}) = 0
[pid  8502] poll([{fd=6, events=POLLIN}], 1, 4936 <unfinished ...>
[pid  8501] <... nanosleep resumed> NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643926, tv_nsec=893186044}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643927, tv_nsec=893514826}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643928, tv_nsec=893813844}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, ^Cstrace: Process 8500 detached
strace: Process 8501 detached
 <detached ...>
strace: Process 8502 detached
strace: Process 8503 detached
strace: Process 8504 detached
strace: Process 8505 detached
strace: Process 8506 detached
strace: Process 8507 detached
[root@s113 ~]# strace -s 256 -f -p `pidof glusterd`
strace: Process 8500 attached with 8 threads
[pid  8507] epoll_wait(3,  <unfinished ...>
[pid  8506] futex(0x564af7956bd8, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid  8505] restart_syscall(<... resuming interrupted restart_syscall ...> <unfinished ...>
[pid  8504] restart_syscall(<... resuming interrupted restart_syscall ...> <unfinished ...>
[pid  8503] restart_syscall(<... resuming interrupted nanosleep ...> <unfinished ...>
[pid  8502] restart_syscall(<... resuming interrupted poll ...> <unfinished ...>
[pid  8501] restart_syscall(<... resuming interrupted nanosleep ...> <unfinished ...>
[pid  8500] futex(0x7fab80cd39d0, FUTEX_WAIT, 8507, NULL <unfinished ...>
[pid  8501] <... restart_syscall resumed> ) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643932, tv_nsec=894731222}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643933, tv_nsec=895043740}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643934, tv_nsec=895447872}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643935, tv_nsec=895853099}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0},  <unfinished ...>
[pid  8502] <... restart_syscall resumed> ) = 0
[pid  8502] socket(AF_NETLINK, SOCK_RAW|SOCK_CLOEXEC, NETLINK_ROUTE) = 10
[pid  8502] bind(10, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 0
[pid  8502] getsockname(10, {sa_family=AF_NETLINK, nl_pid=8500, nl_groups=00000000}, [12]) = 0
[pid  8502] sendto(10, {{len=20, type=RTM_GETLINK, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110379, pid=0}, {ifi_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110379, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_LOOPBACK, ifi_index=if_nametoindex("lo"), ifi_flags=IFF_UP|IFF_LOOPBACK|IFF_RUNNING|0x10000, ifi_change=0}, [{{nla_len=7, nla_type=IFLA_IFNAME}, "lo"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 0}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 65536}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=12, nla_type=IFLA_QDISC}, "noqueue"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 0}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\x00\x00\x00\x00\x00\x00"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=14626287, tx_packets=14626287, rx_bytes=2376187450, tx_bytes=2376187450, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=14626287, tx_packets=14626287, rx_bytes=2376187450, tx_bytes=2376187450, rx_errors=0, tx_errors=0, rx_dropped=0, tx_dropped=0, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x27\x00\x00\xe8\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x60\x02\x0a\x00\x08\x00\x01\x00\x00\x00\x00\x80\x14\x00\x05\x00\xff\xff\x00\x00\x90\x00\x00\x00\xd0\x62\x00\x00\xe8\x03\x00\x00\xcc\x00\x02\x00\x00\x00\x00\x00\x40\x00\x00\x00\x00\x00\x01\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\xff\xff\xff\xff\xa0\x0f\x00\x00\xe8\x03\x00\x00\xff\xff\xff\xff\x80\x3a\x09\x00\x80\x51\x01\x00\x03\x00\x00\x00\x58\x02\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x60\xea\x00\x00\x00\x00\x00\x00"...}]}, {{len=1268, type=RTM_NEWLINK, flags=NLM_F_MULTI, seq=1514110379, pid=8500}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_ETHER, ifi_index=if_nametoindex("eth0"), ifi_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST|0x10000, ifi_change=0}, [{{nla_len=9, nla_type=IFLA_IFNAME}, "eth0"}, {{nla_len=8, nla_type=IFLA_TXQLEN}, 1000}, {{nla_len=5, nla_type=IFLA_OPERSTATE}, 6}, {{nla_len=5, nla_type=IFLA_LINKMODE}, 0}, {{nla_len=8, nla_type=IFLA_MTU}, 1500}, {{nla_len=8, nla_type=IFLA_GROUP}, 0}, {{nla_len=8, nla_type=IFLA_PROMISCUITY}, 0}, {{nla_len=8, nla_type=IFLA_NUM_TX_QUEUES}, 1}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SEGS}, 65535}, {{nla_len=8, nla_type=IFLA_GSO_MAX_SIZE}, 65536}, {{nla_len=8, nla_type=IFLA_NUM_RX_QUEUES}, 1}, {{nla_len=5, nla_type=IFLA_CARRIER}, 1}, {{nla_len=8, nla_type=IFLA_QDISC}, "htb"}, {{nla_len=8, nla_type=IFLA_CARRIER_CHANGES}, 1}, {{nla_len=5, nla_type=IFLA_PROTO_DOWN}, 0}, {{nla_len=36, nla_type=IFLA_MAP}, {mem_start=0, mem_end=0, base_addr=0, irq=0, dma=0, port=0}}, {{nla_len=10, nla_type=IFLA_ADDRESS}, "\x7e\xdd\xa4\x4e\x06\x9b"}, {{nla_len=10, nla_type=IFLA_BROADCAST}, "\xff\xff\xff\xff\xff\xff"}, {{nla_len=196, nla_type=IFLA_STATS64}, {rx_packets=41984907, tx_packets=8028791, rx_bytes=4375457069, tx_bytes=94626865761, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=100, nla_type=IFLA_STATS}, {rx_packets=41984907, tx_packets=8028791, rx_bytes=80489773, tx_bytes=137585249, rx_errors=0, tx_errors=0, rx_dropped=19, tx_dropped=3384, multicast=0, collisions=0, rx_length_errors=0, rx_over_errors=0, rx_crc_errors=0, rx_frame_errors=0, rx_fifo_errors=0, rx_missed_errors=0, tx_aborted_errors=0, tx_carrier_errors=0, tx_fifo_errors=0, tx_heartbeat_errors=0, tx_window_errors=0, rx_compressed=0, tx_compressed=0, rx_nohandler=0}}, {{nla_len=12, nla_type=IFLA_XDP}, {{nla_len=5, nla_type=IFLA_XDP_ATTACHED}, 0}}, {{nla_len=744, nla_type=IFLA_AF_SPEC}, "\x84\x00\x02\x00\x80\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x27\x00\x00\xe8\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x60\x02\x0a\x00\x08\x00\x01\x00\x10\x00\x00\x80\x14\x00\x05\x00\xff\xff\x00\x00\xc1\x07\x00\x00\x70\x6b\x00\x00\xe8\x03\x00\x00\xcc\x00\x02\x00\x00\x00\x00\x00\x40\x00\x00\x00\xdc\x05\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\xff\xff\xff\xff\xa0\x0f\x00\x00\xe8\x03\x00\x00\x00\x00\x00\x00\x80\x3a\x09\x00\x80\x51\x01\x00\x03\x00\x00\x00\x58\x02\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x60\xea\x00\x00\x00\x00\x00\x00"...}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 2536
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110379, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] sendto(10, {{len=20, type=RTM_GETADDR, flags=NLM_F_REQUEST|NLM_F_DUMP, seq=1514110380, pid=0}, {ifa_family=AF_UNSPEC, ...}}, 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=76, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110380, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 127.0.0.1}, {{nla_len=8, nla_type=IFA_LOCAL}, 127.0.0.1}, {{nla_len=7, nla_type=IFA_LABEL}, "lo"}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=1909}}]}, {{len=88, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110380, pid=8500}, {ifa_family=AF_INET, ifa_prefixlen=27, ifa_flags=0, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("eth0")}, [{{nla_len=8, nla_type=IFA_ADDRESS}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_LOCAL}, 83.246.80.131}, {{nla_len=8, nla_type=IFA_BROADCAST}, 83.246.80.159}, {{nla_len=9, nla_type=IFA_LABEL}, "eth0"}, {{nla_len=8, nla_type=IFA_FLAGS}, 0}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=762048, ifa_valid=762048, cstamp=2290, tstamp=254192327}}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 164
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110380, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=128, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, ::1}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=144, tstamp=144}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}, {{len=72, type=RTM_NEWADDR, flags=NLM_F_MULTI, seq=1514110380, pid=8500}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("eth0")}, [{{nla_len=20, nla_type=IFA_ADDRESS}, fe80::7cdd:a4ff:fe4e:69b}, {{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1985, tstamp=1985}}, {{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT}]}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 144
[pid  8502] recvmsg(10, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1514110380, pid=8500}, 0}, iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20
[pid  8502] close(10)                   = 0
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643935, tv_nsec=972702646}) = 0
[pid  8502] sendto(6, ".\236\327B\0\0\0\0\0\0\0\2\0\1\206\240\0\0\0\2\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\2\10\256\300\0\0\0\1\0\0\0\0\0\0\0\0", 56, 0, {sa_family=AF_INET, sin_port=htons(111), sin_addr=inet_addr("127.0.0.1")}, 16) = 56
[pid  8502] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643935, tv_nsec=973285296}) = 0
[pid  8502] poll([{fd=6, events=POLLIN}], 1, 5000 <unfinished ...>
[pid  8501] <... nanosleep resumed> NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643936, tv_nsec=896322694}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, NULL) = 0
[pid  8501] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2643937, tv_nsec=896676446}) = 0
[pid  8501] nanosleep({tv_sec=1, tv_nsec=0}, ^Cstrace: Process 8500 detached

..aso....aso....aso....aso..


(@devs: I know you, you will do this on every update/upgrade the server does :D )

Requested Action: 

Move init to daemon startup time, as any ordenary service does.

Comment 1 customercare 2017-12-24 10:40:05 UTC
Verified on a second server, it stucks here: 

glusterd --xlator-option *.upgrade=on -N

Comment 2 Kaleb KEITHLEY 2017-12-24 14:22:14 UTC
Is this actually an update? I could be mean and close this as WORKSFORME because I don't get this ever when I do a fresh _installs_.

It would be helpful if you could run the install in, e.g., the C locale. While the majority of our devs are multi-lingual, German isn't common. (Hint, Gluster Inc. is/was a company in India.)

I speak some German, but would not claim to be fluent. It would be helpful even for me. Thanks

Comment 3 customercare 2017-12-24 14:30:21 UTC
No, it was an install, BUT.. maybe not a clean one. Those servers had have had gluster on it in 2016. Because of some nasty bugs, like the famous port iteration bug or the SSL bug, the "i take ages on an os upgrade" (bug), i removed the rpms in 2016. The data got removed later, so besides (maybe) /var/lib/glusterd/, there was no data left over. I created my test volumes from scratch again.

Comment 4 customercare 2017-12-24 14:30:44 UTC
BTW: Merry chrismas :D

Comment 5 Kaleb KEITHLEY 2018-01-10 20:49:51 UTC
I have set up two pairs of systems: one pair of f26, and one pair of f27.

I have tried multiple combinations of: 
 + install glusterfs-3.10.8, create+start a volume, stop volume, remove gluster, install glusterfs-3.10.9.
 + install glusterfs-3.10.8, create+start a volume, stop volume, remove gluster, install glusterfs-3.12.4.
 + etc.

I've done the second install with both nodes up and running, and also with one node shut off (in case glusterd was trying to communicate with the other node's glusterd.

I am not able to reproduce the long delay after installing glusterfs-server.

I'm changing the product and components to GlusterFS and glusterd in case those developers can think of anything that could be going on.

Comment 6 Shyamsundar 2018-06-20 18:27:12 UTC
This bug reported is against a version of Gluster that is no longer maintained (or has been EOL'd). See https://www.gluster.org/release-schedule/ for the versions currently maintained.

As a result this bug is being closed.

If the bug persists on a maintained version of gluster or against the mainline gluster repository, request that it be reopened and the Version field be marked appropriately.

Comment 7 Shyamsundar 2018-06-20 18:27:46 UTC
This bug reported is against a version of Gluster that is no longer maintained
(or has been EOL'd). See https://www.gluster.org/release-schedule/ for the
versions currently maintained.

As a result this bug is being closed.

If the bug persists on a maintained version of gluster or against the mainline
gluster repository, request that it be reopened and the Version field be marked
appropriately.