+++ This bug was initially created as a clone of Bug #1385525 +++ Description of problem: ======================= Getting the below warning messages in glusterd and volume mount logs continuously on SSL SETUP when one of the cluster node glusterd is down/node is down. Message frequency is very high and if cluster node is down for any known reason for days, it will consume lot of log storage. [2016-10-17 08:15:55.008898] W [socket.c:590:__socket_rwv] 0-management: readv on 10.70.43.190:24007 failed (No data available) [2016-10-17 08:15:55.009082] W [socket.c:590:__socket_rwv] 0-management: readv on 10.70.43.190:24007 failed (No data available) [2016-10-17 08:15:55.009289] W [socket.c:590:__socket_rwv] 0-management: readv on 10.70.43.190:24007 failed (No data available) [2016-10-17 08:15:55.009546] W [socket.c:590:__socket_rwv] 0-management: readv on 10.70.43.190:24007 failed (No data available) [2016-10-17 08:15:55.009736] W [socket.c:590:__socket_rwv] 0-management: readv on 10.70.43.190:24007 failed (No data available) Version-Release number of selected component (if applicable): ============================================================= glusterfs-3.8.4-2 How reproducible: ================= Always Steps to Reproduce: =================== 1. Have two node cluster 2. Create a SSL configuration. 3. Create and start any volume type 4. fuse mount the volume. 5. Stop glusterd on vol file server node/ shutdown the node 6. Check the glusterd logs in the peer nodes and the volume mount log for the above mentioned warning messages Actual results: =============== Continuous warning messages getting when one of the cluster node is down on SSL setup. Expected results: ================= There should be control on throwing the warning messages or it should not throw the warning messages. Additional info: ================ I used backup vol file server while mounting the volume then also issue remains same in the volume mount log. --- Additional comment from Red Hat Bugzilla Rules Engine on 2016-10-17 05:10:35 EDT --- This bug is automatically being proposed for the current release of Red Hat Gluster Storage 3 under active development, by setting the release flag 'rhgs‑3.2.0' to '?'. If this bug should be proposed for a different release, please manually change the proposed release flag. --- Additional comment from Byreddy on 2016-10-18 01:16:56 EDT --- This issue is not there in the last GA build. --- Additional comment from Mohit Agrawal on 2016-10-18 07:26:58 EDT --- Hi, At the time of start glusterd process it returns a port to the other peer node on which it will communicate with glusterd(24007).After stop glusterd on a node means one end point of port is closed and other point is still used by socket(glusterd 24007),glusterd(socket_poller) on other node call's continuously socket_event_poll_in that calls __socket_rwv to read data from socket unless buffer is not clear. Because other end point of socket is disconnected so it prints the message "0-management: readv on 10.65.7.252:24007 failed (No data available)". After changed the condition to log the message issue will resolve. Regards Mohit Agrawal
REVIEW: http://review.gluster.org/15677 (rpc/socket.c : Modify __socket_rwv code in case of ENODATA error code.) posted (#1) for review on master by MOHIT AGRAWAL (moagrawa)
REVIEW: http://review.gluster.org/15677 (rpc/socket.c : Modify __socket_rwv code in case of ENODATA error code.) posted (#2) for review on master by MOHIT AGRAWAL (moagrawa)
REVIEW: http://review.gluster.org/15677 (rpc/socket.c : Modify socket_poller code in case of ENODATA error code.) posted (#3) for review on master by MOHIT AGRAWAL (moagrawa)
COMMIT: http://review.gluster.org/15677 committed in master by Raghavendra G (rgowdapp) ------ commit ec64ce2e1684003f4e7a20d4372e414bfbddb6fb Author: Mohit Agrawal <moagrawa> Date: Wed Oct 19 07:27:40 2016 +0530 rpc/socket.c : Modify socket_poller code in case of ENODATA error code. Problem: Continuous warning message(ENODATA) are coming in socket_rwv while SSL is enabled. Solution: To avoid the warning message update one condition in socket_poller loop code before break from loop in case of error returned by poll functions. BUG: 1386450 Change-Id: I19b3a92d4c3ba380738379f5679c1c354f0ab9b1 Signed-off-by: Mohit Agrawal <moagrawa> Reviewed-on: http://review.gluster.org/15677 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Raghavendra G <rgowdapp>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report. glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html [2] https://www.gluster.org/pipermail/gluster-users/