Bug 1503244 - socket poller error in glusterd logs
Summary: socket poller error in glusterd logs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterd
Version: rhgs-3.3
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: RHGS 3.4.0
Assignee: Mohit Agrawal
QA Contact: Rajesh Madaka
URL:
Whiteboard:
: 1301460 (view as bug list)
Depends On: 1503246 1507747 1507749
Blocks: 1503134 1526371 1530512
TreeView+ depends on / blocked
 
Reported: 2017-10-17 15:56 UTC by SATHEESARAN
Modified: 2019-04-27 01:31 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1526371 (view as bug list)
Environment:
Last Closed: 2018-09-04 06:38:02 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 None None None 2018-09-04 06:39:45 UTC

Description SATHEESARAN 2017-10-17 15:56:59 UTC
Description of problem:
-----------------------
With RHHI setup of 3 node gluster cluster in place with SSL/TLS encryption enabled, observed continuous error messages in glusterd logs in one of the node as follows:
[2017-10-13 13:01:35.114409] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53419 failed (Input/output error)

Version-Release number of selected component (if applicable):
--------------------------------------------------------------
glusterfs-3.8.4-45.el7rhgs

How reproducible:
-----------------
Just observed it once, haven't tried to reproduce it

Steps to Reproduce:
-------------------
1. Create a 3 node gluster cluster with RHGS 3.3.0 with TLS/SSL encryption enabled
2. Update the cluster to RHGS 3.3.1 interim build
3. After sometime, observe the glusterd logs

Actual results:
---------------
Socket poller errors seen with gluserd logs


Expected results:
-----------------
No errors in glusterd logs

Additional info:

Comment 1 SATHEESARAN 2017-10-17 15:57:34 UTC
I see such a continuous error messages logged only in one node.

[2017-10-13 13:01:35.114409] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53419 failed (Input/output error)
[2017-10-13 13:01:43.127455] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53528 failed (Input/output error)
[2017-10-13 13:01:53.245466] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53646 failed (Input/output error)
[2017-10-13 13:02:01.992096] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53754 failed (Input/output error)
[2017-10-13 13:02:10.152669] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.73:1007 failed (Input/output error)
[2017-10-13 13:02:19.175127] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:53950 failed (Input/output error)
[2017-10-13 13:02:27.191875] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.74:54054 failed (Input/output error)
[2017-10-13 13:02:37.723957] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.73:58632 failed (Input/output error)
[2017-10-13 13:02:46.043035] E [socket.c:2631:socket_poller] 0-socket.management: socket_poller 10.70.36.75:59950 failed (Input/output error)

Comment 2 Atin Mukherjee 2017-10-17 16:11:46 UTC
Thanks to Mohit for the analysis. This had happened because of glusterd reporting back a stale port.

An upstream patch https://review.gluster.org/#/c/18541 is posted for review.

Comment 5 Raghavendra G 2017-12-01 04:19:57 UTC
*** Bug 1301460 has been marked as a duplicate of this bug. ***

Comment 12 Rajesh Madaka 2018-05-15 17:49:33 UTC
Followed steps mentioned in above desc.

i have found only one message of socket_poller error after brick restart in glusterd.log file.

socket_poller errors are not generating continuously in glusterd.log file.

Verified in below verison:


glusterfs-server-3.12.2-9

Comment 14 errata-xmlrpc 2018-09-04 06:38:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607


Note You need to log in before you can comment on or make changes to this bug.