Bug 1611546

Summary:	Log file glustershd.log being filled with errors
Product:	[Community] GlusterFS	Reporter:	tranthienthanh <minhdanh>
Component:	rpc	Assignee:	Rinku <rkothiya>
Status:	CLOSED UPSTREAM	QA Contact:
Severity:	medium	Docs Contact:
Priority:	low
Version:	mainline	CC:	bugs, pasik
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-12 12:56:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description tranthienthanh 2018-08-02 10:25:53 UTC

Description of problem:

Log file `glustershd.log` is being filled with this error:

```
[2018-08-02 10:13:13.210769] W [rpc-clnt.c:1739:rpc_clnt_submit] 4-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
[2018-08-02 10:13:13.211105] W [rpc-clnt.c:1739:rpc_clnt_submit] 4-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
[2018-08-02 10:13:13.219039] W [rpc-clnt.c:1739:rpc_clnt_submit] 2-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
[2018-08-02 10:13:13.219345] W [rpc-clnt.c:1739:rpc_clnt_submit] 2-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
[2018-08-02 10:13:13.226175] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
[2018-08-02 10:13:13.226456] W [rpc-clnt.c:1739:rpc_clnt_submit] 0-vol_d5fd2148f6bf0348a44506763571109e-client-2: error returned while attempting to connect to host:(null), port:0
```

The log file size increased to 10GB within about 2 days it this is causing disk space shortage on my servers.


Version-Release number of selected component (if applicable): glusterfs 3.12.10



How reproducible:


Steps to Reproduce:

Not sure how to reproduce this.
We have a cluster of 4 nodes of glusterfs (version 4.0.2) running on Kubernetes.
The glusterfs client version is 3.12.
This happens on some nodes that have a container mounts a glusterfs volume.

Actual results:

This is causing disk space shortage on my servers.

Expected results:

How to avoid this error or at least reduce the number of errors logged?

Additional info:

According to this: https://github.com/gluster/glusterd2/issues/940#issuecomment-408038184

the errors seem to be expected behaviors, but too much logs is a problem.

Comment 1 Shyamsundar 2018-10-23 14:53:57 UTC

Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.

Comment 3 Worker Ant 2020-03-12 12:56:13 UTC

This bug is moved to https://github.com/gluster/glusterfs/issues/970, and will be tracked there from now on. Visit GitHub issues URL for further details