Bug 1142402

Summary:	DHT: nfs.log getting filled with "I" logs
Product:	[Community] GlusterFS	Reporter:	Shyamsundar <srangana>
Component:	distribute	Assignee:	Shyamsundar <srangana>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.6.0	CC:	bugs, gluster-bugs, nbalacha, saujain, ssamanta, vsomyaju
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.6.0beta1	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1138602	Environment:
Last Closed:	2014-11-11 08:38:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1138288, 1138602, 1206120
Bug Blocks:	1117822

Description Shyamsundar 2014-09-16 17:27:02 UTC

+++ This bug was initially created as a clone of Bug #1138602 +++

+++ This bug was initially created as a clone of Bug #1138288 +++

Description of problem:

nfs.log getting filled with these logs,

[2014-09-04 01:40:27.517380] I [dht-common.c:1828:dht_lookup_cbk] 0-dist-rep-dht: Entry /run16165/fileop_L1_8/fileop_L1_8_L2_14/fileop_dir_8_14_29 missing on subvol dist-rep-replicate-5
[2014-09-04 01:40:27.520240] I [dht-common.c:1085:dht_lookup_everywhere_done] 0-dist-rep-dht: STATUS: hashed_subvol dist-rep-replicate-5 cached_subvol null
[2014-09-04 01:40:27.533708] I [dht-common.c:1828:dht_lookup_cbk] 0-dist-rep-dht: Entry /run16165/fileop_L1_8/fileop_L1_8_L2_15 missing on subvol dist-rep-replicate-2
[2014-09-04 01:40:27.535634] I [dht-common.c:1085:dht_lookup_everywhere_done] 0-dist-rep-dht: STATUS: hashed_subvol dist-rep-replicate-2 cached_subvol null
[2014-09-04 01:40:27.550821] I [dht-common.c:1828:dht_lookup_cbk] 0-dist-rep-dht: Entry /run16165/fileop_L1_8/fileop_L1_8_L2_15/fileop_dir_8_15_0 missing on subvol dist-rep-replicate-5
[2014-09-04 01:40:27.553221] I [dht-common.c:1085:dht_lookup_everywhere_done] 0-dist-rep-dht: STATUS: hashed_subvol dist-rep-replicate-5 cached_subvol null
[2014-09-04 01:40:27.575100] I [dht-common.c:1828:dht_lookup_cbk] 0-dist-rep-dht: Entry /run16165/fileop_L1_8/fileop_L1_8_L2_15/fileop_dir_8_15_1 missing on subvol dist-rep-replicate-0
[2014-09-04 01:40:27.576977] I [dht-common.c:1085:dht_lookup_everywhere_done] 0-dist-rep-dht: STATUS: hashed_subvol dist-rep-replicate-0 cached_subvol null

How reproducible:
happen to be regularly seen on the above mentioned build

Steps to Reproduce:
1. create a 6x2 volume
2. start the volume
3. mount the volume over nfs
4. start untar of kernel on mount-point

Actual results:
nfs.log getting filled with the above mentioned messages

Expected results:
Not expecting so many messages

Additional info:

--- Additional comment from Anand Avati on 2014-09-05 05:48:45 EDT ---

REVIEW: http://review.gluster.org/8621 (cluster/dht: Changed log level to DEBUG) posted (#1) for review on master by venkatesh somyajulu (vsomyaju)

--- Additional comment from Anand Avati on 2014-09-09 03:03:34 EDT ---

REVIEW: http://review.gluster.org/8621 (cluster/dht: Changed log level to DEBUG) posted (#2) for review on master by venkatesh somyajulu (vsomyaju)

--- Additional comment from Anand Avati on 2014-09-11 06:17:55 EDT ---

COMMIT: http://review.gluster.org/8621 committed in master by Vijay Bellur (vbellur) 
------
commit 976ecc99b4716f9189f0affdcba0ff8ea8d09d9e
Author: Venkatesh Somyajulu <vsomyaju>
Date:   Tue Sep 9 12:32:20 2014 +0530

    cluster/dht: Changed log level to DEBUG
    
    Change-Id: I7a4ee0c5a6a94bd4f31aff510a2971750913ed45
    BUG: 1138602
    Signed-off-by: Venkatesh Somyajulu <vsomyaju>
    Reviewed-on: http://review.gluster.org/8621
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Xavier Hernandez <xhernandez>
    Reviewed-by: susant palai <spalai>
    Reviewed-by: Vijay Bellur <vbellur>

--- Additional comment from Anand Avati on 2014-09-11 08:03:43 EDT ---

REVIEW: http://review.gluster.org/8697 (cluster/dht: Changed log level to DEBUG) posted (#1) for review on master by venkatesh somyajulu (vsomyaju)

--- Additional comment from Anand Avati on 2014-09-16 05:32:16 EDT ---

COMMIT: http://review.gluster.org/8697 committed in master by Vijay Bellur (vbellur) 
------
commit c7437f35df1127c05a4df0f7bf3ae95b71b7831e
Author: Venkatesh Somyajulu <vsomyaju>
Date:   Thu Sep 11 17:31:59 2014 +0530

    cluster/dht: Changed log level to DEBUG
    
    Change-Id: Ia4dde95367b44d63f57f0840c2a2f351b1cfb072
    BUG: 1138602
    Signed-off-by: Venkatesh Somyajulu <vsomyaju>
    Reviewed-on: http://review.gluster.org/8697
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 1 Anand Avati 2014-09-16 17:54:04 UTC

REVIEW: http://review.gluster.org/8749 (cluster/dht: Changed log level to DEBUG) posted (#1) for review on release-3.6 by Shyamsundar Ranganathan (srangana)

Comment 2 Anand Avati 2014-09-18 17:50:16 UTC

COMMIT: http://review.gluster.org/8749 committed in release-3.6 by Vijay Bellur (vbellur) 
------
commit 488a073752a467e737f40034427eb749626549ce
Author: Venkatesh Somyajulu <vsomyaju>
Date:   Tue Sep 16 13:52:54 2014 -0400

    cluster/dht: Changed log level to DEBUG
    
    Change-Id: I7a4ee0c5a6a94bd4f31aff510a2971750913ed45
    BUG: 1142402
    Signed-off-by: Venkatesh Somyajulu <vsomyaju>
    Reviewed-on-master: http://review.gluster.org/8621
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Xavier Hernandez <xhernandez>
    Reviewed-by: susant palai <spalai>
    Reviewed-by: Vijay Bellur <vbellur>
    Reviewed-on: http://review.gluster.org/8749

Comment 3 Niels de Vos 2014-09-22 12:46:19 UTC

A beta release for GlusterFS 3.6.0 has been released. Please verify if the release solves this bug report for you. In case the glusterfs-3.6.0beta1 release does not have a resolution for this issue, leave a comment in this bug and move the status to ASSIGNED. If this release fixes the problem for you, leave a note and change the status to VERIFIED.

Packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update (possibly an "updates-testing" repository) infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-September/018836.html
[2] http://supercolony.gluster.org/pipermail/gluster-users/

Comment 4 Niels de Vos 2014-11-11 08:38:58 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users