Description of problem: Log mknod failure due to EEXIST in DEBUG level in protocol/client. Without this, there would be excessive logging of EEXIST errors from protocol/client,server in logs of sharded volumes when multiple attempts are made from the same mount to create /.shard or the individual block shards as part of inode write fops. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Here is an excerpt from the logs when dd is executed on the mount (which possibly writes through multiple threads): <client-logs> ... ... [2015-05-15 09:19:21.890807] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.890958] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.891049] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.891148] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.893121] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.893277] W [client-rpc-fops.c:308:client3_3_mkdir_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard [2015-05-15 09:19:21.893375] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:21.893502] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:21.893575] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:21.893703] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:21.893820] I [MSGID: 109036] [dht-common.c:6692:dht_log_new_layout_for_dir_selfheal] 0-dis-dht: Setting layout of /.shard with [Subvol_name: dis-client-0, Err: -1 , Start: 0 , Stop: 4294967295 , Hash: 1 ], [2015-05-15 09:19:21.895055] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:21.895924] I [dht-layout.c:697:dht_layout_normalize] 0-dis-dht: Found anomalies in /.shard (gfid = 00000000-0000-0000-0000-000000000000). Holes=1 overlaps=0 [2015-05-15 09:19:22.175998] W [client-rpc-fops.c:241:client3_3_mknod_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard/72c4bc4a-ff12-4e2f-b231-992a64cdde82.1 [2015-05-15 09:19:22.176119] W [client-rpc-fops.c:241:client3_3_mknod_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard/72c4bc4a-ff12-4e2f-b231-992a64cdde82.1 [2015-05-15 09:19:22.451790] W [client-rpc-fops.c:241:client3_3_mknod_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard/72c4bc4a-ff12-4e2f-b231-992a64cdde82.2 [2015-05-15 09:19:22.451908] W [client-rpc-fops.c:241:client3_3_mknod_cbk] 0-dis-client-0: remote operation failed: File exists. Path: /.shard/72c4bc4a-ff12-4e2f-b231-992a64cdde82.2 ... ... </client-logs> <brick-logs> ... ... [2015-05-15 09:19:21.890271] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 31: MKDIR /.shard (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:21.890662] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 32: MKDIR (null) (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:21.890832] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 33: MKDIR (null) (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:21.891038] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 36: MKDIR (null) (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:21.892857] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 39: MKDIR (null) (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:21.893040] I [MSGID: 115056] [server-rpc-fops.c:487:server_mkdir_cbk] 0-dis-server: 42: MKDIR (null) (00000000-0000-0000-0000-000000000001/.shard) ==> (File exists) [File exists] [2015-05-15 09:19:22.175831] I [MSGID: 115057] [server-rpc-fops.c:535:server_mknod_cbk] 0-dis-server: 176: MKNOD (null) (be318638-e8a0-4c6d-977d-7a937aa84806/72c4bc4a-ff12-4e2f-b231-992a64cdde82.1) ==> (File exists) [File exists] [2015-05-15 09:19:22.175966] I [MSGID: 115057] [server-rpc-fops.c:535:server_mknod_cbk] 0-dis-server: 177: MKNOD (null) (be318638-e8a0-4c6d-977d-7a937aa84806/72c4bc4a-ff12-4e2f-b231-992a64cdde82.1) ==> (File exists) [File exists] [2015-05-15 09:19:22.451713] I [MSGID: 115057] [server-rpc-fops.c:535:server_mknod_cbk] 0-dis-server: 277: MKNOD (null) (be318638-e8a0-4c6d-977d-7a937aa84806/72c4bc4a-ff12-4e2f-b231-992a64cdde82.2) ==> (File exists) [File exists] [2015-05-15 09:19:22.451822] I [MSGID: 115057] [server-rpc-fops.c:535:server_mknod_cbk] 0-dis-server: 278: MKNOD (null) (be318638-e8a0-4c6d-977d-7a937aa84806/72c4bc4a-ff12-4e2f-b231-992a64cdde82.2) ==> (File exists) [File exists] ... ... </brick-logs>
Patch posted at http://review.gluster.org/#/c/10791/
COMMIT: http://review.gluster.org/10791 committed in master by Pranith Kumar Karampuri (pkarampu) ------ commit f92a415dd4fdd34c97fa6af4af528e1b50a4265f Author: Krutika Dhananjay <kdhananj> Date: Wed May 13 14:25:44 2015 +0200 protocol/client,server: Move EEXIST logs in mkdir and mknod to DEBUG level Change-Id: I0481258de8da36cbee7c046f53b20359badaf064 BUG: 1221889 Signed-off-by: Krutika Dhananjay <kdhananj> Reviewed-on: http://review.gluster.org/10791 Tested-by: Gluster Build System <jenkins.com> Tested-by: NetBSD Build System Reviewed-by: Pranith Kumar Karampuri <pkarampu>
Fix for this BZ is already present in a GlusterFS release. You can find clone of this BZ, fixed in a GlusterFS release and closed. Hence closing this mainline BZ as well.
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report. glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/ [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user